2024-04-25T18:35:01.392 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T18:35:01.392 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --verbose --owner scheduled_leonidus@teuthology --name leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619 -- /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/orig.config.yaml 2024-04-25T18:35:01.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T18:35:01.544 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7673619' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 7797745e6d8b3ac036504dc61491d368615024da owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4419 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 7797745e6d8b3ac036504dc61491d368615024da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDnECq695Oy06KssWYFbiELkWY17v3e5TDsEVsbsJjjwu/7YvYcJF+E0fLVTHihXxQXo41yJMQkiwafl4jZNfE= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKi9oq7PQUTn5GbBL7S11VPstxddLx2pNB7hAsqwwcJjVL5d4MjvvaKmp3X4cuCTxaQEoTAMjDGpwx96YhDLTcM= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHk86Ti+sIRhj5VQfinQWmzbS8ebK+xnkS+dqIoZ162ZzgLN+/KV7jxMQBHMJIxj0SNF18mNwUIUGSP41h15LKc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_17:43:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T18:35:01.544 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa; will attempt to use it 2024-04-25T18:35:01.545 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks 2024-04-25T18:35:01.545 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T18:35:01.547 INFO:teuthology.task.internal:Checking packages... 2024-04-25T18:35:01.569 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T18:35:01.569 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T18:35:01.570 INFO:teuthology.packaging:ref: None 2024-04-25T18:35:01.570 INFO:teuthology.packaging:tag: None 2024-04-25T18:35:01.570 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T18:35:01.570 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:35:01.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T18:35:01.744 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T18:35:01.745 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T18:35:01.751 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T18:35:01.752 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T18:35:01.769 INFO:teuthology.task.internal:Saving configuration 2024-04-25T18:35:01.790 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T18:35:01.796 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T18:35:01.822 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:15.392764', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDnECq695Oy06KssWYFbiELkWY17v3e5TDsEVsbsJjjwu/7YvYcJF+E0fLVTHihXxQXo41yJMQkiwafl4jZNfE='} 2024-04-25T18:35:01.845 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:15.394664', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKi9oq7PQUTn5GbBL7S11VPstxddLx2pNB7hAsqwwcJjVL5d4MjvvaKmp3X4cuCTxaQEoTAMjDGpwx96YhDLTcM='} 2024-04-25T18:35:01.867 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:15.391099', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHk86Ti+sIRhj5VQfinQWmzbS8ebK+xnkS+dqIoZ162ZzgLN+/KV7jxMQBHMJIxj0SNF18mNwUIUGSP41h15LKc='} 2024-04-25T18:35:01.868 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T18:35:01.874 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T18:35:01.874 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T18:35:01.874 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T18:35:01.874 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T18:35:01.974 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f76861900d0>, signals=[15]) 2024-04-25T18:35:01.975 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T18:35:01.981 INFO:teuthology.task.internal:Opening connections... 2024-04-25T18:35:01.981 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T18:35:01.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:35:02.061 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-04-25T18:35:02.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:35:02.137 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-04-25T18:35:02.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:35:02.211 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T18:35:02.219 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-25T18:35:02.236 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-25T18:35:02.236 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-25T18:35:02.293 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:35:02.294 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:35:02.295 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:35:02.295 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-25T18:35:02.321 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-04-25T18:35:02.339 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-04-25T18:35:02.339 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-04-25T18:35:02.396 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:35:02.397 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:35:02.397 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:35:02.397 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:35:02.397 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:35:02.397 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:35:02.397 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-25T18:35:02.429 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-04-25T18:35:02.447 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-04-25T18:35:02.447 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-04-25T18:35:02.503 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-04-25T18:35:02.503 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="9" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="9" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:35:02.504 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:35:02.505 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:35:02.505 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:35:02.505 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:35:02.505 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-04-25T18:35:02.528 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T18:35:02.539 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T18:35:02.545 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T18:35:02.545 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:35:02.548 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:35:02.550 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:35:02.566 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T18:35:02.572 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T18:35:02.572 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:35:02.604 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:35:02.607 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:35:02.719 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T18:35:02.794 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T18:35:02.808 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T18:35:02.808 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T18:35:02.808 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T18:35:02.808 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:35:02.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:35:02.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T18:35:03.071 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:35:03.071 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T18:35:03.071 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-25T18:35:03.088 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-437.el9.x86_64 2024-04-25T18:35:03.088 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:35:03.089 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T18:35:03.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-04-25T18:35:05.431 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T18:35:07.641 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T18:35:11.916 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-25T18:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-25T18:35:17.232 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-04-25T18:35:22.240 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T18:35:24.205 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-25T18:35:24.206 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-04-25T18:35:24.206 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-25T18:35:24.306 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:35:24.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:35:24.610 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T18:35:24.611 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T18:35:24.613 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-25T18:36:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25 18:36:02 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T18:36:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-04-25 18:36:02-- 2024-04-25T18:36:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 38s 2024-04-25T18:36:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 341M in 38s (9.08 MB/s) 2024-04-25T18:36:02.531 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:36:02.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T18:36:04.626 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-04-25T18:36:04.862 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-04-25T18:36:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-04-25T18:36:12.943 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T18:36:49.733 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T18:36:49.733 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T18:36:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T18:36:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T18:36:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T18:36:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T18:36:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T18:36:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T18:36:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T18:36:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T18:36:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T18:36:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T18:36:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T18:36:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T18:36:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T18:36:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T18:36:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T18:36:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T18:36:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T18:36:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T18:36:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T18:36:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T18:36:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T18:36:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T18:36:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T18:36:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T18:36:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T18:36:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T18:36:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T18:36:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T18:36:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T18:36:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T18:36:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T18:36:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T18:36:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T18:36:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T18:36:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T18:36:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T18:36:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T18:36:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T18:36:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T18:36:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T18:36:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T18:36:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T18:36:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T18:36:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T18:36:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T18:36:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T18:36:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T18:36:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T18:36:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T18:36:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T18:36:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T18:36:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T18:36:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T18:36:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T18:36:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T18:36:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T18:36:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T18:36:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T18:36:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T18:36:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T18:36:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T18:36:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T18:36:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T18:36:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T18:36:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T18:36:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T18:36:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T18:36:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T18:36:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T18:36:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T18:36:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T18:36:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T18:36:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T18:36:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T18:36:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T18:36:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T18:36:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T18:36:49.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T18:36:49.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T18:36:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T18:36:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T18:36:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T18:36:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T18:36:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T18:36:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T18:36:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T18:36:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T18:36:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T18:36:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T18:36:49.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T18:36:49.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T18:36:49.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T18:36:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T18:36:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T18:36:49.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T18:36:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T18:36:49.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T18:36:49.908 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T18:36:49.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T18:36:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T18:36:49.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T18:36:49.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T18:36:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T18:36:49.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T18:36:49.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T18:36:49.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T18:36:49.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T18:36:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T18:36:49.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T18:36:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T18:36:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T18:36:49.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T18:36:49.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T18:36:49.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T18:36:49.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T18:36:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T18:36:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T18:36:49.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T18:36:49.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T18:36:49.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T18:36:49.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T18:36:49.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T18:36:49.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T18:36:49.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T18:36:49.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T18:36:49.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T18:36:49.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T18:36:49.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T18:36:49.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T18:36:49.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T18:36:49.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T18:36:49.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T18:36:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T18:36:49.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T18:36:49.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T18:36:49.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T18:36:49.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T18:36:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:36:49.952 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:49.952 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T18:36:49.974 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:36:49.975 INFO:teuthology.orchestra.run.smithi086.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-25T18:36:49.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-04-25T18:36:50.061 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T18:36:50.062 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:36:50.063 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T18:36:50.064 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T18:36:50.064 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:50.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T18:36:50.557 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-25T18:36:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T18:36:52.037 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-25T18:36:52.040 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-25T18:36:52.054 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.9Yw5VrtA6h 2024-04-25T18:36:52.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9Yw5VrtA6h 2024-04-25T18:36:52.136 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.9Yw5VrtA6h 2024-04-25T18:36:52.270 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.9Yw5VrtA6h is 6KB 2024-04-25T18:36:52.289 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.9Yw5VrtA6h 2024-04-25T18:36:52.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T18:36:52.370 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T18:36:52.370 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T18:36:52.370 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T18:36:52.370 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T18:36:52.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T18:36:52.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-04-25T18:37:22.521 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T18:37:22.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:37:22.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:37:41.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-25T18:37:50.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:37:50.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:37:53.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-25T18:38:05.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:38:05.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:38:23.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-25T18:38:38.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:38:38.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:38:38.896 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-25T18:38:39.525 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:38:39.525 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T18:38:39.525 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-25T18:38:39.561 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T18:38:39.561 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T18:38:39.561 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T18:38:39.561 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T18:38:40.563 DEBUG:teuthology.parallel:result is None 2024-04-25T18:38:40.563 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T18:38:40.571 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T18:38:40.571 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:38:40.576 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:38:40.579 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:38:40.618 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T18:38:40.626 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T18:38:40.641 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T18:38:40.641 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:38:40.666 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:38:40.669 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:38:40.822 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T18:38:40.830 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T18:38:40.831 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:38:40.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:38:40.847 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:38:40.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:38:40.864 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:38:40.868 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:38:40.890 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:38:40.923 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:38:40.938 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:38:40.943 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:38:40.970 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:38:40.982 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:38:41.036 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:38:41.040 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T18:38:41.048 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T18:38:41.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:38:41.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:38:41.058 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:38:41.120 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T18:38:41.130 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T18:38:41.130 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:38:41.134 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:38:41.137 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:38:41.172 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:38:41.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:38:41.365 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:38:41.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:38:41.513 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:38:41.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:38:41.608 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:38:41.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:38:41.703 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:38:41.788 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:38:41.829 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:38:41.829 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:38:41.895 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:38:41.940 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:38:41.996 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:38:42.073 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:38:42.121 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:38:42.121 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:38:42.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-25T18:38:42.194 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-04-25T18:38:42.196 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-04-25T18:38:42.230 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:38:42.263 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:38:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:38:42.875 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T18:38:42.884 INFO:teuthology.task.internal:Starting timer... 2024-04-25T18:38:42.884 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T18:38:42.899 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T18:38:42.910 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T18:38:42.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-04-25T18:38:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-04-25T18:38:43.033 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-04-25T18:38:43.102 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-04-25T18:38:43.104 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-04-25T18:38:43.182 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-04-25T18:38:43.184 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T18:38:43.184 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-04-25T18:38:43.236 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-04-25T18:38:43.236 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-04-25T18:38:43.284 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-04-25T18:38:43.284 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-04-25T18:38:43.308 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-04-25T18:38:43.309 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2024-04-25T18:38:43.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:38:43.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:38:43.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:38:43.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:38:43.402 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:38:43.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:38:43.429 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T18:38:43.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:38:43.558 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:38:43.582 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:38:43.609 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T18:38:43.618 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T18:38:43.632 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T18:38:43.633 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T18:43:44.677 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-04-25T18:43:44.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:43:44.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:43:44.786 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-25T18:43:44.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T18:43:44.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-25T18:43:44.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:43:45.022 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-25T18:43:45.115 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-25T18:43:45.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-25T18:43:45.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:43:45.190 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-04-25T18:43:45.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-25T18:43:45.266 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T18:43:45.276 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T18:43:45.276 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:43:45.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:43:45.284 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:43:45.284 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:43:45.286 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:43:45.286 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:43:45.316 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:43:45.331 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:43:45.334 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:43:45.349 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:43:45.356 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-04-25T18:43:45.367 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-04-25T18:43:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:43:45.372 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-04-25T18:43:45.381 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:43:45.383 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-04-25T18:43:45.394 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:43:45.396 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:43:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:43:45.410 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:43:45.456 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:43:45.459 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:43:45.459 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-25T18:43:45.459 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.460 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.460 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.460 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.463 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:43:45.466 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:43:45.466 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-04-25T18:43:45.466 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.466 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.466 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.467 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.476 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-04-25T18:43:45.508 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-04-25T18:43:45.548 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:43:45.587 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:43:45.734 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:43:45.742 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:43:45.743 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-25T18:43:45.743 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.743 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.743 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.743 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:43:45.744 INFO:teuthology.run_tasks:Running task install... 2024-04-25T18:43:45.754 DEBUG:teuthology.task.install:project ceph 2024-04-25T18:43:45.754 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T18:43:45.754 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T18:43:45.755 INFO:teuthology.task.install:Using flavor: default 2024-04-25T18:43:45.760 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T18:43:45.760 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T18:43:45.761 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:43:45.761 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:43:45.763 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:43:45.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:43:45.764 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:43:45.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:43:45.905 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:43:45.905 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:43:45.985 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:43:45.985 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:43:45.989 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:43:45.989 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:43:46.089 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:43:46.089 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:43:46.090 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:43:46.096 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:43:46.096 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:43:46.096 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:43:46.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:43:46.124 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:43:46.160 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:43:46.161 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:43:46.161 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:43:46.178 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:43:46.178 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:43:46.187 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:43:46.187 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:43:46.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:43:46.253 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:43:46.268 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:43:46.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:43:46.329 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:43:46.332 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-04-25T18:43:46.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-04-25T18:43:46.393 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-04-25T18:43:46.395 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-04-25T18:43:46.415 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:43:46.496 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-04-25T18:43:46.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-04-25T18:43:46.561 INFO:teuthology.orchestra.run.smithi183.stdout:48 files removed 2024-04-25T18:43:46.588 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:43:46.620 INFO:teuthology.orchestra.run.smithi194.stdout:48 files removed 2024-04-25T18:43:46.648 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:43:46.805 INFO:teuthology.orchestra.run.smithi086.stdout:48 files removed 2024-04-25T18:43:46.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:43:47.217 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 269 kB/s | 83 kB 00:00 2024-04-25T18:43:47.314 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-04-25T18:43:47.584 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-25T18:43:47.739 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T18:43:47.759 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 172 kB/s | 83 kB 00:00 2024-04-25T18:43:47.993 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T18:43:48.149 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T18:43:48.327 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-25T18:43:48.734 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T18:43:48.777 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-25T18:43:48.845 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-04-25T18:43:49.455 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T18:43:50.622 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-25T18:43:50.910 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T18:43:51.557 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T18:43:54.764 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T18:43:55.077 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T18:43:56.176 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-25T18:43:56.404 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T18:43:56.759 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T18:43:57.746 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-25T18:43:57.854 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-25T18:43:59.047 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-04-25T18:43:59.902 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T18:44:02.746 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T18:44:04.221 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T18:44:04.905 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:44:04.906 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:44:04.917 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:44:04.917 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:44:04.918 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:44:04.919 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:44:04.921 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:44:04.923 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:44:04.924 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:44:04.985 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:44:05.002 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 4 Packages 2024-04-25T18:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:44:05.016 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 222 M 2024-04-25T18:44:05.016 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-04-25T18:44:05.755 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T18:44:05.930 INFO:teuthology.orchestra.run.smithi183.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T18:44:06.364 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:44:06.364 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:44:06.375 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:44:06.375 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:44:06.376 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:44:06.377 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:44:06.379 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:44:06.381 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:44:06.382 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:44:06.441 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout:======================================================================================= 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout:======================================================================================= 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:44:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:44:06.462 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:44:06.463 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:44:06.464 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:44:06.465 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:44:06.466 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:44:06.467 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:44:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:44:06.469 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:44:06.470 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:44:06.471 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:44:06.472 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout:======================================================================================= 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout:Install 148 Packages 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 4 Packages 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:44:06.473 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 222 M 2024-04-25T18:44:06.474 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-25T18:44:06.882 INFO:teuthology.orchestra.run.smithi183.stdout:(2/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-25T18:44:06.964 INFO:teuthology.orchestra.run.smithi183.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-25T18:44:07.105 INFO:teuthology.orchestra.run.smithi183.stdout:(4/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 757 kB/s | 890 kB 00:01 2024-04-25T18:44:07.372 INFO:teuthology.orchestra.run.smithi183.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 5.8 MB/s | 2.4 MB 00:00 2024-04-25T18:44:07.823 INFO:teuthology.orchestra.run.smithi194.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T18:44:08.114 INFO:teuthology.orchestra.run.smithi183.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 6.3 MB/s | 4.7 MB 00:00 2024-04-25T18:44:08.156 INFO:teuthology.orchestra.run.smithi183.stdout:(7/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 1.5 MB 00:01 2024-04-25T18:44:08.168 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:44:08.170 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:44:08.191 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:44:08.192 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:44:08.193 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:44:08.195 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:44:08.197 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:44:08.200 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:44:08.201 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:44:08.272 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:44:08.301 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:44:08.302 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:44:08.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:44:08.304 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:44:08.305 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:44:08.306 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:44:08.307 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:44:08.308 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:44:08.309 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:44:08.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:44:08.311 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:44:08.312 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 4 Packages 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:44:08.313 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 222 M 2024-04-25T18:44:08.314 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-25T18:44:09.089 INFO:teuthology.orchestra.run.smithi194.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 703 kB/s | 890 kB 00:01 2024-04-25T18:44:09.306 INFO:teuthology.orchestra.run.smithi194.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 668 kB/s | 145 kB 00:00 2024-04-25T18:44:09.373 INFO:teuthology.orchestra.run.smithi194.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.7 MB/s | 5.4 MB 00:01 2024-04-25T18:44:09.857 INFO:teuthology.orchestra.run.smithi194.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T18:44:10.148 INFO:teuthology.orchestra.run.smithi086.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-25T18:44:11.382 INFO:teuthology.orchestra.run.smithi086.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 722 kB/s | 890 kB 00:01 2024-04-25T18:44:11.540 INFO:teuthology.orchestra.run.smithi194.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-25T18:44:11.557 INFO:teuthology.orchestra.run.smithi086.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 832 kB/s | 145 kB 00:00 2024-04-25T18:44:11.698 INFO:teuthology.orchestra.run.smithi183.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 3.0 MB/s | 11 MB 00:03 2024-04-25T18:44:11.856 INFO:teuthology.orchestra.run.smithi183.stdout:(9/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 4.5 MB/s | 17 MB 00:03 2024-04-25T18:44:11.881 INFO:teuthology.orchestra.run.smithi183.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 137 kB/s | 25 kB 00:00 2024-04-25T18:44:11.924 INFO:teuthology.orchestra.run.smithi194.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.3 MB/s | 4.7 MB 00:02 2024-04-25T18:44:11.982 INFO:teuthology.orchestra.run.smithi183.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-25T18:44:12.090 INFO:teuthology.orchestra.run.smithi183.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 249 kB/s | 27 kB 00:00 2024-04-25T18:44:12.199 INFO:teuthology.orchestra.run.smithi183.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 293 kB/s | 32 kB 00:00 2024-04-25T18:44:12.516 INFO:teuthology.orchestra.run.smithi183.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 747 kB 00:00 2024-04-25T18:44:12.633 INFO:teuthology.orchestra.run.smithi183.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-25T18:44:12.750 INFO:teuthology.orchestra.run.smithi183.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.1 MB/s | 126 kB 00:00 2024-04-25T18:44:12.832 INFO:teuthology.orchestra.run.smithi086.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-25T18:44:13.026 INFO:teuthology.orchestra.run.smithi183.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.8 MB/s | 500 kB 00:00 2024-04-25T18:44:13.432 INFO:teuthology.orchestra.run.smithi086.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 1.5 MB 00:00 2024-04-25T18:44:14.733 INFO:teuthology.orchestra.run.smithi086.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 4.7 MB 00:01 2024-04-25T18:44:15.484 INFO:teuthology.orchestra.run.smithi183.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 5.4 MB 00:02 2024-04-25T18:44:15.601 INFO:teuthology.orchestra.run.smithi183.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 387 kB/s | 45 kB 00:00 2024-04-25T18:44:15.726 INFO:teuthology.orchestra.run.smithi183.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.1 MB/s | 135 kB 00:00 2024-04-25T18:44:15.968 INFO:teuthology.orchestra.run.smithi183.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 668 kB/s | 161 kB 00:00 2024-04-25T18:44:16.327 INFO:teuthology.orchestra.run.smithi183.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 895 kB/s | 320 kB 00:00 2024-04-25T18:44:16.585 INFO:teuthology.orchestra.run.smithi183.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 298 kB 00:00 2024-04-25T18:44:16.874 INFO:teuthology.orchestra.run.smithi086.stdout:(7/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 7.9 MB/s | 17 MB 00:02 2024-04-25T18:44:16.927 INFO:teuthology.orchestra.run.smithi183.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 291 kB/s | 99 kB 00:00 2024-04-25T18:44:17.053 INFO:teuthology.orchestra.run.smithi183.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 682 kB/s | 85 kB 00:00 2024-04-25T18:44:17.865 INFO:teuthology.orchestra.run.smithi194.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 11 MB 00:05 2024-04-25T18:44:17.982 INFO:teuthology.orchestra.run.smithi194.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 216 kB/s | 25 kB 00:00 2024-04-25T18:44:18.600 INFO:teuthology.orchestra.run.smithi086.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 6.1 MB/s | 11 MB 00:01 2024-04-25T18:44:18.708 INFO:teuthology.orchestra.run.smithi086.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 235 kB/s | 25 kB 00:00 2024-04-25T18:44:19.036 INFO:teuthology.orchestra.run.smithi086.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.2 MB/s | 21 MB 00:09 2024-04-25T18:44:19.409 INFO:teuthology.orchestra.run.smithi086.stdout:(11/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x 563 kB/s | 5.4 MB 00:09 2024-04-25T18:44:19.442 INFO:teuthology.orchestra.run.smithi086.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 556 kB/s | 225 kB 00:00 2024-04-25T18:44:19.460 INFO:teuthology.orchestra.run.smithi183.stdout:(26/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.5 MB/s | 21 MB 00:13 2024-04-25T18:44:19.509 INFO:teuthology.orchestra.run.smithi086.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 273 kB/s | 27 kB 00:00 2024-04-25T18:44:19.519 INFO:teuthology.orchestra.run.smithi183.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 1.3 MB/s | 3.1 MB 00:02 2024-04-25T18:44:19.543 INFO:teuthology.orchestra.run.smithi086.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 318 kB/s | 32 kB 00:00 2024-04-25T18:44:19.578 INFO:teuthology.orchestra.run.smithi183.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 170 kB 00:00 2024-04-25T18:44:19.645 INFO:teuthology.orchestra.run.smithi183.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 272 kB/s | 34 kB 00:00 2024-04-25T18:44:19.711 INFO:teuthology.orchestra.run.smithi183.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.0 MB/s | 142 kB 00:00 2024-04-25T18:44:19.751 INFO:teuthology.orchestra.run.smithi086.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 785 kB/s | 163 kB 00:00 2024-04-25T18:44:20.021 INFO:teuthology.orchestra.run.smithi086.stdout:(16/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.4 MB/s | 747 kB 00:00 2024-04-25T18:44:20.293 INFO:teuthology.orchestra.run.smithi086.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.8 MB/s | 500 kB 00:00 2024-04-25T18:44:20.368 INFO:teuthology.orchestra.run.smithi086.stdout:(18/152): librados-devel-19.0.0-3156.gd69f13d8. 205 kB/s | 126 kB 00:00 2024-04-25T18:44:20.485 INFO:teuthology.orchestra.run.smithi086.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 388 kB/s | 45 kB 00:00 2024-04-25T18:44:20.677 INFO:teuthology.orchestra.run.smithi086.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 706 kB/s | 135 kB 00:00 2024-04-25T18:44:20.886 INFO:teuthology.orchestra.run.smithi086.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 777 kB/s | 161 kB 00:00 2024-04-25T18:44:21.057 INFO:teuthology.orchestra.run.smithi194.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.5 MB/s | 21 MB 00:13 2024-04-25T18:44:21.245 INFO:teuthology.orchestra.run.smithi086.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 897 kB/s | 320 kB 00:00 2024-04-25T18:44:21.465 INFO:teuthology.orchestra.run.smithi194.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 551 kB/s | 225 kB 00:00 2024-04-25T18:44:21.545 INFO:teuthology.orchestra.run.smithi086.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 994 kB/s | 298 kB 00:00 2024-04-25T18:44:21.573 INFO:teuthology.orchestra.run.smithi194.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 250 kB/s | 27 kB 00:00 2024-04-25T18:44:21.687 INFO:teuthology.orchestra.run.smithi086.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 703 kB/s | 99 kB 00:00 2024-04-25T18:44:21.699 INFO:teuthology.orchestra.run.smithi194.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 253 kB/s | 32 kB 00:00 2024-04-25T18:44:21.804 INFO:teuthology.orchestra.run.smithi086.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 739 kB/s | 85 kB 00:00 2024-04-25T18:44:22.388 INFO:teuthology.orchestra.run.smithi086.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.6 MB/s | 5.4 MB 00:02 2024-04-25T18:44:22.612 INFO:teuthology.orchestra.run.smithi183.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 1.2 MB/s | 3.5 MB 00:02 2024-04-25T18:44:22.716 INFO:teuthology.orchestra.run.smithi194.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 734 kB/s | 747 kB 00:01 2024-04-25T18:44:22.887 INFO:teuthology.orchestra.run.smithi183.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 906 kB/s | 249 kB 00:00 2024-04-25T18:44:22.995 INFO:teuthology.orchestra.run.smithi183.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 456 kB/s | 49 kB 00:00 2024-04-25T18:44:23.000 INFO:teuthology.orchestra.run.smithi194.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 575 kB/s | 163 kB 00:00 2024-04-25T18:44:23.104 INFO:teuthology.orchestra.run.smithi183.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T18:44:23.192 INFO:teuthology.orchestra.run.smithi194.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 660 kB/s | 126 kB 00:00 2024-04-25T18:44:23.205 INFO:teuthology.orchestra.run.smithi086.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 209 kB/s | 170 kB 00:00 2024-04-25T18:44:23.379 INFO:teuthology.orchestra.run.smithi183.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 993 kB/s | 273 kB 00:00 2024-04-25T18:44:23.413 INFO:teuthology.orchestra.run.smithi086.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 165 kB/s | 34 kB 00:00 2024-04-25T18:44:23.463 INFO:teuthology.orchestra.run.smithi183.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-04-25T18:44:23.580 INFO:teuthology.orchestra.run.smithi183.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 157 kB/s | 18 kB 00:00 2024-04-25T18:44:23.689 INFO:teuthology.orchestra.run.smithi183.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T18:44:23.781 INFO:teuthology.orchestra.run.smithi183.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-25T18:44:23.830 INFO:teuthology.orchestra.run.smithi086.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 342 kB/s | 142 kB 00:00 2024-04-25T18:44:23.831 INFO:teuthology.orchestra.run.smithi183.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T18:44:23.842 INFO:teuthology.orchestra.run.smithi194.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 770 kB/s | 500 kB 00:00 2024-04-25T18:44:23.873 INFO:teuthology.orchestra.run.smithi183.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T18:44:23.956 INFO:teuthology.orchestra.run.smithi183.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T18:44:23.998 INFO:teuthology.orchestra.run.smithi183.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T18:44:24.040 INFO:teuthology.orchestra.run.smithi183.stdout:(44/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.1 MB/s | 762 kB 00:00 2024-04-25T18:44:24.066 INFO:teuthology.orchestra.run.smithi183.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T18:44:24.091 INFO:teuthology.orchestra.run.smithi183.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T18:44:24.125 INFO:teuthology.orchestra.run.smithi183.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T18:44:24.158 INFO:teuthology.orchestra.run.smithi183.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-04-25T18:44:24.192 INFO:teuthology.orchestra.run.smithi183.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T18:44:24.226 INFO:teuthology.orchestra.run.smithi183.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T18:44:24.242 INFO:teuthology.orchestra.run.smithi183.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T18:44:24.276 INFO:teuthology.orchestra.run.smithi183.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T18:44:24.281 INFO:teuthology.orchestra.run.smithi086.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 1.2 MB/s | 3.1 MB 00:02 2024-04-25T18:44:24.301 INFO:teuthology.orchestra.run.smithi183.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 563 kB/s | 33 kB 00:00 2024-04-25T18:44:24.343 INFO:teuthology.orchestra.run.smithi183.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 452 kB/s | 18 kB 00:00 2024-04-25T18:44:24.402 INFO:teuthology.orchestra.run.smithi183.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T18:44:24.427 INFO:teuthology.orchestra.run.smithi183.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 138 kB/s | 11 kB 00:00 2024-04-25T18:44:24.461 INFO:teuthology.orchestra.run.smithi183.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T18:44:24.486 INFO:teuthology.orchestra.run.smithi183.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-04-25T18:44:24.520 INFO:teuthology.orchestra.run.smithi183.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T18:44:24.545 INFO:teuthology.orchestra.run.smithi183.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T18:44:24.579 INFO:teuthology.orchestra.run.smithi183.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 442 kB/s | 25 kB 00:00 2024-04-25T18:44:24.604 INFO:teuthology.orchestra.run.smithi183.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-25T18:44:24.637 INFO:teuthology.orchestra.run.smithi183.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T18:44:24.663 INFO:teuthology.orchestra.run.smithi183.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T18:44:24.696 INFO:teuthology.orchestra.run.smithi183.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-04-25T18:44:24.930 INFO:teuthology.orchestra.run.smithi183.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-25T18:44:24.997 INFO:teuthology.orchestra.run.smithi183.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-25T18:44:25.039 INFO:teuthology.orchestra.run.smithi183.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T18:44:25.081 INFO:teuthology.orchestra.run.smithi183.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T18:44:25.106 INFO:teuthology.orchestra.run.smithi183.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T18:44:25.173 INFO:teuthology.orchestra.run.smithi183.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T18:44:25.257 INFO:teuthology.orchestra.run.smithi183.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-25T18:44:25.282 INFO:teuthology.orchestra.run.smithi183.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 824 kB/s | 87 kB 00:00 2024-04-25T18:44:25.324 INFO:teuthology.orchestra.run.smithi183.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.9 MB/s | 244 kB 00:00 2024-04-25T18:44:25.374 INFO:teuthology.orchestra.run.smithi183.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.1 MB/s | 248 kB 00:00 2024-04-25T18:44:25.450 INFO:teuthology.orchestra.run.smithi183.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-04-25T18:44:25.550 INFO:teuthology.orchestra.run.smithi183.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 147 kB/s | 26 kB 00:00 2024-04-25T18:44:25.692 INFO:teuthology.orchestra.run.smithi183.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 79 kB/s | 19 kB 00:00 2024-04-25T18:44:25.717 INFO:teuthology.orchestra.run.smithi183.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-04-25T18:44:25.743 INFO:teuthology.orchestra.run.smithi183.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-04-25T18:44:26.035 INFO:teuthology.orchestra.run.smithi183.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 860 kB/s | 250 kB 00:00 2024-04-25T18:44:26.219 INFO:teuthology.orchestra.run.smithi183.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-04-25T18:44:26.244 INFO:teuthology.orchestra.run.smithi183.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-04-25T18:44:26.269 INFO:teuthology.orchestra.run.smithi183.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T18:44:26.286 INFO:teuthology.orchestra.run.smithi183.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T18:44:26.320 INFO:teuthology.orchestra.run.smithi183.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T18:44:26.346 INFO:teuthology.orchestra.run.smithi183.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T18:44:26.370 INFO:teuthology.orchestra.run.smithi183.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T18:44:26.395 INFO:teuthology.orchestra.run.smithi183.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T18:44:26.420 INFO:teuthology.orchestra.run.smithi183.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T18:44:26.521 INFO:teuthology.orchestra.run.smithi183.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-04-25T18:44:26.763 INFO:teuthology.orchestra.run.smithi183.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 184 kB/s | 44 kB 00:00 2024-04-25T18:44:27.097 INFO:teuthology.orchestra.run.smithi183.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T18:44:27.122 INFO:teuthology.orchestra.run.smithi183.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 619 kB/s | 221 kB 00:00 2024-04-25T18:44:27.147 INFO:teuthology.orchestra.run.smithi183.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 725 kB/s | 36 kB 00:00 2024-04-25T18:44:27.172 INFO:teuthology.orchestra.run.smithi183.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T18:44:27.206 INFO:teuthology.orchestra.run.smithi183.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T18:44:27.231 INFO:teuthology.orchestra.run.smithi183.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 997 kB/s | 58 kB 00:00 2024-04-25T18:44:27.265 INFO:teuthology.orchestra.run.smithi183.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T18:44:27.290 INFO:teuthology.orchestra.run.smithi183.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T18:44:27.324 INFO:teuthology.orchestra.run.smithi183.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-04-25T18:44:27.349 INFO:teuthology.orchestra.run.smithi183.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-04-25T18:44:27.399 INFO:teuthology.orchestra.run.smithi183.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T18:44:28.133 INFO:teuthology.orchestra.run.smithi183.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 421 kB/s | 309 kB 00:00 2024-04-25T18:44:28.166 INFO:teuthology.orchestra.run.smithi183.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 672 kB/s | 548 kB 00:00 2024-04-25T18:44:28.250 INFO:teuthology.orchestra.run.smithi183.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-04-25T18:44:28.359 INFO:teuthology.orchestra.run.smithi183.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-25T18:44:28.475 INFO:teuthology.orchestra.run.smithi183.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 417 kB/s | 49 kB 00:00 2024-04-25T18:44:28.617 INFO:teuthology.orchestra.run.smithi183.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-04-25T18:44:28.643 INFO:teuthology.orchestra.run.smithi183.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 404 kB/s | 67 kB 00:00 2024-04-25T18:44:28.726 INFO:teuthology.orchestra.run.smithi183.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T18:44:28.843 INFO:teuthology.orchestra.run.smithi183.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-25T18:44:28.927 INFO:teuthology.orchestra.run.smithi183.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-25T18:44:28.952 INFO:teuthology.orchestra.run.smithi183.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-25T18:44:29.036 INFO:teuthology.orchestra.run.smithi183.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-25T18:44:29.052 INFO:teuthology.orchestra.run.smithi183.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T18:44:29.145 INFO:teuthology.orchestra.run.smithi183.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T18:44:29.209 INFO:teuthology.orchestra.run.smithi183.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-25T18:44:29.270 INFO:teuthology.orchestra.run.smithi183.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T18:44:29.312 INFO:teuthology.orchestra.run.smithi183.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 342 kB/s | 35 kB 00:00 2024-04-25T18:44:29.379 INFO:teuthology.orchestra.run.smithi183.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-25T18:44:29.488 INFO:teuthology.orchestra.run.smithi183.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-25T18:44:29.571 INFO:teuthology.orchestra.run.smithi183.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-04-25T18:44:29.596 INFO:teuthology.orchestra.run.smithi183.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-25T18:44:29.688 INFO:teuthology.orchestra.run.smithi183.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T18:44:29.714 INFO:teuthology.orchestra.run.smithi183.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-04-25T18:44:29.797 INFO:teuthology.orchestra.run.smithi183.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-25T18:44:29.822 INFO:teuthology.orchestra.run.smithi183.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T18:44:29.923 INFO:teuthology.orchestra.run.smithi183.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-25T18:44:29.948 INFO:teuthology.orchestra.run.smithi183.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-04-25T18:44:29.964 INFO:teuthology.orchestra.run.smithi086.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 591 kB/s | 3.5 MB 00:06 2024-04-25T18:44:30.040 INFO:teuthology.orchestra.run.smithi183.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 674 kB/s | 79 kB 00:00 2024-04-25T18:44:30.065 INFO:teuthology.orchestra.run.smithi183.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2024-04-25T18:44:30.149 INFO:teuthology.orchestra.run.smithi183.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-25T18:44:30.174 INFO:teuthology.orchestra.run.smithi183.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-25T18:44:30.266 INFO:teuthology.orchestra.run.smithi183.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-04-25T18:44:30.291 INFO:teuthology.orchestra.run.smithi183.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-04-25T18:44:30.372 INFO:teuthology.orchestra.run.smithi086.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 611 kB/s | 249 kB 00:00 2024-04-25T18:44:30.375 INFO:teuthology.orchestra.run.smithi183.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-25T18:44:30.400 INFO:teuthology.orchestra.run.smithi183.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T18:44:30.484 INFO:teuthology.orchestra.run.smithi183.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-25T18:44:30.489 INFO:teuthology.orchestra.run.smithi086.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 425 kB/s | 49 kB 00:00 2024-04-25T18:44:30.509 INFO:teuthology.orchestra.run.smithi183.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-25T18:44:30.593 INFO:teuthology.orchestra.run.smithi183.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-04-25T18:44:30.606 INFO:teuthology.orchestra.run.smithi086.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 141 kB/s | 16 kB 00:00 2024-04-25T18:44:30.618 INFO:teuthology.orchestra.run.smithi183.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T18:44:30.702 INFO:teuthology.orchestra.run.smithi183.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 826 kB/s | 90 kB 00:00 2024-04-25T18:44:30.735 INFO:teuthology.orchestra.run.smithi183.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T18:44:30.810 INFO:teuthology.orchestra.run.smithi183.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-04-25T18:44:30.852 INFO:teuthology.orchestra.run.smithi183.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T18:44:30.944 INFO:teuthology.orchestra.run.smithi183.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T18:44:30.998 INFO:teuthology.orchestra.run.smithi086.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 699 kB/s | 273 kB 00:00 2024-04-25T18:44:31.458 INFO:teuthology.orchestra.run.smithi194.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 721 kB/s | 5.4 MB 00:07 2024-04-25T18:44:31.567 INFO:teuthology.orchestra.run.smithi194.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 417 kB/s | 45 kB 00:00 2024-04-25T18:44:31.900 INFO:teuthology.orchestra.run.smithi194.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 404 kB/s | 135 kB 00:00 2024-04-25T18:44:32.107 INFO:teuthology.orchestra.run.smithi086.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 688 kB/s | 762 kB 00:01 2024-04-25T18:44:32.224 INFO:teuthology.orchestra.run.smithi086.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 158 kB/s | 18 kB 00:00 2024-04-25T18:44:32.284 INFO:teuthology.orchestra.run.smithi194.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 421 kB/s | 161 kB 00:00 2024-04-25T18:44:32.382 INFO:teuthology.orchestra.run.smithi086.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-04-25T18:44:32.474 INFO:teuthology.orchestra.run.smithi086.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-25T18:44:32.516 INFO:teuthology.orchestra.run.smithi086.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 195 kB 00:00 2024-04-25T18:44:32.591 INFO:teuthology.orchestra.run.smithi086.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T18:44:32.683 INFO:teuthology.orchestra.run.smithi086.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T18:44:32.725 INFO:teuthology.orchestra.run.smithi086.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T18:44:32.767 INFO:teuthology.orchestra.run.smithi086.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T18:44:32.817 INFO:teuthology.orchestra.run.smithi086.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T18:44:32.859 INFO:teuthology.orchestra.run.smithi086.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T18:44:32.910 INFO:teuthology.orchestra.run.smithi194.stdout:(22/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 806 kB/s | 17 MB 00:21 2024-04-25T18:44:32.934 INFO:teuthology.orchestra.run.smithi194.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 493 kB/s | 320 kB 00:00 2024-04-25T18:44:32.944 INFO:teuthology.orchestra.run.smithi086.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 8.0 MB/s | 689 kB 00:00 2024-04-25T18:44:32.961 INFO:teuthology.orchestra.run.smithi183.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 3.1 MB 00:02 2024-04-25T18:44:33.018 INFO:teuthology.orchestra.run.smithi086.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 943 kB 00:00 2024-04-25T18:44:33.045 INFO:teuthology.orchestra.run.smithi183.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 17 MB/s | 1.3 MB 00:00 2024-04-25T18:44:33.060 INFO:teuthology.orchestra.run.smithi086.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-04-25T18:44:33.094 INFO:teuthology.orchestra.run.smithi086.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 659 kB/s | 21 kB 00:00 2024-04-25T18:44:33.143 INFO:teuthology.orchestra.run.smithi194.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 477 kB/s | 99 kB 00:00 2024-04-25T18:44:33.144 INFO:teuthology.orchestra.run.smithi086.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-04-25T18:44:33.145 INFO:teuthology.orchestra.run.smithi183.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T18:44:33.186 INFO:teuthology.orchestra.run.smithi086.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 798 kB/s | 33 kB 00:00 2024-04-25T18:44:33.311 INFO:teuthology.orchestra.run.smithi086.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T18:44:33.335 INFO:teuthology.orchestra.run.smithi194.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 445 kB/s | 85 kB 00:00 2024-04-25T18:44:33.354 INFO:teuthology.orchestra.run.smithi086.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 442 kB/s | 18 kB 00:00 2024-04-25T18:44:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 339 kB/s | 11 kB 00:00 2024-04-25T18:44:33.429 INFO:teuthology.orchestra.run.smithi086.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-25T18:44:33.471 INFO:teuthology.orchestra.run.smithi086.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T18:44:33.555 INFO:teuthology.orchestra.run.smithi086.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T18:44:33.613 INFO:teuthology.orchestra.run.smithi086.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T18:44:33.647 INFO:teuthology.orchestra.run.smithi086.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 771 kB/s | 25 kB 00:00 2024-04-25T18:44:33.689 INFO:teuthology.orchestra.run.smithi086.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T18:44:33.727 INFO:teuthology.orchestra.run.smithi194.stdout:(26/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 364 kB/s | 298 kB 00:00 2024-04-25T18:44:33.739 INFO:teuthology.orchestra.run.smithi086.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.8 MB/s | 292 kB 00:00 2024-04-25T18:44:33.781 INFO:teuthology.orchestra.run.smithi086.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.5 MB/s | 188 kB 00:00 2024-04-25T18:44:33.815 INFO:teuthology.orchestra.run.smithi086.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 1.1 MB/s | 35 kB 00:00 2024-04-25T18:44:33.995 INFO:teuthology.orchestra.run.smithi183.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.1 MB/s | 3.4 MB 00:03 2024-04-25T18:44:33.998 INFO:teuthology.orchestra.run.smithi086.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-25T18:44:34.249 INFO:teuthology.orchestra.run.smithi086.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T18:44:34.310 INFO:teuthology.orchestra.run.smithi194.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 292 kB/s | 170 kB 00:00 2024-04-25T18:44:34.341 INFO:teuthology.orchestra.run.smithi086.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2024-04-25T18:44:34.416 INFO:teuthology.orchestra.run.smithi086.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T18:44:34.458 INFO:teuthology.orchestra.run.smithi086.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.2 MB/s | 133 kB 00:00 2024-04-25T18:44:34.527 INFO:teuthology.orchestra.run.smithi194.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 158 kB/s | 34 kB 00:00 2024-04-25T18:44:34.658 INFO:teuthology.orchestra.run.smithi086.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T18:44:34.725 INFO:teuthology.orchestra.run.smithi086.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T18:44:34.759 INFO:teuthology.orchestra.run.smithi086.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-04-25T18:44:34.967 INFO:teuthology.orchestra.run.smithi086.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-04-25T18:44:34.994 INFO:teuthology.orchestra.run.smithi194.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 305 kB/s | 142 kB 00:00 2024-04-25T18:44:35.017 INFO:teuthology.orchestra.run.smithi086.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-25T18:44:35.068 INFO:teuthology.orchestra.run.smithi086.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T18:44:35.093 INFO:teuthology.orchestra.run.smithi086.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 1.0 MB/s | 26 kB 00:00 2024-04-25T18:44:35.118 INFO:teuthology.orchestra.run.smithi086.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 791 kB/s | 19 kB 00:00 2024-04-25T18:44:35.177 INFO:teuthology.orchestra.run.smithi086.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T18:44:35.219 INFO:teuthology.orchestra.run.smithi086.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 852 kB/s | 35 kB 00:00 2024-04-25T18:44:35.394 INFO:teuthology.orchestra.run.smithi086.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T18:44:35.461 INFO:teuthology.orchestra.run.smithi086.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T18:44:35.512 INFO:teuthology.orchestra.run.smithi086.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T18:44:35.554 INFO:teuthology.orchestra.run.smithi086.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.9 MB/s | 77 kB 00:00 2024-04-25T18:44:35.596 INFO:teuthology.orchestra.run.smithi086.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T18:44:35.663 INFO:teuthology.orchestra.run.smithi086.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-25T18:44:35.704 INFO:teuthology.orchestra.run.smithi086.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.8 MB/s | 159 kB 00:00 2024-04-25T18:44:35.755 INFO:teuthology.orchestra.run.smithi086.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T18:44:35.797 INFO:teuthology.orchestra.run.smithi086.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T18:44:35.839 INFO:teuthology.orchestra.run.smithi086.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-25T18:44:36.264 INFO:teuthology.orchestra.run.smithi086.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-25T18:44:36.307 INFO:teuthology.orchestra.run.smithi086.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-25T18:44:36.348 INFO:teuthology.orchestra.run.smithi086.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-25T18:44:36.415 INFO:teuthology.orchestra.run.smithi086.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-25T18:44:36.457 INFO:teuthology.orchestra.run.smithi086.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 869 kB/s | 36 kB 00:00 2024-04-25T18:44:36.491 INFO:teuthology.orchestra.run.smithi086.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T18:44:36.783 INFO:teuthology.orchestra.run.smithi086.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-04-25T18:44:36.826 INFO:teuthology.orchestra.run.smithi086.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-25T18:44:36.867 INFO:teuthology.orchestra.run.smithi086.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-25T18:44:36.909 INFO:teuthology.orchestra.run.smithi086.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-25T18:44:36.951 INFO:teuthology.orchestra.run.smithi086.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T18:44:36.984 INFO:teuthology.orchestra.run.smithi086.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 686 kB/s | 22 kB 00:00 2024-04-25T18:44:37.051 INFO:teuthology.orchestra.run.smithi086.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T18:44:37.327 INFO:teuthology.orchestra.run.smithi086.stdout:(103/152): ceph-mgr-diskprediction-local-19.0.0 579 kB/s | 7.4 MB 00:13 2024-04-25T18:44:37.885 INFO:teuthology.orchestra.run.smithi086.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 658 kB/s | 548 kB 00:00 2024-04-25T18:44:37.994 INFO:teuthology.orchestra.run.smithi086.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 466 kB/s | 309 kB 00:00 2024-04-25T18:44:38.019 INFO:teuthology.orchestra.run.smithi086.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-25T18:44:38.169 INFO:teuthology.orchestra.run.smithi086.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 174 kB/s | 26 kB 00:00 2024-04-25T18:44:38.286 INFO:teuthology.orchestra.run.smithi086.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2024-04-25T18:44:38.412 INFO:teuthology.orchestra.run.smithi086.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 541 kB/s | 67 kB 00:00 2024-04-25T18:44:38.511 INFO:teuthology.orchestra.run.smithi194.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 611 kB/s | 3.1 MB 00:05 2024-04-25T18:44:38.512 INFO:teuthology.orchestra.run.smithi086.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-04-25T18:44:38.545 INFO:teuthology.orchestra.run.smithi086.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-25T18:44:38.646 INFO:teuthology.orchestra.run.smithi086.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 838 kB 00:00 2024-04-25T18:44:38.704 INFO:teuthology.orchestra.run.smithi086.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-25T18:44:38.738 INFO:teuthology.orchestra.run.smithi086.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 477 kB/s | 43 kB 00:00 2024-04-25T18:44:38.797 INFO:teuthology.orchestra.run.smithi086.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 352 kB/s | 32 kB 00:00 2024-04-25T18:44:38.830 INFO:teuthology.orchestra.run.smithi086.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-04-25T18:44:38.914 INFO:teuthology.orchestra.run.smithi086.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T18:44:38.947 INFO:teuthology.orchestra.run.smithi086.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T18:44:39.039 INFO:teuthology.orchestra.run.smithi086.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T18:44:39.065 INFO:teuthology.orchestra.run.smithi086.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-25T18:44:39.157 INFO:teuthology.orchestra.run.smithi086.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T18:44:39.240 INFO:teuthology.orchestra.run.smithi086.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi086.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T18:44:39.341 INFO:teuthology.orchestra.run.smithi086.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-25T18:44:39.383 INFO:teuthology.orchestra.run.smithi086.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-25T18:44:39.441 INFO:teuthology.orchestra.run.smithi086.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-25T18:44:39.484 INFO:teuthology.orchestra.run.smithi086.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-04-25T18:44:39.542 INFO:teuthology.orchestra.run.smithi086.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T18:44:39.659 INFO:teuthology.orchestra.run.smithi086.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 399 kB/s | 46 kB 00:00 2024-04-25T18:44:39.709 INFO:teuthology.orchestra.run.smithi086.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-25T18:44:39.759 INFO:teuthology.orchestra.run.smithi086.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 795 kB/s | 79 kB 00:00 2024-04-25T18:44:39.809 INFO:teuthology.orchestra.run.smithi086.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-04-25T18:44:39.868 INFO:teuthology.orchestra.run.smithi086.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-25T18:44:39.910 INFO:teuthology.orchestra.run.smithi086.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-25T18:44:39.969 INFO:teuthology.orchestra.run.smithi086.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 904 kB/s | 90 kB 00:00 2024-04-25T18:44:40.011 INFO:teuthology.orchestra.run.smithi086.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-25T18:44:40.069 INFO:teuthology.orchestra.run.smithi086.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-04-25T18:44:40.119 INFO:teuthology.orchestra.run.smithi086.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T18:44:40.170 INFO:teuthology.orchestra.run.smithi086.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 592 kB/s | 59 kB 00:00 2024-04-25T18:44:40.220 INFO:teuthology.orchestra.run.smithi086.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-04-25T18:44:40.270 INFO:teuthology.orchestra.run.smithi086.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 513 kB/s | 51 kB 00:00 2024-04-25T18:44:40.329 INFO:teuthology.orchestra.run.smithi086.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T18:44:40.388 INFO:teuthology.orchestra.run.smithi086.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 769 kB/s | 90 kB 00:00 2024-04-25T18:44:40.454 INFO:teuthology.orchestra.run.smithi086.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T18:44:40.488 INFO:teuthology.orchestra.run.smithi086.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-04-25T18:44:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T18:44:40.639 INFO:teuthology.orchestra.run.smithi086.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T18:44:41.669 INFO:teuthology.orchestra.run.smithi194.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 543 kB/s | 3.5 MB 00:06 2024-04-25T18:44:42.069 INFO:teuthology.orchestra.run.smithi194.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 622 kB/s | 249 kB 00:00 2024-04-25T18:44:42.203 INFO:teuthology.orchestra.run.smithi194.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 370 kB/s | 49 kB 00:00 2024-04-25T18:44:42.311 INFO:teuthology.orchestra.run.smithi194.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T18:44:42.787 INFO:teuthology.orchestra.run.smithi194.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 575 kB/s | 273 kB 00:00 2024-04-25T18:44:43.870 INFO:teuthology.orchestra.run.smithi194.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 704 kB/s | 762 kB 00:01 2024-04-25T18:44:43.987 INFO:teuthology.orchestra.run.smithi194.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 157 kB/s | 18 kB 00:00 2024-04-25T18:44:44.087 INFO:teuthology.orchestra.run.smithi194.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 803 kB/s | 80 kB 00:00 2024-04-25T18:44:44.154 INFO:teuthology.orchestra.run.smithi194.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2024-04-25T18:44:44.188 INFO:teuthology.orchestra.run.smithi194.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 195 kB 00:00 2024-04-25T18:44:44.232 INFO:teuthology.orchestra.run.smithi194.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 253 kB 00:00 2024-04-25T18:44:44.299 INFO:teuthology.orchestra.run.smithi194.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T18:44:44.332 INFO:teuthology.orchestra.run.smithi194.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T18:44:44.366 INFO:teuthology.orchestra.run.smithi194.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T18:44:44.399 INFO:teuthology.orchestra.run.smithi194.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T18:44:44.450 INFO:teuthology.orchestra.run.smithi194.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T18:44:44.492 INFO:teuthology.orchestra.run.smithi194.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 16 MB/s | 689 kB 00:00 2024-04-25T18:44:44.550 INFO:teuthology.orchestra.run.smithi194.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T18:44:44.584 INFO:teuthology.orchestra.run.smithi194.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T18:44:44.618 INFO:teuthology.orchestra.run.smithi194.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 647 kB/s | 21 kB 00:00 2024-04-25T18:44:44.660 INFO:teuthology.orchestra.run.smithi194.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T18:44:44.693 INFO:teuthology.orchestra.run.smithi194.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 986 kB/s | 33 kB 00:00 2024-04-25T18:44:44.794 INFO:teuthology.orchestra.run.smithi194.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T18:44:44.830 INFO:teuthology.orchestra.run.smithi194.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 553 kB/s | 18 kB 00:00 2024-04-25T18:44:44.861 INFO:teuthology.orchestra.run.smithi194.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-04-25T18:44:44.895 INFO:teuthology.orchestra.run.smithi194.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 160 kB 00:00 2024-04-25T18:44:44.922 INFO:teuthology.orchestra.run.smithi086.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 798 kB/s | 3.4 MB 00:04 2024-04-25T18:44:44.928 INFO:teuthology.orchestra.run.smithi194.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T18:44:44.987 INFO:teuthology.orchestra.run.smithi194.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T18:44:45.006 INFO:teuthology.orchestra.run.smithi086.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 17 MB/s | 1.3 MB 00:00 2024-04-25T18:44:45.037 INFO:teuthology.orchestra.run.smithi194.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-25T18:44:45.062 INFO:teuthology.orchestra.run.smithi194.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 1.0 MB/s | 25 kB 00:00 2024-04-25T18:44:45.096 INFO:teuthology.orchestra.run.smithi194.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T18:44:45.138 INFO:teuthology.orchestra.run.smithi194.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T18:44:45.139 INFO:teuthology.orchestra.run.smithi086.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-25T18:44:45.180 INFO:teuthology.orchestra.run.smithi194.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T18:44:45.213 INFO:teuthology.orchestra.run.smithi194.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-04-25T18:44:45.339 INFO:teuthology.orchestra.run.smithi194.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-25T18:44:45.464 INFO:teuthology.orchestra.run.smithi194.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-04-25T18:44:45.531 INFO:teuthology.orchestra.run.smithi194.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-25T18:44:45.590 INFO:teuthology.orchestra.run.smithi194.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-04-25T18:44:45.639 INFO:teuthology.orchestra.run.smithi194.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.7 MB/s | 133 kB 00:00 2024-04-25T18:44:45.816 INFO:teuthology.orchestra.run.smithi194.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-25T18:44:45.883 INFO:teuthology.orchestra.run.smithi194.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T18:44:45.925 INFO:teuthology.orchestra.run.smithi194.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.0 MB/s | 87 kB 00:00 2024-04-25T18:44:46.183 INFO:teuthology.orchestra.run.smithi194.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-25T18:44:46.226 INFO:teuthology.orchestra.run.smithi194.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-25T18:44:46.267 INFO:teuthology.orchestra.run.smithi194.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-25T18:44:46.309 INFO:teuthology.orchestra.run.smithi194.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-04-25T18:44:46.351 INFO:teuthology.orchestra.run.smithi194.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-04-25T18:44:46.393 INFO:teuthology.orchestra.run.smithi194.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T18:44:46.427 INFO:teuthology.orchestra.run.smithi194.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-04-25T18:44:46.585 INFO:teuthology.orchestra.run.smithi194.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T18:44:46.627 INFO:teuthology.orchestra.run.smithi194.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-25T18:44:46.669 INFO:teuthology.orchestra.run.smithi194.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-04-25T18:44:46.703 INFO:teuthology.orchestra.run.smithi194.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2024-04-25T18:44:46.736 INFO:teuthology.orchestra.run.smithi194.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-04-25T18:44:46.773 INFO:teuthology.orchestra.run.smithi086.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 525 kB/s | 3.1 MB 00:06 2024-04-25T18:44:46.787 INFO:teuthology.orchestra.run.smithi194.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T18:44:46.820 INFO:teuthology.orchestra.run.smithi194.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 4.7 MB/s | 159 kB 00:00 2024-04-25T18:44:46.862 INFO:teuthology.orchestra.run.smithi194.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-04-25T18:44:46.896 INFO:teuthology.orchestra.run.smithi194.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-04-25T18:44:46.930 INFO:teuthology.orchestra.run.smithi194.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T18:44:47.322 INFO:teuthology.orchestra.run.smithi194.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-25T18:44:47.346 INFO:teuthology.orchestra.run.smithi194.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.6 MB/s | 42 kB 00:00 2024-04-25T18:44:47.372 INFO:teuthology.orchestra.run.smithi194.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.7 MB/s | 44 kB 00:00 2024-04-25T18:44:47.414 INFO:teuthology.orchestra.run.smithi194.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-25T18:44:47.439 INFO:teuthology.orchestra.run.smithi194.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.4 MB/s | 36 kB 00:00 2024-04-25T18:44:47.464 INFO:teuthology.orchestra.run.smithi194.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.9 MB/s | 47 kB 00:00 2024-04-25T18:44:47.506 INFO:teuthology.orchestra.run.smithi194.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-04-25T18:44:47.532 INFO:teuthology.orchestra.run.smithi194.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 2.3 MB/s | 58 kB 00:00 2024-04-25T18:44:47.565 INFO:teuthology.orchestra.run.smithi194.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.7 MB/s | 94 kB 00:00 2024-04-25T18:44:47.599 INFO:teuthology.orchestra.run.smithi194.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-04-25T18:44:47.641 INFO:teuthology.orchestra.run.smithi194.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T18:44:47.674 INFO:teuthology.orchestra.run.smithi194.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 670 kB/s | 22 kB 00:00 2024-04-25T18:44:47.725 INFO:teuthology.orchestra.run.smithi194.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 17 MB/s | 863 kB 00:00 2024-04-25T18:44:48.008 INFO:teuthology.orchestra.run.smithi194.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-25T18:44:48.167 INFO:teuthology.orchestra.run.smithi194.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-04-25T18:44:48.284 INFO:teuthology.orchestra.run.smithi194.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-25T18:44:48.584 INFO:teuthology.orchestra.run.smithi194.stdout:(106/152): ceph-mgr-diskprediction-local-19.0.0 750 kB/s | 7.4 MB 00:10 2024-04-25T18:44:48.693 INFO:teuthology.orchestra.run.smithi194.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-25T18:44:48.810 INFO:teuthology.orchestra.run.smithi194.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 418 kB/s | 49 kB 00:00 2024-04-25T18:44:48.968 INFO:teuthology.orchestra.run.smithi194.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-25T18:44:48.994 INFO:teuthology.orchestra.run.smithi194.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-25T18:44:49.152 INFO:teuthology.orchestra.run.smithi194.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 828 kB/s | 151 kB 00:00 2024-04-25T18:44:49.278 INFO:teuthology.orchestra.run.smithi194.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-25T18:44:49.386 INFO:teuthology.orchestra.run.smithi194.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-25T18:44:49.411 INFO:teuthology.orchestra.run.smithi194.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-04-25T18:44:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 203 kB/s | 32 kB 00:00 2024-04-25T18:44:49.570 INFO:teuthology.orchestra.run.smithi194.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-04-25T18:44:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 824 kB/s | 172 kB 00:00 2024-04-25T18:44:49.821 INFO:teuthology.orchestra.run.smithi194.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-04-25T18:44:49.929 INFO:teuthology.orchestra.run.smithi194.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 328 kB/s | 35 kB 00:00 2024-04-25T18:44:49.988 INFO:teuthology.orchestra.run.smithi194.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.3 MB/s | 303 kB 00:00 2024-04-25T18:44:50.088 INFO:teuthology.orchestra.run.smithi194.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-25T18:44:50.213 INFO:teuthology.orchestra.run.smithi194.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T18:44:50.314 INFO:teuthology.orchestra.run.smithi194.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-25T18:44:50.431 INFO:teuthology.orchestra.run.smithi194.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-04-25T18:44:50.481 INFO:teuthology.orchestra.run.smithi194.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 106 kB/s | 18 kB 00:00 2024-04-25T18:44:50.623 INFO:teuthology.orchestra.run.smithi194.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 121 kB/s | 23 kB 00:00 2024-04-25T18:44:50.733 INFO:teuthology.orchestra.run.smithi194.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 85 kB/s | 19 kB 00:00 2024-04-25T18:44:50.757 INFO:teuthology.orchestra.run.smithi194.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2024-04-25T18:44:50.907 INFO:teuthology.orchestra.run.smithi194.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 367 kB/s | 46 kB 00:00 2024-04-25T18:44:51.008 INFO:teuthology.orchestra.run.smithi194.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-04-25T18:44:51.041 INFO:teuthology.orchestra.run.smithi194.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 592 kB/s | 79 kB 00:00 2024-04-25T18:44:51.158 INFO:teuthology.orchestra.run.smithi194.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 385 kB/s | 58 kB 00:00 2024-04-25T18:44:51.216 INFO:teuthology.orchestra.run.smithi194.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-25T18:44:51.292 INFO:teuthology.orchestra.run.smithi194.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-04-25T18:44:51.375 INFO:teuthology.orchestra.run.smithi194.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 571 kB/s | 90 kB 00:00 2024-04-25T18:44:51.425 INFO:teuthology.orchestra.run.smithi194.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 212 kB/s | 28 kB 00:00 2024-04-25T18:44:51.484 INFO:teuthology.orchestra.run.smithi194.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-25T18:44:51.567 INFO:teuthology.orchestra.run.smithi194.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-25T18:44:51.618 INFO:teuthology.orchestra.run.smithi194.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-04-25T18:44:51.677 INFO:teuthology.orchestra.run.smithi194.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-25T18:44:51.743 INFO:teuthology.orchestra.run.smithi194.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 410 kB/s | 51 kB 00:00 2024-04-25T18:44:51.852 INFO:teuthology.orchestra.run.smithi194.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-25T18:44:51.886 INFO:teuthology.orchestra.run.smithi194.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 632 kB/s | 90 kB 00:00 2024-04-25T18:44:51.994 INFO:teuthology.orchestra.run.smithi194.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 236 kB/s | 20 kB 00:00 2024-04-25T18:44:52.061 INFO:teuthology.orchestra.run.smithi194.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-04-25T18:44:52.187 INFO:teuthology.orchestra.run.smithi194.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-25T18:44:52.470 INFO:teuthology.orchestra.run.smithi194.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-25T18:44:57.270 INFO:teuthology.orchestra.run.smithi194.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 684 kB/s | 3.4 MB 00:05 2024-04-25T18:44:57.345 INFO:teuthology.orchestra.run.smithi194.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T18:44:57.479 INFO:teuthology.orchestra.run.smithi194.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-25T18:44:58.526 INFO:teuthology.orchestra.run.smithi183.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.1 MB/s | 49 MB 00:46 2024-04-25T18:44:58.530 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:44:58.530 INFO:teuthology.orchestra.run.smithi183.stdout:Total 4.1 MB/s | 222 MB 00:53 2024-04-25T18:44:59.595 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-04-25T18:44:59.740 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-04-25T18:44:59.740 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-04-25T18:45:01.499 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-04-25T18:45:01.499 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-04-25T18:45:03.120 INFO:teuthology.orchestra.run.smithi194.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 302 kB/s | 3.1 MB 00:10 2024-04-25T18:45:04.254 INFO:teuthology.orchestra.run.smithi086.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.1 MB/s | 49 MB 00:45 2024-04-25T18:45:04.264 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:45:04.264 INFO:teuthology.orchestra.run.smithi086.stdout:Total 4.0 MB/s | 222 MB 00:55 2024-04-25T18:45:07.110 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-25T18:45:07.350 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-25T18:45:07.350 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-25T18:45:07.409 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-04-25T18:45:07.487 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:45:07.533 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:45:07.587 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:45:07.629 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:45:07.655 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:45:07.697 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:45:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:45:08.123 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:08.227 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:08.257 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:45:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:45:08.575 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:45:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:08.707 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:08.729 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:45:08.791 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:45:08.813 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:45:08.833 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:45:09.030 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:45:09.062 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:45:09.171 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:45:09.264 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:45:09.304 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:45:09.338 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:45:09.402 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:45:09.437 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:45:09.481 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:45:09.513 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:45:09.542 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:45:09.609 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:45:09.663 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:45:09.703 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:45:09.738 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:45:09.814 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:45:09.849 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:45:09.892 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:45:09.930 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:45:09.998 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:45:10.036 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-25T18:45:10.036 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-25T18:45:10.044 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:45:10.118 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:45:10.256 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:45:10.313 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:45:10.828 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:45:10.907 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:45:10.938 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:45:10.989 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:45:11.027 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:45:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:45:11.225 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:45:11.280 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:45:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:45:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:45:11.447 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:45:11.483 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:45:11.530 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:45:11.566 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:45:11.609 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:45:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:45:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:45:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:45:11.862 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:45:11.903 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:45:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:45:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:45:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:45:12.111 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:45:12.136 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:45:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:45:12.336 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:45:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:45:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:45:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:45:12.478 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:45:12.518 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:45:12.548 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:45:12.582 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:45:12.616 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:45:12.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:45:12.692 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:45:12.778 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:45:12.822 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:45:12.854 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:45:12.909 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:45:13.015 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:45:13.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:45:13.781 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:45:13.836 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:45:13.869 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:45:13.967 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:45:14.027 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:45:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:45:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:45:14.324 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:45:14.352 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:45:14.430 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:45:15.068 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:45:15.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:45:16.305 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-25T18:45:16.383 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:45:16.431 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:45:16.464 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:45:16.492 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:45:16.521 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:45:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:45:16.625 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:45:16.689 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:45:16.729 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:45:16.845 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:45:16.864 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:45:16.897 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:45:16.996 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:45:17.026 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:45:17.063 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:45:17.305 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:45:17.402 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:17.539 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:45:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:45:17.587 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:17.619 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:45:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:45:17.743 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:17.794 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:45:18.145 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:45:18.213 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:45:18.240 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:18.397 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:18.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:45:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:45:18.544 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:45:18.611 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:45:18.912 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:45:18.961 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:45:19.121 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:45:19.238 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:45:19.286 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:45:19.334 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:45:19.444 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:45:19.490 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:45:19.550 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:45:19.594 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:45:19.624 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:45:19.719 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:45:19.788 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:45:19.827 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:45:19.881 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:45:19.941 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:19.981 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:19.991 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:45:20.031 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:45:20.084 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:45:20.121 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:45:20.236 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:45:20.293 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:45:20.419 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:45:20.520 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:20.559 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:45:20.598 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:45:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:45:20.650 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:45:20.651 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:45:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:20.670 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:45:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:45:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:45:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:45:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:45:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:45:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:45:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:45:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:45:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:45:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:45:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:45:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:45:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:45:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:45:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:45:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:45:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:45:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:45:22.718 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:45:22.755 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:45:22.811 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:45:22.941 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:45:22.980 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:45:23.026 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:45:23.061 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:45:23.094 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:45:23.283 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:45:23.465 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:45:23.509 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:45:23.552 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:45:23.595 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:45:23.653 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:45:23.703 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:45:23.745 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:45:23.791 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:45:23.841 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:45:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:45:23.965 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:45:24.107 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:45:24.217 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:45:24.265 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:45:24.335 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:45:24.482 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:45:24.544 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:45:25.473 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:45:25.526 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:45:25.568 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:45:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:45:25.754 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:45:25.851 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:45:25.886 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:45:26.113 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:45:26.141 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:45:26.255 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:45:27.091 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:45:27.269 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:45:27.400 INFO:teuthology.orchestra.run.smithi194.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 724 kB/s | 49 MB 01:09 2024-04-25T18:45:27.404 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:45:27.404 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.7 MB/s | 222 MB 01:20 2024-04-25T18:45:28.456 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-25T18:45:28.600 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-25T18:45:28.600 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-25T18:45:28.883 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:45:28.972 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:45:29.011 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:45:29.055 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:45:29.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:45:29.469 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:45:29.597 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:45:29.633 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:45:29.698 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:45:30.249 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:45:30.280 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:45:30.354 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:45:30.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:45:30.422 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-25T18:45:30.422 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-25T18:45:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:45:32.082 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-04-25T18:45:32.082 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-04-25T18:45:32.082 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-04-25T18:45:32.082 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-04-25T18:45:32.082 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-04-25T18:45:32.083 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-04-25T18:45:32.083 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-04-25T18:45:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:45:32.271 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:45:32.271 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:32.271 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:45:32.271 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:33.001 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:33.045 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:33.687 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:33.731 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:45:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:45:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:45:33.767 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:33.767 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:45:33.767 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:45:33.767 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:45:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:33.893 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:45:33.893 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:45:33.893 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:45:33.893 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:33.957 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:45:33.992 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:45:34.014 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:45:34.015 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:34.015 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:45:34.015 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:45:34.015 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:45:34.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:34.452 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:45:34.474 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:45:34.474 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:34.474 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:45:34.474 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:45:34.474 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:45:34.474 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:34.573 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:45:34.622 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:45:34.677 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:45:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:45:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:45:34.996 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:45:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:45:35.224 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:45:35.269 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:45:35.301 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:45:35.329 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:45:35.790 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:45:35.819 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:45:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:45:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:45:35.950 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:45:36.361 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-25T18:45:36.445 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:45:36.482 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:45:36.536 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:45:36.578 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:45:36.604 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:45:36.646 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:45:36.965 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:45:36.983 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:45:37.027 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:45:37.072 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:37.191 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:37.239 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:37.330 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:37.377 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:45:37.524 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:45:37.573 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:45:37.602 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:37.704 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:37.728 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:45:37.773 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:45:37.795 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:45:37.815 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:45:37.988 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:45:38.019 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:45:38.036 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:45:38.070 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:45:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:45:38.176 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:45:38.263 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:45:38.282 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:45:38.302 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:45:38.312 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:45:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:45:38.338 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:38.338 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:45:38.338 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:45:38.338 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:45:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:38.338 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:45:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:45:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:45:38.400 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:45:38.436 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:45:38.479 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:45:38.497 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:45:38.512 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:45:38.540 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:45:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:45:38.607 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:45:38.669 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:45:38.701 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:45:38.736 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:45:38.820 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:45:38.855 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:45:38.898 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:45:38.927 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:45:39.004 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:45:39.050 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:45:39.132 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:45:39.288 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:45:39.344 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:45:39.472 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:45:39.499 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:45:39.522 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:45:39.522 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:39.522 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:45:39.522 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:45:39.522 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:45:39.522 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:39.567 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:45:39.588 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:45:39.588 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:39.588 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:45:39.588 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:39.642 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:45:39.664 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:45:39.664 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:39.665 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:45:39.665 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:45:39.665 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:45:39.665 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:39.875 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:45:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:45:39.968 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:45:39.990 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:45:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:45:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:45:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:45:39.990 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:39.993 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:45:40.036 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:45:40.074 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:45:40.116 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:45:40.279 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:45:40.335 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:45:40.390 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:45:40.463 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:45:40.502 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:45:40.539 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:45:40.586 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:45:40.621 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:45:40.665 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:45:40.707 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:45:40.842 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:45:40.895 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:45:40.926 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:45:40.967 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:45:41.060 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:45:41.101 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:45:41.141 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:45:41.166 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:45:41.191 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:45:41.266 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:45:41.391 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:45:41.423 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:45:41.451 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:45:41.485 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:45:41.524 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:45:41.564 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:45:41.602 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:45:41.653 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:45:41.728 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:45:41.762 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:45:41.849 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:45:41.893 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:45:41.933 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:45:41.980 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:45:42.079 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:45:42.133 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:45:42.845 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:45:42.897 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:45:42.939 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:45:43.046 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:45:43.106 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:45:43.194 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:45:43.233 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:45:43.411 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:45:43.439 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:45:43.516 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:45:44.151 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:45:44.270 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:45:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:45:44.612 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:45:44.653 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:45:44.685 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:45:44.712 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:45:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:45:44.934 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:45:44.964 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:45:45.491 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:45:45.567 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:45:45.575 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:45:45.594 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:45:45.611 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:45:45.628 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:45:45.631 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:45:45.940 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:45:45.982 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:45:46.073 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:45:46.112 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:45:46.148 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:45:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:45:46.549 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:45:46.590 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:45:46.625 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:45:46.652 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:45:46.670 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:45:46.705 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:45:46.705 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:45:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:45:46.745 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:45:46.805 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:45:46.838 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:45:46.838 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:45:46.930 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-04-25T18:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-04-25T18:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-04-25T18:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-04-25T18:45:46.959 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-04-25T18:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-04-25T18:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-04-25T18:45:46.960 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:46.994 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:45:47.202 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:45:47.249 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:45:47.250 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:48.914 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:45:48.991 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:45:48.991 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:48.991 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:45:48.991 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:45:48.991 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:45:48.991 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:49.014 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:49.077 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:49.256 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:45:49.324 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:45:49.324 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:49.324 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:45:49.324 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:45:49.324 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:45:49.325 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:49.615 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:45:49.661 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:45:49.732 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:45:49.732 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:45:49.732 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:45:49.732 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:45:49.762 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:45:49.815 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:45:49.882 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:45:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:45:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:45:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:45:49.883 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:50.016 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:45:50.077 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:45:50.151 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:45:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:45:50.461 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:45:50.547 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:45:50.656 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:45:50.821 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:45:50.873 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:45:50.917 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:45:50.951 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:45:51.738 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:45:51.788 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:45:51.846 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:45:51.954 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:45:51.989 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:45:52.849 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:45:52.850 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:45:52.851 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:45:52.852 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:45:52.852 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:45:52.852 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:45:52.852 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:45:52.852 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:45:52.852 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:45:52.864 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:45:52.865 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:45:52.866 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:45:52.867 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:45:52.868 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:45:52.869 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:45:52.870 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:45:52.871 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:45:52.872 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:45:52.873 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:45:53.310 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:45:53.358 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:45:54.456 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:45:54.483 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:45:54.629 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:45:54.807 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:45:54.843 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:45:54.915 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:45:54.915 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:54.915 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:45:54.915 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:45:54.915 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:45:54.915 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:45:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.919 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.920 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:45:54.921 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.922 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:45:54.923 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.924 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:45:54.925 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:45:54.926 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:45:54.927 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:45:54.928 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:45:54.929 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:45:54.930 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:45:54.930 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-25T18:45:54.930 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-04-25T18:45:54.978 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:45:55.014 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:45:55.119 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:45:55.189 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:45:55.339 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:56.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:45:56.238 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:45:56.304 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:45:56.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:56.347 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:45:56.415 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:45:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:45:56.416 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:56.482 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:45:56.554 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:45:56.555 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:56.555 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:45:56.555 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:45:56.555 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:45:56.555 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:45:56.932 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:45:57.000 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:45:57.000 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:45:57.000 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:45:57.000 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:45:57.000 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:45:57.000 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:46:00.933 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:46:00.933 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-04-25T18:46:00.933 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-04-25T18:46:00.934 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-04-25T18:46:00.934 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-04-25T18:46:00.934 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-04-25T18:46:00.934 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /root 2024-04-25T18:46:00.934 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-04-25T18:46:00.934 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:01.106 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:46:01.122 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:46:01.122 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:01.122 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:46:01.122 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:46:02.098 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:46:02.202 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:46:02.245 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:46:02.294 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:46:02.609 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:46:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:46:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:46:02.634 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:02.634 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:46:02.634 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:46:02.634 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:46:02.634 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:02.677 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:46:02.714 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:46:02.859 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:46:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:46:02.882 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:02.882 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:46:02.882 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:46:02.882 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:46:02.882 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:03.327 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:46:03.348 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:46:03.348 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:03.349 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:46:03.349 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:46:03.349 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:46:03.349 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:03.456 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:46:03.513 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:46:03.577 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:46:03.647 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:46:03.695 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:46:03.752 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:46:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:46:03.810 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:46:03.896 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:46:04.009 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:46:04.147 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:46:04.185 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:46:04.217 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:46:04.245 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:46:04.580 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:46:04.609 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:46:04.641 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:46:04.702 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:46:04.742 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:46:05.105 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:46:05.218 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:46:05.300 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:46:05.482 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:46:05.524 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:46:05.524 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:46:05.650 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:46:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:46:05.689 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:46:05.773 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:46:05.810 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:46:05.855 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:46:05.986 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:46:06.256 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:46:06.256 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T18:46:06.256 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:46:06.770 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:46:06.794 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:46:06.896 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:46:07.049 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:46:07.087 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:46:07.109 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:46:07.109 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:07.109 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:46:07.110 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:46:07.110 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:46:07.110 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:07.155 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:46:07.165 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:46:07.262 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:46:07.311 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:46:08.238 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:46:08.265 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:46:08.288 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:46:08.288 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:08.288 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:46:08.288 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:46:08.288 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:46:08.288 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:08.333 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:46:08.354 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:46:08.354 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:08.355 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:46:08.355 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:08.400 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:46:08.422 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:46:08.422 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:08.422 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:46:08.423 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:46:08.423 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:46:08.423 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:08.734 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:46:08.756 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:46:08.756 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:46:08.756 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:46:08.756 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:46:08.756 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:46:08.756 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:46:11.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:46:11.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:46:11.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:46:11.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:46:11.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:46:11.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:46:11.970 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:46:11.971 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:46:11.983 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:46:11.983 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:46:11.984 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:46:11.985 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:46:11.986 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:46:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:46:11.988 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:46:11.989 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:46:11.990 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:46:11.991 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:46:11.992 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:46:13.346 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:46:13.395 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:46:13.435 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:46:13.468 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:46:13.495 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:46:13.677 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:46:13.725 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:46:13.755 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:46:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:46:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:46:14.343 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-25T18:46:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:46:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:46:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:46:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:46:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:46:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:46:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:46:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:46:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:46:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:46:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:46:14.355 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-25T18:46:14.385 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:46:14.419 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:46:14.438 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:46:15.004 DEBUG:teuthology.parallel:result is None 2024-04-25T18:46:15.268 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:46:15.357 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:46:15.397 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:46:15.478 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:46:15.519 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:46:15.519 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:46:15.603 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:46:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:46:15.636 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:46:15.809 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:46:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:46:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:46:21.453 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:46:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:46:21.455 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:46:21.456 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:46:21.457 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:46:21.458 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:46:21.459 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:46:21.460 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:46:21.461 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:46:21.462 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:46:21.463 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:46:21.464 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-25T18:46:23.584 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.586 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.587 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:46:23.588 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:46:23.589 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:46:23.590 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:46:23.591 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:46:23.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:46:23.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:46:23.594 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:46:23.595 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-25T18:46:23.986 DEBUG:teuthology.parallel:result is None 2024-04-25T18:46:23.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:24.215 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:46:24.280 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:46:24.281 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:46:24.281 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:46:24.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:24.513 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:46:24.539 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:46:24.539 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:46:24.540 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:46:24.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:24.790 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:46:24.812 INFO:teuthology.orchestra.run.smithi194.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:46:24.812 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:46:24.812 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:46:24.814 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T18:46:24.814 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:46:24.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:46:24.906 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:46:24.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:46:24.937 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:46:24.937 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:46:24.963 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T18:46:24.964 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:46:24.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:46:25.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:46:25.159 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:46:25.159 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:46:25.188 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:46:25.254 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:46:25.254 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:46:25.279 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:46:25.342 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T18:46:25.342 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:46:25.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:46:25.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:46:25.523 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:46:25.523 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:46:25.547 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:46:25.611 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:46:25.611 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:46:25.635 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:46:25.697 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T18:46:25.698 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:46:25.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:46:25.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:46:25.841 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:46:25.841 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:46:25.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:46:25.927 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:46:25.927 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:46:25.950 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:46:26.014 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T18:46:26.104 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T18:46:26.104 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:26.104 INFO:tasks.cephadm:Cluster fsid is 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:46:26.105 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T18:46:26.105 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.194'} 2024-04-25T18:46:26.105 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-04-25T18:46:26.105 INFO:tasks.cephadm:First mgr is x 2024-04-25T18:46:26.105 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T18:46:26.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-04-25T18:46:26.178 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-04-25T18:46:26.201 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2024-04-25T18:46:26.224 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T18:46:26.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:26.542 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T18:46:26.806 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:26.807 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T18:46:26.808 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:27.323 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:46 /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:27.324 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:27.922 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:46 /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:27.922 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:28.486 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:46 /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:28.487 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:28.547 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:28.566 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:46:28.598 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T18:46:28.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:46:28.606 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:46:28.608 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:46:30.065 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:46:30.188 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:46:30.680 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:47:02.082 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-25T18:47:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:47:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:47:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2024-04-25T18:47:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:47:02.083 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2024-04-25T18:47:02.083 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-25T18:48:16.523 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T18:48:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:48:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:48:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-04-25T18:48:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:48:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-04-25T18:48:16.524 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T18:49:10.823 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T18:49:10.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-04-25T18:49:10.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-04-25T18:49:10.968 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2024-04-25T18:49:11.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-04-25T18:49:11.080 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-04-25T18:49:11.107 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2024-04-25T18:49:11.135 INFO:tasks.cephadm:Writing seed config... 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T18:49:11.137 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T18:49:11.138 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T18:49:11.139 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T18:49:11.140 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T18:49:11.140 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T18:49:11.140 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T18:49:11.140 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T18:49:11.140 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:49:11.140 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T18:49:11.192 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1ec2eb56-0334-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T18:49:11.193 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T18:49:11.240 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T18:49:11.285 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T18:49:11.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:49:11.552 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:49:11.552 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '1ec2eb56-0334-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-04-25T18:49:11.552 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T18:49:11.552 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-04-25T18:49:11.623 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T18:49:11.623 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-04-25T18:49:11.623 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-04-25T18:49:11.638 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T18:49:11.638 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T18:49:11.653 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T18:49:11.653 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-25T18:49:11.669 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-25T18:49:11.683 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-25T18:49:11.684 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-25T18:49:11.684 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-04-25T18:49:11.737 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T18:49:11.737 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T18:49:11.737 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-04-25T18:49:11.738 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-04-25T18:49:11.753 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T18:49:11.753 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T18:49:11.767 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T18:49:11.767 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-25T18:49:11.786 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-25T18:49:11.803 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-25T18:49:11.803 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-25T18:49:11.804 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-04-25T18:49:11.804 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:11.804 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 139642674826400 on /run/cephadm/1ec2eb56-0334-11ef-bc93-c7b262605968.lock 2024-04-25T18:49:11.805 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 139642674826400 acquired on /run/cephadm/1ec2eb56-0334-11ef-bc93-c7b262605968.lock 2024-04-25T18:49:11.806 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-04-25T18:49:11.807 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-04-25T18:49:11.808 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-25T18:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.86 metric 100 2024-04-25T18:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-04-25T18:49:11.832 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T18:49:11.833 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-25T18:49:11.833 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-04-25T18:49:11.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T18:49:11.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T18:49:11.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T18:49:11.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T18:49:11.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T18:49:11.844 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T18:49:11.846 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-25T18:49:11.846 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-25T18:49:11.846 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T18:49:11.848 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T18:49:11.850 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T18:49:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T18:49:13.876 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:49:13.877 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:49:13.877 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T18:49:15.145 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-04-25T18:49:15.145 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-04-25T18:49:16.272 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCrpSpmWEASIxAAsAiO7DOm80O4dJJ4sblNGg== 2024-04-25T18:49:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCspSpmIHBLLhAAfTULcjXb1A9yySFjRB8Kjw== 2024-04-25T18:49:18.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCtpSpmqLChNRAAPc8ItKdY6h9YAqvn7dQLTw== 2024-04-25T18:49:18.516 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-04-25T18:49:19.780 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:19.781 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T18:49:19.781 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:49:19.781 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-04-25T18:49:20.965 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-04-25T18:49:21.368 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T18:49:21.630 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T18:49:21.916 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968.target. 2024-04-25T18:49:21.917 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968.target. 2024-04-25T18:49:22.292 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a 2024-04-25T18:49:22.292 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T18:49:22.565 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968.target.wants/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@.service. 2024-04-25T18:49:23.208 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:22 smithi086 podman[27397]: 2024-04-25 18:49:22.943794733 +0000 UTC m=+0.133057862 container create af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:49:23.208 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:23 smithi086 podman[27397]: 2024-04-25 18:49:23.206250933 +0000 UTC m=+0.395514062 container init af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:49:23.365 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:49:23.365 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-25T18:49:23.384 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:49:23.384 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-25T18:49:23.384 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-25T18:49:23.384 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T18:49:23.385 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-04-25T18:49:23.385 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-04-25T18:49:23.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:23 smithi086 podman[27397]: 2024-04-25 18:49:23.217033301 +0000 UTC m=+0.406296429 container start af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:49:23.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:23 smithi086 ceph-mon[27416]: mkfs 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:23.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:23 smithi086 ceph-mon[27416]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:49:23.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:23 smithi086 bash[27397]: af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab 2024-04-25T18:49:23.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:23 smithi086 systemd[1]: Started Ceph mon.a for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:49:24.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:49:24.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: monmap epoch 1 2024-04-25T18:49:24.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:24.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: last_changed 2024-04-25T18:49:19.080254+0000 2024-04-25T18:49:24.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: min_mon_release 19 (squid) 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: election_strategy: 1 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: fsmap 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: mgrmap e1: no daemons active 2024-04-25T18:49:24.452 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:24 smithi086 ceph-mon[27416]: from='client.? 172.21.15.86:0/321475710' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.777353s) 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:24.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.977+0000 7fe0fd200700 1 Processor -- start 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.977+0000 7fe0fd200700 1 -- start start 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.978+0000 7fe0fd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f810ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.978+0000 7fe0fd200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f810b0f0 con 0x7fe0f810a700 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.979+0000 7fe0f5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f810ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.979+0000 7fe0f5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f810ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42310/0 (socket says 172.21.15.86:42310) 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.979+0000 7fe0f5e00700 1 -- 172.21.15.86:0/3528655977 learned_addr learned my addr 172.21.15.86:0/3528655977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:24.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.980+0000 7fe0f5e00700 1 -- 172.21.15.86:0/3528655977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f810b8b0 con 0x7fe0f810a700 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.980+0000 7fe0f5e00700 1 --2- 172.21.15.86:0/3528655977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f810ab20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80098a0 tx=0x7fe0e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e874c112f0a6988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.980+0000 7fe0f4a00700 1 -- 172.21.15.86:0/3528655977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e8004160 con 0x7fe0f810a700 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.980+0000 7fe0f4a00700 1 -- 172.21.15.86:0/3528655977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8004300 con 0x7fe0f810a700 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.980+0000 7fe0f4a00700 1 -- 172.21.15.86:0/3528655977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e80036a0 con 0x7fe0f810a700 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 -- 172.21.15.86:0/3528655977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 msgr2=0x7fe0f810ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 --2- 172.21.15.86:0/3528655977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f810ab20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80098a0 tx=0x7fe0e8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 -- 172.21.15.86:0/3528655977 shutdown_connections 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 --2- 172.21.15.86:0/3528655977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f810ab20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 -- 172.21.15.86:0/3528655977 >> 172.21.15.86:0/3528655977 conn(0x7fe0f807dc00 msgr2=0x7fe0f81084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 -- 172.21.15.86:0/3528655977 shutdown_connections 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.981+0000 7fe0fd200700 1 -- 172.21.15.86:0/3528655977 wait complete. 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.982+0000 7fe0fd200700 1 Processor -- start 2024-04-25T18:49:24.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.982+0000 7fe0fd200700 1 -- start start 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.983+0000 7fe0fd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.983+0000 7fe0fd200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f810bde0 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.983+0000 7fe0f5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.983+0000 7fe0f5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42316/0 (socket says 172.21.15.86:42316) 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.983+0000 7fe0f5e00700 1 -- 172.21.15.86:0/321475710 learned_addr learned my addr 172.21.15.86:0/321475710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.983+0000 7fe0f5e00700 1 -- 172.21.15.86:0/321475710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f819e330 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0f5e00700 1 --2- 172.21.15.86:0/321475710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f819ddf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e8003db0 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8011630 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f819e5c0 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0e8011950 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f81a12b0 con 0x7fe0f810a700 2024-04-25T18:49:24.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.984+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe0e801a460 con 0x7fe0f810a700 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.985+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe0e801b780 con 0x7fe0f810a700 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.985+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f80640a0 con 0x7fe0f810a700 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:23.989+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe0e801b980 con 0x7fe0f810a700 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.067+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe0f802e1d0 con 0x7fe0f810a700 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.068+0000 7fe0eea00700 1 -- 172.21.15.86:0/321475710 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe0e801bb60 con 0x7fe0f810a700 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.069+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 msgr2=0x7fe0f819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.069+0000 7fe0fd200700 1 --2- 172.21.15.86:0/321475710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f819ddf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8004750 comp rx=0 tx=0).stop 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.069+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 shutdown_connections 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.069+0000 7fe0fd200700 1 --2- 172.21.15.86:0/321475710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f810a700 0x7fe0f819ddf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.069+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 >> 172.21.15.86:0/321475710 conn(0x7fe0f807dc00 msgr2=0x7fe0f8194a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.069+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 shutdown_connections 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:24.070+0000 7fe0fd200700 1 -- 172.21.15.86:0/321475710 wait complete. 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-04-25T18:49:24.763 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T18:49:25.802 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:25 smithi086 ceph-mon[27416]: from='client.? 172.21.15.86:0/1513939813' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T18:49:25.802 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:25 smithi086 ceph-mon[27416]: from='client.? 172.21.15.86:0/1513939813' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T18:49:26.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:26.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T18:49:26.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:26.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T18:49:26.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-25T18:49:26.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T18:49:26.084 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T18:49:26.084 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T18:49:26.084 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T18:49:26.084 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:26.084 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T18:49:26.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T18:49:26.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:26.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T18:49:26.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T18:49:26.085 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T18:49:26.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T18:49:26.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f561a000700 1 Processor -- start 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f561a000700 1 -- start start 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f561a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f561a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f561407c830 con 0x7f561407de40 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f5613400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:26.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f5613400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42328/0 (socket says 172.21.15.86:42328) 2024-04-25T18:49:26.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f5613400700 1 -- 172.21.15.86:0/731666733 learned_addr learned my addr 172.21.15.86:0/731666733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:26.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.343+0000 7f5613400700 1 -- 172.21.15.86:0/731666733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561407c9b0 con 0x7f561407de40 2024-04-25T18:49:26.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.344+0000 7f5613400700 1 --2- 172.21.15.86:0/731666733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561407c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5604009e80 tx=0x7f560400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25835c271021420d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:26.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.344+0000 7f5612a00700 1 -- 172.21.15.86:0/731666733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5604017070 con 0x7f561407de40 2024-04-25T18:49:26.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.344+0000 7f5612a00700 1 -- 172.21.15.86:0/731666733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5604003a20 con 0x7f561407de40 2024-04-25T18:49:26.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.344+0000 7f5612a00700 1 -- 172.21.15.86:0/731666733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5604003d20 con 0x7f561407de40 2024-04-25T18:49:26.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 -- 172.21.15.86:0/731666733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 msgr2=0x7f561407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:26.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 --2- 172.21.15.86:0/731666733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561407c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5604009e80 tx=0x7f560400b330 comp rx=0 tx=0).stop 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 -- 172.21.15.86:0/731666733 shutdown_connections 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 --2- 172.21.15.86:0/731666733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561407c2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 -- 172.21.15.86:0/731666733 >> 172.21.15.86:0/731666733 conn(0x7f5614103ab0 msgr2=0x7f5614105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 -- 172.21.15.86:0/731666733 shutdown_connections 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.345+0000 7f561a000700 1 -- 172.21.15.86:0/731666733 wait complete. 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.346+0000 7f561a000700 1 Processor -- start 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.346+0000 7f561a000700 1 -- start start 2024-04-25T18:49:26.090 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.346+0000 7f561a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:26.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.346+0000 7f561a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f561407cee0 con 0x7f561407de40 2024-04-25T18:49:26.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.346+0000 7f5613400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:26.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f5613400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42336/0 (socket says 172.21.15.86:42336) 2024-04-25T18:49:26.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f5613400700 1 -- 172.21.15.86:0/1513939813 learned_addr learned my addr 172.21.15.86:0/1513939813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:26.092 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f5613400700 1 -- 172.21.15.86:0/1513939813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561419e6d0 con 0x7f561407de40 2024-04-25T18:49:26.093 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f5613400700 1 --2- 172.21.15.86:0/1513939813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561419e190 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5604015040 tx=0x7f5604004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:26.093 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5604022020 con 0x7f561407de40 2024-04-25T18:49:26.093 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5604018040 con 0x7f561407de40 2024-04-25T18:49:26.093 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.347+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561419e960 con 0x7f561407de40 2024-04-25T18:49:26.093 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.348+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a1650 con 0x7f561407de40 2024-04-25T18:49:26.093 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.348+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5604017070 con 0x7f561407de40 2024-04-25T18:49:26.094 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.348+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5604025020 con 0x7f561407de40 2024-04-25T18:49:26.094 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.348+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f560400f050 con 0x7f561407de40 2024-04-25T18:49:26.094 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.349+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f0005360 con 0x7f561407de40 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.352+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f560402c030 con 0x7f561407de40 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.428+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f55f0002430 con 0x7f561407de40 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.436+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f560401e480 con 0x7f561407de40 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.438+0000 7f5610c00700 1 -- 172.21.15.86:0/1513939813 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5604003a20 con 0x7f561407de40 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.438+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 msgr2=0x7f561419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.438+0000 7f561a000700 1 --2- 172.21.15.86:0/1513939813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561419e190 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5604015040 tx=0x7f5604004ad0 comp rx=0 tx=0).stop 2024-04-25T18:49:26.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.439+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 shutdown_connections 2024-04-25T18:49:26.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.439+0000 7f561a000700 1 --2- 172.21.15.86:0/1513939813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561407de40 0x7f561419e190 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:26.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.439+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 >> 172.21.15.86:0/1513939813 conn(0x7f5614103ab0 msgr2=0x7f5614103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:26.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.439+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 shutdown_connections 2024-04-25T18:49:26.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:25.439+0000 7f561a000700 1 -- 172.21.15.86:0/1513939813 wait complete. 2024-04-25T18:49:26.096 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-04-25T18:49:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:26 smithi086 ceph-mon[27416]: from='client.? 172.21.15.86:0/4104527682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:30.738 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.684+0000 7fcc3b800700 1 Processor -- start 2024-04-25T18:49:30.738 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.684+0000 7fcc3b800700 1 -- start start 2024-04-25T18:49:30.738 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.684+0000 7fcc3b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc341088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.684+0000 7fcc3b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34108ec0 con 0x7fcc341084d0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.685+0000 7fcc38e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc341088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.685+0000 7fcc38e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc341088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42338/0 (socket says 172.21.15.86:42338) 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.685+0000 7fcc38e00700 1 -- 172.21.15.86:0/2331033661 learned_addr learned my addr 172.21.15.86:0/2331033661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.686+0000 7fcc38e00700 1 -- 172.21.15.86:0/2331033661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc34109680 con 0x7fcc341084d0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.686+0000 7fcc38e00700 1 --2- 172.21.15.86:0/2331033661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc341088f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc280098a0 tx=0x7fcc28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c668d91104b829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc33400700 1 -- 172.21.15.86:0/2331033661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28004160 con 0x7fcc341084d0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc33400700 1 -- 172.21.15.86:0/2331033661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc28004300 con 0x7fcc341084d0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc33400700 1 -- 172.21.15.86:0/2331033661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28011af0 con 0x7fcc341084d0 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc3b800700 1 -- 172.21.15.86:0/2331033661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 msgr2=0x7fcc341088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc3b800700 1 --2- 172.21.15.86:0/2331033661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc341088f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc280098a0 tx=0x7fcc28009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc3b800700 1 -- 172.21.15.86:0/2331033661 shutdown_connections 2024-04-25T18:49:30.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc3b800700 1 --2- 172.21.15.86:0/2331033661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc341088f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.687+0000 7fcc3b800700 1 -- 172.21.15.86:0/2331033661 >> 172.21.15.86:0/2331033661 conn(0x7fcc34103b10 msgr2=0x7fcc34105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.688+0000 7fcc3b800700 1 -- 172.21.15.86:0/2331033661 shutdown_connections 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.688+0000 7fcc3b800700 1 -- 172.21.15.86:0/2331033661 wait complete. 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc3b800700 1 Processor -- start 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc3b800700 1 -- start start 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc3b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc3419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc3b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3410ca30 con 0x7fcc341084d0 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc38e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc3419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc38e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc3419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42342/0 (socket says 172.21.15.86:42342) 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.689+0000 7fcc38e00700 1 -- 172.21.15.86:0/4104527682 learned_addr learned my addr 172.21.15.86:0/4104527682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.690+0000 7fcc38e00700 1 -- 172.21.15.86:0/4104527682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3419cc60 con 0x7fcc341084d0 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.690+0000 7fcc38e00700 1 --2- 172.21.15.86:0/4104527682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc3419c720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc28000c00 tx=0x7fcc28011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.690+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28003b20 con 0x7fcc341084d0 2024-04-25T18:49:30.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.690+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc2801a8c0 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.690+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc28024940 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.690+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3419cef0 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.691+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3419fbe0 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.691+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc28022020 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.691+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc280195c0 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.692+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc340640a0 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.695+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcc28042e50 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.770+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcc3402df80 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.771+0000 7fcc31600700 1 -- 172.21.15.86:0/4104527682 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fcc28031020 con 0x7fcc341084d0 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.772+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 msgr2=0x7fcc3419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.772+0000 7fcc3b800700 1 --2- 172.21.15.86:0/4104527682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc3419c720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc28000c00 tx=0x7fcc28011b70 comp rx=0 tx=0).stop 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.772+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 shutdown_connections 2024-04-25T18:49:30.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.772+0000 7fcc3b800700 1 --2- 172.21.15.86:0/4104527682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341084d0 0x7fcc3419c720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:30.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.772+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 >> 172.21.15.86:0/4104527682 conn(0x7fcc34103b10 msgr2=0x7fcc34104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:30.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.772+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 shutdown_connections 2024-04-25T18:49:30.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:26.773+0000 7fcc3b800700 1 -- 172.21.15.86:0/4104527682 wait complete. 2024-04-25T18:49:30.742 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-04-25T18:49:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:30 smithi086 systemd[1]: Stopping Ceph mon.a for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:49:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a[27411]: 2024-04-25T18:49:35.787+0000 7fb225e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:49:36.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a[27411]: 2024-04-25T18:49:35.787+0000 7fb225e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T18:49:36.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:35 smithi086 podman[27659]: 2024-04-25 18:49:35.962106349 +0000 UTC m=+4.815235392 container died af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:49:36.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:36 smithi086 podman[27659]: 2024-04-25 18:49:36.586187449 +0000 UTC m=+5.439316522 container cleanup af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416) 2024-04-25T18:49:36.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:36 smithi086 bash[27659]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a 2024-04-25T18:49:37.210 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:36 smithi086 podman[27673]: 2024-04-25 18:49:36.820059673 +0000 UTC m=+0.840752368 container remove af8ffc1c7142815a8994c04f148b3ad827921836e015a13486b6511ec1f61eab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T18:49:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 systemd[1]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T18:49:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 systemd[1]: Stopped Ceph mon.a for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:49:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 systemd[1]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.838s CPU time. 2024-04-25T18:49:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 systemd[1]: Starting Ceph mon.a for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:49:37.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 podman[27765]: 2024-04-25 18:49:37.517286958 +0000 UTC m=+0.034627941 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:49:37.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 podman[27765]: 2024-04-25 18:49:37.612554296 +0000 UTC m=+0.129895303 container create 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T18:49:37.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 podman[27765]: 2024-04-25 18:49:37.85795161 +0000 UTC m=+0.375292660 container init 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:49:37.869 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 podman[27765]: 2024-04-25 18:49:37.868263141 +0000 UTC m=+0.385604132 container start 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T18:49:37.993 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T18:49:38.143 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T18:49:38.143 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T18:49:38.143 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: pidfile_write: ignore empty --pid-file 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: load: jerasure load: lrc 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: RocksDB version: 7.9.2 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Git sha 0 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: DB SUMMARY 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: DB Session ID: MNZ01W78AGJ8KCY640EH 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: CURRENT file: CURRENT 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105684 ; 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.error_if_exists: 0 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.create_if_missing: 0 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.paranoid_checks: 1 2024-04-25T18:49:38.144 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.env: 0x558a589fecc0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.info_log: 0x558a8672fc40 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.statistics: (nil) 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.use_fsync: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_log_file_size: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.allow_fallocate: 1 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T18:49:38.145 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.use_direct_reads: 0 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.db_log_dir: 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.wal_dir: 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T18:49:38.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.write_buffer_manager: 0x558a866c30e0 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T18:49:38.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.unordered_write: 0 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.row_cache: None 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.wal_filter: None 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.two_write_queues: 0 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T18:49:38.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.wal_compression: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.atomic_flush: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.log_readahead_size: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_background_jobs: 2 2024-04-25T18:49:38.149 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_background_compactions: -1 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_subcompactions: 1 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_open_files: -1 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T18:49:38.150 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_background_flushes: -1 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Compression algorithms supported: 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kZSTD supported: 0 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kXpressCompression supported: 0 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kZlibCompression supported: 1 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kSnappyCompression supported: 1 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kLZ4Compression supported: 1 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: kBZip2Compression supported: 0 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T18:49:38.151 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.merge_operator: 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_filter: None 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a8672fe00) 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-04-25T18:49:38.152 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T18:49:38.153 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T18:49:38.153 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-04-25T18:49:38.153 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x558a866f51f0 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-04-25T18:49:38.154 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-04-25T18:49:38.155 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression: NoCompression 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.num_levels: 7 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T18:49:38.156 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:49:38.157 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T18:49:38.158 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T18:49:38.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.inplace_update_support: 0 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T18:49:38.160 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.bloom_locality: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.max_successive_merges: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.ttl: 2592000 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.enable_blob_files: false 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.min_blob_size: 0 2024-04-25T18:49:38.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0d2a2f89-4309-4cc4-9131-d650a081db08 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070977926413, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T18:49:38.162 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070977928280, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100791, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98679, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90897, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714070977, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0d2a2f89-4309-4cc4-9131-d650a081db08", "db_session_id": "MNZ01W78AGJ8KCY640EH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070977928432, "job": 1, "event": "recovery_finished"} 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a8680a000 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: DB pointer 0x558a867f4000 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T18:49:38.163 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 100.29 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 100.29 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:49:38.164 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 83.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 13.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 13.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x558a866f51f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T18:49:38.165 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???) e1 preinit fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).mds e1 new map 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).mds e1 print_map 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:49:38.166 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 bash[27765]: 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 systemd[1]: Started Ceph mon.a for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: monmap epoch 1 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: last_changed 2024-04-25T18:49:19.080254+0000 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: min_mon_release 19 (squid) 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: election_strategy: 1 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: fsmap 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T18:49:38.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:37 smithi086 ceph-mon[27783]: mgrmap e1: no daemons active 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.673+0000 7f453f200700 1 Processor -- start 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.674+0000 7f453f200700 1 -- start start 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.674+0000 7f453f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453810a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.674+0000 7f453f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f453810af10 con 0x7f453810a500 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.675+0000 7f453de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453810a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.675+0000 7f453de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453810a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43950/0 (socket says 172.21.15.86:43950) 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.675+0000 7f453de00700 1 -- 172.21.15.86:0/3473071016 learned_addr learned my addr 172.21.15.86:0/3473071016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:39.430 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.675+0000 7f453de00700 1 -- 172.21.15.86:0/3473071016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453810b6e0 con 0x7f453810a500 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.676+0000 7f453de00700 1 --2- 172.21.15.86:0/3473071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453810a900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f452c0098a0 tx=0x7f452c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383decb6f208cee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.676+0000 7f453ca00700 1 -- 172.21.15.86:0/3473071016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f452c004160 con 0x7f453810a500 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.676+0000 7f453ca00700 1 -- 172.21.15.86:0/3473071016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f452c004300 con 0x7f453810a500 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.676+0000 7f453ca00700 1 -- 172.21.15.86:0/3473071016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f452c011af0 con 0x7f453810a500 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 -- 172.21.15.86:0/3473071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 msgr2=0x7f453810a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 --2- 172.21.15.86:0/3473071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453810a900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f452c0098a0 tx=0x7f452c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 -- 172.21.15.86:0/3473071016 shutdown_connections 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 --2- 172.21.15.86:0/3473071016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453810a900 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 -- 172.21.15.86:0/3473071016 >> 172.21.15.86:0/3473071016 conn(0x7f453807de20 msgr2=0x7f45381084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 -- 172.21.15.86:0/3473071016 shutdown_connections 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.677+0000 7f453f200700 1 -- 172.21.15.86:0/3473071016 wait complete. 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453f200700 1 Processor -- start 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453f200700 1 -- start start 2024-04-25T18:49:39.431 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f453810ea90 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43964/0 (socket says 172.21.15.86:43964) 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.679+0000 7f453de00700 1 -- 172.21.15.86:0/3783166424 learned_addr learned my addr 172.21.15.86:0/3783166424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.680+0000 7f453de00700 1 -- 172.21.15.86:0/3783166424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453819ec90 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.680+0000 7f453de00700 1 --2- 172.21.15.86:0/3783166424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453819e750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f452c009fb0 tx=0x7f452c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.680+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f452c0036a0 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.680+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f452c01a430 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.681+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453819ef20 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.681+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f452c019550 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.681+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45381a1c10 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.681+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f452c019b90 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.681+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f452c024730 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.682+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4538069b90 con 0x7f453810a500 2024-04-25T18:49:39.432 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.685+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f452c043e50 con 0x7f453810a500 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.761+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f453802df80 con 0x7f453810a500 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.764+0000 7f4532a00700 1 -- 172.21.15.86:0/3783166424 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f452c032030 con 0x7f453810a500 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.765+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 msgr2=0x7f453819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.765+0000 7f453f200700 1 --2- 172.21.15.86:0/3783166424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453819e750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f452c009fb0 tx=0x7f452c011940 comp rx=0 tx=0).stop 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.765+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 shutdown_connections 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.765+0000 7f453f200700 1 --2- 172.21.15.86:0/3783166424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f453810a500 0x7f453819e750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.765+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 >> 172.21.15.86:0/3783166424 conn(0x7f453807de20 msgr2=0x7f4538107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.766+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 shutdown_connections 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:38.766+0000 7f453f200700 1 -- 172.21.15.86:0/3783166424 wait complete. 2024-04-25T18:49:39.433 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T18:49:39.457 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:49:39.457 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-04-25T18:49:39.458 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T18:49:39.459 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T18:49:39.890 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x 2024-04-25T18:49:39.890 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T18:49:40.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:39 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3783166424' entity='client.admin' 2024-04-25T18:49:40.140 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968.target.wants/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@.service. 2024-04-25T18:49:40.500 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:40 smithi086 systemd[1]: Starting Ceph mgr.x for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:49:40.807 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:40 smithi086 podman[27995]: 2024-04-25 18:49:40.428101485 +0000 UTC m=+0.034702888 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:49:40.807 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:40 smithi086 podman[27995]: 2024-04-25 18:49:40.534177956 +0000 UTC m=+0.140779395 container create 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:49:41.411 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:49:41.411 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-25T18:49:41.426 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:49:41.426 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-25T18:49:41.426 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-25T18:49:41.426 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T18:49:41.443 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:49:41.443 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-25T18:49:41.459 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:49:41.459 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-25T18:49:41.459 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-25T18:49:41.459 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T18:49:41.459 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-04-25T18:49:41.459 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-04-25T18:49:41.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 podman[27995]: 2024-04-25 18:49:41.281299319 +0000 UTC m=+0.887900765 container init 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T18:49:41.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 podman[27995]: 2024-04-25 18:49:41.291719053 +0000 UTC m=+0.898320462 container start 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T18:49:41.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 bash[27995]: 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a 2024-04-25T18:49:41.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 systemd[1]: Started Ceph mgr.x for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:49:41.571 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:41.522+0000 7f4548466200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:49:41.921 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:41.780+0000 7f4548466200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:49:41.921 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:41.919+0000 7f4548466200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:49:42.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:42 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1226684642' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:49:42.458 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:42.340+0000 7f4548466200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:49:42.458 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:42.456+0000 7f4548466200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:49:43.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:43.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:49:43.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T18:49:43.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:49:43.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-25T18:49:43.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:49:43.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:49:43.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.028 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:49:43.028 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:49:43.028 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:49:23.295094+0000", 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:49:43.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.112+0000 7f687d400700 1 Processor -- start 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.113+0000 7f687d400700 1 -- start start 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.113+0000 7f687d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.113+0000 7f687d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f687807c830 con 0x7f687807de40 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.114+0000 7f6876800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.114+0000 7f6876800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38058/0 (socket says 172.21.15.86:38058) 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.114+0000 7f6876800700 1 -- 172.21.15.86:0/3231100993 learned_addr learned my addr 172.21.15.86:0/3231100993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.114+0000 7f6876800700 1 -- 172.21.15.86:0/3231100993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687807c9b0 con 0x7f687807de40 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.114+0000 7f6876800700 1 --2- 172.21.15.86:0/3231100993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687807c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f68680098a0 tx=0x7f6868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed29139092f7ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.115+0000 7f6875400700 1 -- 172.21.15.86:0/3231100993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6868004160 con 0x7f687807de40 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.115+0000 7f6875400700 1 -- 172.21.15.86:0/3231100993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6868004300 con 0x7f687807de40 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.115+0000 7f6875400700 1 -- 172.21.15.86:0/3231100993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6868011af0 con 0x7f687807de40 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.115+0000 7f687d400700 1 -- 172.21.15.86:0/3231100993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 msgr2=0x7f687807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:43.030 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.115+0000 7f687d400700 1 --2- 172.21.15.86:0/3231100993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687807c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f68680098a0 tx=0x7f6868009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.115+0000 7f687d400700 1 -- 172.21.15.86:0/3231100993 shutdown_connections 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.116+0000 7f687d400700 1 --2- 172.21.15.86:0/3231100993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687807c2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.116+0000 7f687d400700 1 -- 172.21.15.86:0/3231100993 >> 172.21.15.86:0/3231100993 conn(0x7f6878103ab0 msgr2=0x7f6878105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.116+0000 7f687d400700 1 -- 172.21.15.86:0/3231100993 shutdown_connections 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.116+0000 7f687d400700 1 -- 172.21.15.86:0/3231100993 wait complete. 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.117+0000 7f687d400700 1 Processor -- start 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.117+0000 7f687d400700 1 -- start start 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.118+0000 7f687d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.118+0000 7f687d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f687810ea60 con 0x7f687807de40 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.118+0000 7f6876800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:43.031 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.118+0000 7f6876800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38068/0 (socket says 172.21.15.86:38068) 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.118+0000 7f6876800700 1 -- 172.21.15.86:0/1226684642 learned_addr learned my addr 172.21.15.86:0/1226684642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.118+0000 7f6876800700 1 -- 172.21.15.86:0/1226684642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687819eea0 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f6876800700 1 --2- 172.21.15.86:0/1226684642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687819e960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6868011ac0 tx=0x7f6868011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6868003970 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6868003b10 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687819f130 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6868024610 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781a1e20 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6868024bd0 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.119+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6868019700 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.120+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68780640a0 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.124+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6868043e50 con 0x7f687807de40 2024-04-25T18:49:43.032 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.200+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f687802df80 con 0x7f687807de40 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.201+0000 7f686f400700 1 -- 172.21.15.86:0/1226684642 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6868032030 con 0x7f687807de40 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.202+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 msgr2=0x7f687819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.202+0000 7f687d400700 1 --2- 172.21.15.86:0/1226684642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687819e960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6868011ac0 tx=0x7f6868011ba0 comp rx=0 tx=0).stop 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.203+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 shutdown_connections 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.203+0000 7f687d400700 1 --2- 172.21.15.86:0/1226684642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687807de40 0x7f687819e960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.203+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 >> 172.21.15.86:0/1226684642 conn(0x7f6878103ab0 msgr2=0x7f6878104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.203+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 shutdown_connections 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:42.203+0000 7f687d400700 1 -- 172.21.15.86:0/1226684642 wait complete. 2024-04-25T18:49:43.033 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-04-25T18:49:44.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:44.035+0000 7f4548466200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:49:44.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:44.259+0000 7f4548466200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:49:44.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:44.473+0000 7f4548466200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:49:45.187 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:44.848+0000 7f4548466200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:49:45.187 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:44.968+0000 7f4548466200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:49:45.486 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:45.230+0000 7f4548466200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:49:45.487 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:45.485+0000 7f4548466200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:49:45.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:45 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2505991437' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:49:46.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:46.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:49:46.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T18:49:46.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:49:46.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:49:46.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:49:46.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:49:46.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:49:23.295094+0000", 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.570+0000 7f7bbb800700 1 Processor -- start 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.570+0000 7f7bbb800700 1 -- start start 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.571+0000 7f7bbb800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb410b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:46.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.571+0000 7f7bbb800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb407be80 con 0x7f7bb4108f40 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.571+0000 7f7bb8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb410b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.571+0000 7f7bb8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb410b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38082/0 (socket says 172.21.15.86:38082) 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.571+0000 7f7bb8e00700 1 -- 172.21.15.86:0/3215942159 learned_addr learned my addr 172.21.15.86:0/3215942159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.572+0000 7f7bb8e00700 1 -- 172.21.15.86:0/3215942159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb410b870 con 0x7f7bb4108f40 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.572+0000 7f7bb8e00700 1 --2- 172.21.15.86:0/3215942159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb410b330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098a0 tx=0x7f7ba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd9dd3a73309a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.572+0000 7f7bb3400700 1 -- 172.21.15.86:0/3215942159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8004160 con 0x7f7bb4108f40 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.572+0000 7f7bb3400700 1 -- 172.21.15.86:0/3215942159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ba8004300 con 0x7f7bb4108f40 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.572+0000 7f7bb3400700 1 -- 172.21.15.86:0/3215942159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8011af0 con 0x7f7bb4108f40 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 -- 172.21.15.86:0/3215942159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 msgr2=0x7f7bb410b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 --2- 172.21.15.86:0/3215942159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb410b330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098a0 tx=0x7f7ba8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 -- 172.21.15.86:0/3215942159 shutdown_connections 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 --2- 172.21.15.86:0/3215942159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb410b330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 -- 172.21.15.86:0/3215942159 >> 172.21.15.86:0/3215942159 conn(0x7f7bb41029b0 msgr2=0x7f7bb4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:46.338 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 -- 172.21.15.86:0/3215942159 shutdown_connections 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.573+0000 7f7bbb800700 1 -- 172.21.15.86:0/3215942159 wait complete. 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.574+0000 7f7bbb800700 1 Processor -- start 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.575+0000 7f7bbb800700 1 -- start start 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.575+0000 7f7bbb800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.575+0000 7f7bbb800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb410c830 con 0x7f7bb4108f40 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.575+0000 7f7bb8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.575+0000 7f7bb8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38090/0 (socket says 172.21.15.86:38090) 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.575+0000 7f7bb8e00700 1 -- 172.21.15.86:0/2505991437 learned_addr learned my addr 172.21.15.86:0/2505991437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bb8e00700 1 -- 172.21.15.86:0/2505991437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb419c990 con 0x7f7bb4108f40 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bb8e00700 1 --2- 172.21.15.86:0/2505991437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb419c450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8011ac0 tx=0x7f7ba8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8003970 con 0x7f7bb4108f40 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ba8003b10 con 0x7f7bb4108f40 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8024610 con 0x7f7bb4108f40 2024-04-25T18:49:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb419cc20 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.576+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb419f910 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.577+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ba8024bd0 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.577+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ba8019700 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.578+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb40640a0 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.580+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7ba802ddf0 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.658+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7bb419fae0 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.659+0000 7f7bb1600700 1 -- 172.21.15.86:0/2505991437 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7ba801a9e0 con 0x7f7bb4108f40 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 msgr2=0x7f7bb419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 --2- 172.21.15.86:0/2505991437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb419c450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8011ac0 tx=0x7f7ba8011ba0 comp rx=0 tx=0).stop 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 shutdown_connections 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 --2- 172.21.15.86:0/2505991437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb4108f40 0x7f7bb419c450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 >> 172.21.15.86:0/2505991437 conn(0x7f7bb41029b0 msgr2=0x7f7bb4103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 shutdown_connections 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:45.660+0000 7f7bbb800700 1 -- 172.21.15.86:0/2505991437 wait complete. 2024-04-25T18:49:46.340 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (2/15)... 2024-04-25T18:49:46.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:46.325+0000 7f4548466200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:49:47.247 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:46.863+0000 7f4548466200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:49:47.248 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:46.985+0000 7f4548466200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:49:47.248 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:47.245+0000 7f4548466200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:49:48.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:47.897+0000 7f4548466200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:49:48.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:48.019+0000 7f4548466200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:49:48.609 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:48.334+0000 7f4548466200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:49:48.609 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:48.519+0000 7f4548466200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:49:48.907 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:48.646+0000 7f4548466200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:49:48.907 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:48.799+0000 7f4548466200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:49:49.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:49 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3461335001' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:49:49.482 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:49.176+0000 7f4548466200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:49:49.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:49.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:49:49.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T18:49:49.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:49:49.653 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:49:49.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:49:49.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:49:49.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.656 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:49.657 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:49:49.658 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:49:49.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:49:23.295094+0000", 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:49.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.905+0000 7f0506600700 1 Processor -- start 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.905+0000 7f0506600700 1 -- start start 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.905+0000 7f0506600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f0500107190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.905+0000 7f0506600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f050007be40 con 0x7f0500106d70 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.906+0000 7f04ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f0500107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.906+0000 7f04ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f0500107190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38094/0 (socket says 172.21.15.86:38094) 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.906+0000 7f04ff400700 1 -- 172.21.15.86:0/3980649885 learned_addr learned my addr 172.21.15.86:0/3980649885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.906+0000 7f04ff400700 1 -- 172.21.15.86:0/3980649885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05001076d0 con 0x7f0500106d70 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.907+0000 7f04ff400700 1 --2- 172.21.15.86:0/3980649885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f0500107190 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f04f00098a0 tx=0x7f04f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26107b5b70120c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.907+0000 7f04fe000700 1 -- 172.21.15.86:0/3980649885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f0004160 con 0x7f0500106d70 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.907+0000 7f04fe000700 1 -- 172.21.15.86:0/3980649885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f04f0004300 con 0x7f0500106d70 2024-04-25T18:49:49.668 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.907+0000 7f04fe000700 1 -- 172.21.15.86:0/3980649885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f0011af0 con 0x7f0500106d70 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 -- 172.21.15.86:0/3980649885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 msgr2=0x7f0500107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 --2- 172.21.15.86:0/3980649885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f0500107190 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f04f00098a0 tx=0x7f04f0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 -- 172.21.15.86:0/3980649885 shutdown_connections 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 --2- 172.21.15.86:0/3980649885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f0500107190 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 -- 172.21.15.86:0/3980649885 >> 172.21.15.86:0/3980649885 conn(0x7f05001029b0 msgr2=0x7f0500104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 -- 172.21.15.86:0/3980649885 shutdown_connections 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.908+0000 7f0506600700 1 -- 172.21.15.86:0/3980649885 wait complete. 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.909+0000 7f0506600700 1 Processor -- start 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.909+0000 7f0506600700 1 -- start start 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.909+0000 7f0506600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f050019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.909+0000 7f0506600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0500108700 con 0x7f0500106d70 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.910+0000 7f04ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f050019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:49.669 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.910+0000 7f04ff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f050019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38096/0 (socket says 172.21.15.86:38096) 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.910+0000 7f04ff400700 1 -- 172.21.15.86:0/3461335001 learned_addr learned my addr 172.21.15.86:0/3461335001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.910+0000 7f04ff400700 1 -- 172.21.15.86:0/3461335001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050019eb60 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.910+0000 7f04ff400700 1 --2- 172.21.15.86:0/3461335001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f050019e620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04f0011ac0 tx=0x7f04f0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f0003970 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f04f0003b10 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050019edf0 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f0024610 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05001a1ae0 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04f0024bd0 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.911+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04f0019700 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.912+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05000640a0 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.915+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f04f0043e50 con 0x7f0500106d70 2024-04-25T18:49:49.670 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.993+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f05000008d0 con 0x7f0500106d70 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.994+0000 7f04f7e00700 1 -- 172.21.15.86:0/3461335001 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f04f0032030 con 0x7f0500106d70 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 msgr2=0x7f050019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 --2- 172.21.15.86:0/3461335001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f050019e620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04f0011ac0 tx=0x7f04f0011ba0 comp rx=0 tx=0).stop 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 shutdown_connections 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 --2- 172.21.15.86:0/3461335001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0500106d70 0x7f050019e620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 >> 172.21.15.86:0/3461335001 conn(0x7f05001029b0 msgr2=0x7f0500195280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 shutdown_connections 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:48.995+0000 7f0506600700 1 -- 172.21.15.86:0/3461335001 wait complete. 2024-04-25T18:49:49.671 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (3/15)... 2024-04-25T18:49:50.226 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:49.944+0000 7f4548466200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:49:50.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:50.223+0000 7f4548466200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: Activating manager daemon x 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: mgrmap e2: x(active, starting, since 0.0109198s) 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: Manager daemon x is now available 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' 2024-04-25T18:49:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:50 smithi086 ceph-mon[27783]: from='mgr.14100 172.21.15.86:0/1543456780' entity='mgr.x' 2024-04-25T18:49:52.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:52 smithi086 ceph-mon[27783]: mgrmap e3: x(active, since 1.02248s) 2024-04-25T18:49:53.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:53.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:49:53.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T18:49:53.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:49:53.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:49:53.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:49:53.269 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:49:53.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:49:53.270 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:49:53.271 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:53.280 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:49:53.280 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:49:53.280 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.280 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:49:53.281 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:49:53.282 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:49:23.295094+0000", 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.305+0000 7f2ea0600700 1 Processor -- start 2024-04-25T18:49:53.283 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.305+0000 7f2ea0600700 1 -- start start 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.306+0000 7f2ea0600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.306+0000 7f2ea0600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98110a70 con 0x7f2e98110080 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.306+0000 7f2e9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.306+0000 7f2e9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46562/0 (socket says 172.21.15.86:46562) 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.306+0000 7f2e9dc00700 1 -- 172.21.15.86:0/2983110468 learned_addr learned my addr 172.21.15.86:0/2983110468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.306+0000 7f2e9dc00700 1 -- 172.21.15.86:0/2983110468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9810ece0 con 0x7f2e98110080 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.307+0000 7f2e9dc00700 1 --2- 172.21.15.86:0/2983110468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981104a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098a0 tx=0x7f2e8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdc8956d007fcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.307+0000 7f2e97e00700 1 -- 172.21.15.86:0/2983110468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e8c004160 con 0x7f2e98110080 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.307+0000 7f2e97e00700 1 -- 172.21.15.86:0/2983110468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e8c004300 con 0x7f2e98110080 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.307+0000 7f2e97e00700 1 -- 172.21.15.86:0/2983110468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e8c011af0 con 0x7f2e98110080 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.307+0000 7f2ea0600700 1 -- 172.21.15.86:0/2983110468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 msgr2=0x7f2e981104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.307+0000 7f2ea0600700 1 --2- 172.21.15.86:0/2983110468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981104a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098a0 tx=0x7f2e8c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.308+0000 7f2ea0600700 1 -- 172.21.15.86:0/2983110468 shutdown_connections 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.308+0000 7f2ea0600700 1 --2- 172.21.15.86:0/2983110468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981104a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:53.284 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.308+0000 7f2ea0600700 1 -- 172.21.15.86:0/2983110468 >> 172.21.15.86:0/2983110468 conn(0x7f2e9807dc20 msgr2=0x7f2e98080040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.308+0000 7f2ea0600700 1 -- 172.21.15.86:0/2983110468 shutdown_connections 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.308+0000 7f2ea0600700 1 -- 172.21.15.86:0/2983110468 wait complete. 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.309+0000 7f2ea0600700 1 Processor -- start 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.309+0000 7f2ea0600700 1 -- start start 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.309+0000 7f2ea0600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.309+0000 7f2ea0600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981130a0 con 0x7f2e98110080 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.309+0000 7f2e9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e9dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46572/0 (socket says 172.21.15.86:46572) 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e9dc00700 1 -- 172.21.15.86:0/106730778 learned_addr learned my addr 172.21.15.86:0/106730778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e9dc00700 1 -- 172.21.15.86:0/106730778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e981a3240 con 0x7f2e98110080 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e9dc00700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981a2d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c000c00 tx=0x7f2e8c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e8c003b20 con 0x7f2e98110080 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e8c01a8c0 con 0x7f2e98110080 2024-04-25T18:49:53.285 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e8c024940 con 0x7f2e98110080 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.310+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e981a34d0 con 0x7f2e98110080 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.311+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e981a61b0 con 0x7f2e98110080 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.312+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2e8c022020 con 0x7f2e98110080 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.312+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e980640a0 con 0x7f2e98110080 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.312+0000 7f2e96000700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f2e800395b0 0x7f2e8003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.312+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e8c04d450 con 0x7f2e98110080 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.313+0000 7f2e9d200700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f2e800395b0 0x7f2e8003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:53.288 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.313+0000 7f2e9d200700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f2e800395b0 0x7f2e8003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e84006fd0 tx=0x7f2e84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.319+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e8c00be50 con 0x7f2e98110080 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.622+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2e9802df80 con 0x7f2e98110080 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.623+0000 7f2e96000700 1 -- 172.21.15.86:0/106730778 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2e8c04d1b0 con 0x7f2e98110080 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f2e800395b0 msgr2=0x7f2e8003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f2e800395b0 0x7f2e8003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e84006fd0 tx=0x7f2e84006e40 comp rx=0 tx=0).stop 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 msgr2=0x7f2e981a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981a2d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c000c00 tx=0x7f2e8c011b70 comp rx=0 tx=0).stop 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 shutdown_connections 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f2e800395b0 0x7f2e8003ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 --2- 172.21.15.86:0/106730778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e98110080 0x7f2e981a2d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.626+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 >> 172.21.15.86:0/106730778 conn(0x7f2e9807dc20 msgr2=0x7f2e9807fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.627+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 shutdown_connections 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:52.627+0000 7f2ea0600700 1 -- 172.21.15.86:0/106730778 wait complete. 2024-04-25T18:49:53.289 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-04-25T18:49:53.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:53 smithi086 ceph-mon[27783]: mgrmap e4: x(active, since 2s) 2024-04-25T18:49:53.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:53 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/106730778' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:49:54.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:54 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1283010434' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:54.824 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.889+0000 7fd6e3400700 1 Processor -- start 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.889+0000 7fd6e3400700 1 -- start start 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.890+0000 7fd6e3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc07c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.890+0000 7fd6e3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc07c920 con 0x7fd6dc07dea0 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.890+0000 7fd6e0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc07c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:54.825 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.890+0000 7fd6e0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc07c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46578/0 (socket says 172.21.15.86:46578) 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.890+0000 7fd6e0a00700 1 -- 172.21.15.86:0/3966825518 learned_addr learned my addr 172.21.15.86:0/3966825518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.891+0000 7fd6e0a00700 1 -- 172.21.15.86:0/3966825518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc07caa0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.891+0000 7fd6e0a00700 1 --2- 172.21.15.86:0/3966825518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc07c350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00098a0 tx=0x7fd6d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0684d499db71f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.891+0000 7fd6db400700 1 -- 172.21.15.86:0/3966825518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0004160 con 0x7fd6dc07dea0 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.891+0000 7fd6db400700 1 -- 172.21.15.86:0/3966825518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6d0004300 con 0x7fd6dc07dea0 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.891+0000 7fd6db400700 1 -- 172.21.15.86:0/3966825518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0011af0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 -- 172.21.15.86:0/3966825518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 msgr2=0x7fd6dc07c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 --2- 172.21.15.86:0/3966825518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc07c350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00098a0 tx=0x7fd6d0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 -- 172.21.15.86:0/3966825518 shutdown_connections 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 --2- 172.21.15.86:0/3966825518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc07c350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 -- 172.21.15.86:0/3966825518 >> 172.21.15.86:0/3966825518 conn(0x7fd6dc103ab0 msgr2=0x7fd6dc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 -- 172.21.15.86:0/3966825518 shutdown_connections 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.892+0000 7fd6e3400700 1 -- 172.21.15.86:0/3966825518 wait complete. 2024-04-25T18:49:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.893+0000 7fd6e3400700 1 Processor -- start 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.893+0000 7fd6e3400700 1 -- start start 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.894+0000 7fd6e3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc1a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.894+0000 7fd6e3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc10a650 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.894+0000 7fd6e0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc1a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.894+0000 7fd6e0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc1a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46580/0 (socket says 172.21.15.86:46580) 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.894+0000 7fd6e0a00700 1 -- 172.21.15.86:0/1283010434 learned_addr learned my addr 172.21.15.86:0/1283010434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.894+0000 7fd6e0a00700 1 -- 172.21.15.86:0/1283010434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1a32d0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.895+0000 7fd6e0a00700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc1a2d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0011ac0 tx=0x7fd6d0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.895+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0003970 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.895+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6d0003b10 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.895+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0024610 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.895+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1a3560 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.895+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc1a6210 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.896+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fd6d001a9e0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.896+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6dc0640a0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.897+0000 7fd6d9600700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7fd6c4039600 0x7fd6c403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.897+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6d004d880 con 0x7fd6dc07dea0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.897+0000 7fd6dbe00700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7fd6c4039600 0x7fd6c403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.897+0000 7fd6dbe00700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7fd6c4039600 0x7fd6c403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:53.902+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6d000b3e0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.125+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd6dc02dfb0 con 0x7fd6dc07dea0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.128+0000 7fd6d9600700 1 -- 172.21.15.86:0/1283010434 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fd6d002c350 con 0x7fd6dc07dea0 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.131+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7fd6c4039600 msgr2=0x7fd6c403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.131+0000 7fd6e3400700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7fd6c4039600 0x7fd6c403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8006e40 comp rx=0 tx=0).stop 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.131+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 msgr2=0x7fd6dc1a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.131+0000 7fd6e3400700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc1a2d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0011ac0 tx=0x7fd6d0011ba0 comp rx=0 tx=0).stop 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.131+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 shutdown_connections 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.132+0000 7fd6e3400700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7fd6c4039600 0x7fd6c403bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.132+0000 7fd6e3400700 1 --2- 172.21.15.86:0/1283010434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6dc07dea0 0x7fd6dc1a2d90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.132+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 >> 172.21.15.86:0/1283010434 conn(0x7fd6dc103ab0 msgr2=0x7fd6dc1997e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.132+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 shutdown_connections 2024-04-25T18:49:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:54.132+0000 7fd6e3400700 1 -- 172.21.15.86:0/1283010434 wait complete. 2024-04-25T18:49:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-04-25T18:49:56.026 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:55 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1232038455' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T18:49:56.027 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: ignoring --setuser ceph since I am not root 2024-04-25T18:49:56.027 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: ignoring --setgroup ceph since I am not root 2024-04-25T18:49:56.027 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:56.024+0000 7f2071506200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:49:56.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:56.268+0000 7f2071506200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.374+0000 7f4820200700 1 Processor -- start 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.375+0000 7f4820200700 1 -- start start 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.375+0000 7f4820200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481810b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.375+0000 7f4820200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f481807be40 con 0x7f4818108f80 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.376+0000 7f481d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481810b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.376+0000 7f481d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481810b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46592/0 (socket says 172.21.15.86:46592) 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.376+0000 7f481d800700 1 -- 172.21.15.86:0/474701241 learned_addr learned my addr 172.21.15.86:0/474701241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.376+0000 7f481d800700 1 -- 172.21.15.86:0/474701241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481810b8b0 con 0x7f4818108f80 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.376+0000 7f481d800700 1 --2- 172.21.15.86:0/474701241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481810b370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f480c009e80 tx=0x7f480c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6ac59e43ffb8be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4817e00700 1 -- 172.21.15.86:0/474701241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c017070 con 0x7f4818108f80 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4817e00700 1 -- 172.21.15.86:0/474701241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f480c00b8c0 con 0x7f4818108f80 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4817e00700 1 -- 172.21.15.86:0/474701241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c004490 con 0x7f4818108f80 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4820200700 1 -- 172.21.15.86:0/474701241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 msgr2=0x7f481810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4820200700 1 --2- 172.21.15.86:0/474701241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481810b370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f480c009e80 tx=0x7f480c00b330 comp rx=0 tx=0).stop 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4820200700 1 -- 172.21.15.86:0/474701241 shutdown_connections 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4820200700 1 --2- 172.21.15.86:0/474701241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481810b370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.377+0000 7f4820200700 1 -- 172.21.15.86:0/474701241 >> 172.21.15.86:0/474701241 conn(0x7f48181029b0 msgr2=0x7f4818104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.378+0000 7f4820200700 1 -- 172.21.15.86:0/474701241 shutdown_connections 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.378+0000 7f4820200700 1 -- 172.21.15.86:0/474701241 wait complete. 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.379+0000 7f4820200700 1 Processor -- start 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.379+0000 7f4820200700 1 -- start start 2024-04-25T18:49:56.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.379+0000 7f4820200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.379+0000 7f4820200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f481810c810 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.380+0000 7f481d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.380+0000 7f481d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46600/0 (socket says 172.21.15.86:46600) 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.380+0000 7f481d800700 1 -- 172.21.15.86:0/1232038455 learned_addr learned my addr 172.21.15.86:0/1232038455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.380+0000 7f481d800700 1 -- 172.21.15.86:0/1232038455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481819c980 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.380+0000 7f481d800700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481819c440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f480c00b7b0 tx=0x7f480c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.381+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c020020 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.381+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f480c003d10 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.381+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481819cc10 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.381+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c017040 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.381+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481819f900 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.382+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48180640a0 con 0x7f4818108f80 2024-04-25T18:49:56.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.382+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f480c023020 con 0x7f4818108f80 2024-04-25T18:49:56.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.383+0000 7f4816000700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f4800039560 0x7f480003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:56.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.383+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f480c00f050 con 0x7f4818108f80 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.383+0000 7f481ce00700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f4800039560 0x7f480003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.384+0000 7f481ce00700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f4800039560 0x7f480003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4804006fd0 tx=0x7f4804006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.389+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f480c02a020 con 0x7f4818108f80 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.676+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f481819fad0 con 0x7f4818108f80 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.730+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f481819fad0 con 0x7f4818108f80 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.734+0000 7f4816000700 1 -- 172.21.15.86:0/1232038455 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f480c012950 con 0x7f4818108f80 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f4800039560 msgr2=0x7f480003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f4800039560 0x7f480003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4804006fd0 tx=0x7f4804006e40 comp rx=0 tx=0).stop 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 msgr2=0x7f481819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481819c440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f480c00b7b0 tx=0x7f480c00be90 comp rx=0 tx=0).stop 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 shutdown_connections 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f4800039560 0x7f480003ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 --2- 172.21.15.86:0/1232038455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4818108f80 0x7f481819c440 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:56.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.735+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 >> 172.21.15.86:0/1232038455 conn(0x7f48181029b0 msgr2=0x7f4818103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:56.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.736+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 shutdown_connections 2024-04-25T18:49:56.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:55.736+0000 7f4820200700 1 -- 172.21.15.86:0/1232038455 wait complete. 2024-04-25T18:49:56.710 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:56.400+0000 7f2071506200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:49:56.996 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1232038455' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T18:49:56.996 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-mon[27783]: mgrmap e5: x(active, since 5s) 2024-04-25T18:49:56.996 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:56.802+0000 7f2071506200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:49:56.996 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:56.916+0000 7f2071506200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:49:57.928 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:49:57 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2565121325' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.994+0000 7f1da3a00700 1 Processor -- start 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.995+0000 7f1da3a00700 1 -- start start 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.995+0000 7f1da3a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c107150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:57.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.995+0000 7f1da3a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c07be80 con 0x7f1d9c106d30 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.996+0000 7f1da1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c107150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.996+0000 7f1da1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c107150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46632/0 (socket says 172.21.15.86:46632) 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.996+0000 7f1da1000700 1 -- 172.21.15.86:0/3922672286 learned_addr learned my addr 172.21.15.86:0/3922672286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.996+0000 7f1da1000700 1 -- 172.21.15.86:0/3922672286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c107690 con 0x7f1d9c106d30 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.996+0000 7f1da1000700 1 --2- 172.21.15.86:0/3922672286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c107150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d900098a0 tx=0x7f1d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3857dd3d3fcf2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.997+0000 7f1d9b400700 1 -- 172.21.15.86:0/3922672286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90004160 con 0x7f1d9c106d30 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.997+0000 7f1d9b400700 1 -- 172.21.15.86:0/3922672286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1d90004300 con 0x7f1d9c106d30 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.997+0000 7f1d9b400700 1 -- 172.21.15.86:0/3922672286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90011af0 con 0x7f1d9c106d30 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 -- 172.21.15.86:0/3922672286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 msgr2=0x7f1d9c107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 --2- 172.21.15.86:0/3922672286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c107150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1d900098a0 tx=0x7f1d90009bb0 comp rx=0 tx=0).stop 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 -- 172.21.15.86:0/3922672286 shutdown_connections 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 --2- 172.21.15.86:0/3922672286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c107150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:57.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 -- 172.21.15.86:0/3922672286 >> 172.21.15.86:0/3922672286 conn(0x7f1d9c1029b0 msgr2=0x7f1d9c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 -- 172.21.15.86:0/3922672286 shutdown_connections 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.998+0000 7f1da3a00700 1 -- 172.21.15.86:0/3922672286 wait complete. 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.999+0000 7f1da3a00700 1 Processor -- start 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:56.999+0000 7f1da3a00700 1 -- start start 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da3a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da3a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1086c0 con 0x7f1d9c106d30 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46646/0 (socket says 172.21.15.86:46646) 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da1000700 1 -- 172.21.15.86:0/2565121325 learned_addr learned my addr 172.21.15.86:0/2565121325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da1000700 1 -- 172.21.15.86:0/2565121325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c19eb90 con 0x7f1d9c106d30 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.000+0000 7f1da1000700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c19e650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d90011ac0 tx=0x7f1d90011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.001+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90003970 con 0x7f1d9c106d30 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.001+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1d90003b10 con 0x7f1d9c106d30 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.001+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c19ee20 con 0x7f1d9c106d30 2024-04-25T18:49:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.001+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90024610 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.001+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c1a1b10 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.002+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f1d90024840 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.002+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c0640a0 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.002+0000 7f1d99600700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 0x7f1d8403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.002+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d9004dba0 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.003+0000 7f1d9be00700 1 -- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 msgr2=0x7f1d8403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.003+0000 7f1d9be00700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 0x7f1d8403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.008+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d90019c70 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.204+0000 7f1d9be00700 1 -- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 msgr2=0x7f1d8403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.204+0000 7f1d9be00700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 0x7f1d8403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.288+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1d9c0008d0 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.289+0000 7f1d99600700 1 -- 172.21.15.86:0/2565121325 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1d900172b0 con 0x7f1d9c106d30 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 msgr2=0x7f1d8403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:49:57.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 0x7f1d8403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 msgr2=0x7f1d9c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c19e650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d90011ac0 tx=0x7f1d90011ba0 comp rx=0 tx=0).stop 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 shutdown_connections 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f1d84039600 0x7f1d8403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 --2- 172.21.15.86:0/2565121325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9c106d30 0x7f1d9c19e650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 >> 172.21.15.86:0/2565121325 conn(0x7f1d9c1029b0 msgr2=0x7f1d9c195230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 shutdown_connections 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:57.292+0000 7f1da3a00700 1 -- 172.21.15.86:0/2565121325 wait complete. 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-25T18:49:57.945 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-04-25T18:49:58.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:58 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:58.487+0000 7f2071506200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:49:58.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:58 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:58.714+0000 7f2071506200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:49:59.301 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:58 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:58.925+0000 7f2071506200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:49:59.302 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:59 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:59.299+0000 7f2071506200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:49:59.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:59 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:59.419+0000 7f2071506200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:49:59.922 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:59 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:59.668+0000 7f2071506200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:50:00.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:49:59 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:49:59.921+0000 7f2071506200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:50:01.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:00 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:00.685+0000 7f2071506200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:50:01.524 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:01.226+0000 7f2071506200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:50:01.524 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:01.349+0000 7f2071506200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:50:01.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:01.610+0000 7f2071506200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:50:02.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:02.242+0000 7f2071506200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:50:02.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:02.363+0000 7f2071506200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:50:02.977 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:02.675+0000 7f2071506200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:50:02.977 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:02.849+0000 7f2071506200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:50:02.977 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:02.975+0000 7f2071506200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:50:03.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:03 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:03.124+0000 7f2071506200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:50:03.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:03 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:03.517+0000 7f2071506200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:50:04.541 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:04.256+0000 7f2071506200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:50:04.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:04.538+0000 7f2071506200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:50:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: Active manager daemon x restarted 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: Activating manager daemon x 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: mgrmap e6: x(active, starting, since 0.00880111s) 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: Manager daemon x is now available 2024-04-25T18:50:04.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:04 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: Found migration_current of "None". Setting to last migration. 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:05.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:05 smithi086 ceph-mon[27783]: mgrmap e7: x(active, since 1.01519s) 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.508+0000 7f758f200700 1 Processor -- start 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.508+0000 7f758f200700 1 -- start start 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.509+0000 7f758f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f758810b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.509+0000 7f758f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f758807be20 con 0x7f7588109090 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.509+0000 7f7587e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f758810b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.509+0000 7f7587e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f758810b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46650/0 (socket says 172.21.15.86:46650) 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.509+0000 7f7587e00700 1 -- 172.21.15.86:0/2384960242 learned_addr learned my addr 172.21.15.86:0/2384960242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.510+0000 7f7587e00700 1 -- 172.21.15.86:0/2384960242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758810b9c0 con 0x7f7588109090 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.510+0000 7f7587e00700 1 --2- 172.21.15.86:0/2384960242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f758810b480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f75780098a0 tx=0x7f7578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803c2ca6dc406ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.510+0000 7f7586a00700 1 -- 172.21.15.86:0/2384960242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7578004160 con 0x7f7588109090 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.510+0000 7f7586a00700 1 -- 172.21.15.86:0/2384960242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7578004300 con 0x7f7588109090 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.510+0000 7f7586a00700 1 -- 172.21.15.86:0/2384960242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7578011af0 con 0x7f7588109090 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 -- 172.21.15.86:0/2384960242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 msgr2=0x7f758810b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 --2- 172.21.15.86:0/2384960242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f758810b480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f75780098a0 tx=0x7f7578009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 -- 172.21.15.86:0/2384960242 shutdown_connections 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 --2- 172.21.15.86:0/2384960242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f758810b480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 -- 172.21.15.86:0/2384960242 >> 172.21.15.86:0/2384960242 conn(0x7f7588102a80 msgr2=0x7f7588104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 -- 172.21.15.86:0/2384960242 shutdown_connections 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.511+0000 7f758f200700 1 -- 172.21.15.86:0/2384960242 wait complete. 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.513+0000 7f758f200700 1 Processor -- start 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.513+0000 7f758f200700 1 -- start start 2024-04-25T18:50:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.513+0000 7f758f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f7588108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.513+0000 7f758f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f758810c8b0 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.514+0000 7f7587e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f7588108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.514+0000 7f7587e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f7588108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46666/0 (socket says 172.21.15.86:46666) 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.514+0000 7f7587e00700 1 -- 172.21.15.86:0/3746660149 learned_addr learned my addr 172.21.15.86:0/3746660149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.514+0000 7f7587e00700 1 -- 172.21.15.86:0/3746660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7588108e50 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.514+0000 7f7587e00700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f7588108910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7578011ac0 tx=0x7f7578011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.514+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7578003970 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.515+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7578003b10 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.515+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7588107040 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.515+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7578024610 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.515+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7588107580 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.516+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7578024890 con 0x7f7588109090 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.517+0000 7f7584c00700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.517+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 --> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f756c03c180 con 0x7f756c0395b0 2024-04-25T18:50:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.517+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f757804daa0 con 0x7f7588109090 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.517+0000 7f7587400700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 msgr2=0x7f756c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.517+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.718+0000 7f7587400700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 msgr2=0x7f756c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:58.718+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:59.119+0000 7f7587400700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 msgr2=0x7f756c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:59.119+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:59.920+0000 7f7587400700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 msgr2=0x7f756c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:49:59.920+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:01.522+0000 7f7587400700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 msgr2=0x7f756c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/777604946 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:01.522+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:04.546+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7578024b80 con 0x7f7588109090 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:04.547+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 msgr2=0x7f756c03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:50:06.166 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:04.547+0000 7f7584c00700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.551+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f757804f620 con 0x7f7588109090 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.552+0000 7f7584c00700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f756c03cea0 0x7f756c03f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.552+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7578019e50 con 0x7f756c03cea0 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.554+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f756c03cea0 0x7f756c03f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.557+0000 7f7587400700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f756c03cea0 0x7f756c03f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7574003a10 tx=0x7f7574009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.558+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7578019e50 con 0x7f756c03cea0 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.569+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7588069b90 con 0x7f756c03cea0 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.570+0000 7f7584c00700 1 -- 172.21.15.86:0/3746660149 <== mgr.14120 v2:172.21.15.86:6800/4264123395 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7588069b90 con 0x7f756c03cea0 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f756c03cea0 msgr2=0x7f756c03f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f756c03cea0 0x7f756c03f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7574003a10 tx=0x7f7574009290 comp rx=0 tx=0).stop 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 msgr2=0x7f7588108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f7588108910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7578011ac0 tx=0x7f7578011ba0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.167 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 shutdown_connections 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f756c03cea0 0x7f756c03f290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588109090 0x7f7588108910 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 --2- 172.21.15.86:0/3746660149 >> [v2:172.21.15.86:6800/777604946,v1:172.21.15.86:6801/777604946] conn(0x7f756c0395b0 0x7f756c03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 >> 172.21.15.86:0/3746660149 conn(0x7f7588102a80 msgr2=0x7f75881035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 shutdown_connections 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:05.571+0000 7f758f200700 1 -- 172.21.15.86:0/3746660149 wait complete. 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-04-25T18:50:06.168 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-04-25T18:50:06.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:06 smithi086 ceph-mon[27783]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T18:50:06.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:06 smithi086 ceph-mon[27783]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T18:50:06.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:06 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:06] ENGINE Bus STARTING 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:06] ENGINE Serving on http://172.21.15.86:8765 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:06] ENGINE Serving on https://172.21.15.86:7150 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:06] ENGINE Bus STARTED 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: mgrmap e8: x(active, since 2s) 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:07.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:07 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:07.734 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.816+0000 7ff5f8a00700 1 Processor -- start 2024-04-25T18:50:07.734 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.816+0000 7ff5f8a00700 1 -- start start 2024-04-25T18:50:07.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.817+0000 7ff5f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.739 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.817+0000 7ff5f8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4110a70 con 0x7ff5f4110080 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.817+0000 7ff5f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.817+0000 7ff5f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41318/0 (socket says 172.21.15.86:41318) 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.817+0000 7ff5f1e00700 1 -- 172.21.15.86:0/3602777660 learned_addr learned my addr 172.21.15.86:0/3602777660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.818+0000 7ff5f1e00700 1 -- 172.21.15.86:0/3602777660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f410ece0 con 0x7ff5f4110080 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.818+0000 7ff5f1e00700 1 --2- 172.21.15.86:0/3602777660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41104a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098a0 tx=0x7ff5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2d192996eb2c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.818+0000 7ff5f1400700 1 -- 172.21.15.86:0/3602777660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4004160 con 0x7ff5f4110080 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f1400700 1 -- 172.21.15.86:0/3602777660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff5e4004300 con 0x7ff5f4110080 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f1400700 1 -- 172.21.15.86:0/3602777660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4011af0 con 0x7ff5f4110080 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 -- 172.21.15.86:0/3602777660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 msgr2=0x7ff5f41104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 --2- 172.21.15.86:0/3602777660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41104a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098a0 tx=0x7ff5e4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 -- 172.21.15.86:0/3602777660 shutdown_connections 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 --2- 172.21.15.86:0/3602777660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41104a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.740 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 -- 172.21.15.86:0/3602777660 >> 172.21.15.86:0/3602777660 conn(0x7ff5f407dc20 msgr2=0x7ff5f4080040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 -- 172.21.15.86:0/3602777660 shutdown_connections 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.819+0000 7ff5f8a00700 1 -- 172.21.15.86:0/3602777660 wait complete. 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.820+0000 7ff5f8a00700 1 Processor -- start 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f8a00700 1 -- start start 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41130a0 con 0x7ff5f4110080 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41324/0 (socket says 172.21.15.86:41324) 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f1e00700 1 -- 172.21.15.86:0/1712426084 learned_addr learned my addr 172.21.15.86:0/1712426084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.821+0000 7ff5f1e00700 1 -- 172.21.15.86:0/1712426084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f41a3240 con 0x7ff5f4110080 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.822+0000 7ff5f1e00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41a2d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000c00 tx=0x7ff5e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.822+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4003b20 con 0x7ff5f4110080 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.822+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff5e401a8c0 con 0x7ff5f4110080 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.822+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e4024940 con 0x7ff5f4110080 2024-04-25T18:50:07.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.822+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f41a34d0 con 0x7ff5f4110080 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.822+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f41a61b0 con 0x7ff5f4110080 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.823+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff5e4022020 con 0x7ff5f4110080 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.824+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f40640a0 con 0x7ff5f4110080 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.824+0000 7ff5eaa00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7ff5d8039600 0x7ff5d803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.824+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5e404d7e0 con 0x7ff5f4110080 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.824+0000 7ff5ebe00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7ff5d8039600 0x7ff5d803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.825+0000 7ff5ebe00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7ff5d8039600 0x7ff5d803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc006fd0 tx=0x7ff5dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:06.830+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5e400be50 con 0x7ff5f4110080 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.075+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff5f402dff0 con 0x7ff5d8039600 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.085+0000 7ff5eaa00700 1 -- 172.21.15.86:0/1712426084 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff5f402dff0 con 0x7ff5d8039600 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7ff5d8039600 msgr2=0x7ff5d803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7ff5d8039600 0x7ff5d803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc006fd0 tx=0x7ff5dc006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 msgr2=0x7ff5f41a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41a2d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000c00 tx=0x7ff5e4011b70 comp rx=0 tx=0).stop 2024-04-25T18:50:07.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 shutdown_connections 2024-04-25T18:50:07.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7ff5d8039600 0x7ff5d803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.088+0000 7ff5f8a00700 1 --2- 172.21.15.86:0/1712426084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5f4110080 0x7ff5f41a2d00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.089+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 >> 172.21.15.86:0/1712426084 conn(0x7ff5f407dc20 msgr2=0x7ff5f407fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.089+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 shutdown_connections 2024-04-25T18:50:07.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:07.089+0000 7ff5f8a00700 1 -- 172.21.15.86:0/1712426084 wait complete. 2024-04-25T18:50:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:08 smithi086 ceph-mon[27783]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.293+0000 7f7bf9a00700 1 Processor -- start 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.293+0000 7f7bf9a00700 1 -- start start 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.293+0000 7f7bf9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf410ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.293+0000 7f7bf9a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf410b0d0 con 0x7f7bf410a6e0 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.294+0000 7f7bf2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf410ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.294+0000 7f7bf2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf410ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41340/0 (socket says 172.21.15.86:41340) 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.294+0000 7f7bf2a00700 1 -- 172.21.15.86:0/434248099 learned_addr learned my addr 172.21.15.86:0/434248099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.295+0000 7f7bf2a00700 1 -- 172.21.15.86:0/434248099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf410b890 con 0x7f7bf410a6e0 2024-04-25T18:50:09.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.295+0000 7f7bf2a00700 1 --2- 172.21.15.86:0/434248099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf410ab00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7be40098a0 tx=0x7f7be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0590bb66ac18bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.295+0000 7f7bf1600700 1 -- 172.21.15.86:0/434248099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be4004160 con 0x7f7bf410a6e0 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.295+0000 7f7bf1600700 1 -- 172.21.15.86:0/434248099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7be4004300 con 0x7f7bf410a6e0 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.295+0000 7f7bf1600700 1 -- 172.21.15.86:0/434248099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be4011af0 con 0x7f7bf410a6e0 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 -- 172.21.15.86:0/434248099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 msgr2=0x7f7bf410ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 --2- 172.21.15.86:0/434248099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf410ab00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7be40098a0 tx=0x7f7be4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 -- 172.21.15.86:0/434248099 shutdown_connections 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 --2- 172.21.15.86:0/434248099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf410ab00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 -- 172.21.15.86:0/434248099 >> 172.21.15.86:0/434248099 conn(0x7f7bf407dc20 msgr2=0x7f7bf41084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 -- 172.21.15.86:0/434248099 shutdown_connections 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.296+0000 7f7bf9a00700 1 -- 172.21.15.86:0/434248099 wait complete. 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.297+0000 7f7bf9a00700 1 Processor -- start 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf9a00700 1 -- start start 2024-04-25T18:50:09.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf9a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf410ec40 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41356/0 (socket says 172.21.15.86:41356) 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf2a00700 1 -- 172.21.15.86:0/957385036 learned_addr learned my addr 172.21.15.86:0/957385036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.298+0000 7f7bf2a00700 1 -- 172.21.15.86:0/957385036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf419ee30 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.299+0000 7f7bf2a00700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf419e8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.299+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be4003b20 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.299+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7be401a8c0 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.299+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf419f0c0 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.299+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7be4024940 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.299+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf41a1db0 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.300+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7be4022020 con 0x7f7bf410a6e0 2024-04-25T18:50:09.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.300+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf4069b90 con 0x7f7bf410a6e0 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.301+0000 7f7beb400700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f7bd8039600 0x7f7bd803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.301+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7be404d850 con 0x7f7bf410a6e0 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.301+0000 7f7bf2000700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f7bd8039600 0x7f7bd803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.302+0000 7f7bf2000700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f7bd8039600 0x7f7bd803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7be0006fd0 tx=0x7f7be0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.307+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7be40042a0 con 0x7f7bf410a6e0 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.534+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7bf402dff0 con 0x7f7bd8039600 2024-04-25T18:50:09.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.536+0000 7f7beb400700 1 -- 172.21.15.86:0/957385036 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7bf402dff0 con 0x7f7bd8039600 2024-04-25T18:50:09.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.538+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f7bd8039600 msgr2=0x7f7bd803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:09.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.538+0000 7f7bf9a00700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f7bd8039600 0x7f7bd803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7be0006fd0 tx=0x7f7be0006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:09.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.538+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 msgr2=0x7f7bf419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:09.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.538+0000 7f7bf9a00700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf419e8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4011b70 comp rx=0 tx=0).stop 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.539+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 shutdown_connections 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.539+0000 7f7bf9a00700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f7bd8039600 0x7f7bd803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.539+0000 7f7bf9a00700 1 --2- 172.21.15.86:0/957385036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf410a6e0 0x7f7bf419e8f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.539+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 >> 172.21.15.86:0/957385036 conn(0x7f7bf407dc20 msgr2=0x7f7bf4107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.539+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 shutdown_connections 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:08.539+0000 7f7bf9a00700 1 -- 172.21.15.86:0/957385036 wait complete. 2024-04-25T18:50:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-04-25T18:50:09.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:09 smithi086 ceph-mon[27783]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:10.672 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: Generating public/private rsa key pair. 2024-04-25T18:50:10.672 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: Your identification has been saved in /tmp/tmp7lanv7j5/key. 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: Your public key has been saved in /tmp/tmp7lanv7j5/key.pub. 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: The key fingerprint is: 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: SHA256:mx4o1FWxV9pShdIFlNAIkFCOb/5CyaP0pBrlNSm9YBE ceph-1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: The key's randomart image is: 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: +---[RSA 3072]----+ 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | Eooo+o.*=*o| 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | +.. .o== | 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | o o . +.. | 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | . = . . . | 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | . *.S. | 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | . +.B*= | 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | o.o**. | 2024-04-25T18:50:10.673 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | oo.oo | 2024-04-25T18:50:10.674 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: | .. ... | 2024-04-25T18:50:10.674 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: +----[SHA256]-----+ 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.726+0000 7fe271000700 1 Processor -- start 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.726+0000 7fe271000700 1 -- start start 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.727+0000 7fe271000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c07c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.727+0000 7fe271000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c07c920 con 0x7fe26c07dea0 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.727+0000 7fe269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c07c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.727+0000 7fe269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c07c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41364/0 (socket says 172.21.15.86:41364) 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.727+0000 7fe269e00700 1 -- 172.21.15.86:0/4016984371 learned_addr learned my addr 172.21.15.86:0/4016984371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.728+0000 7fe269e00700 1 -- 172.21.15.86:0/4016984371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c07caa0 con 0x7fe26c07dea0 2024-04-25T18:50:10.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.728+0000 7fe269e00700 1 --2- 172.21.15.86:0/4016984371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c07c350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009e80 tx=0x7fe25c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aa9602a2144664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.729+0000 7fe269400700 1 -- 172.21.15.86:0/4016984371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c017070 con 0x7fe26c07dea0 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.729+0000 7fe269400700 1 -- 172.21.15.86:0/4016984371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe25c00b8c0 con 0x7fe26c07dea0 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.729+0000 7fe269400700 1 -- 172.21.15.86:0/4016984371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c004490 con 0x7fe26c07dea0 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 -- 172.21.15.86:0/4016984371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 msgr2=0x7fe26c07c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 --2- 172.21.15.86:0/4016984371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c07c350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009e80 tx=0x7fe25c00b330 comp rx=0 tx=0).stop 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 -- 172.21.15.86:0/4016984371 shutdown_connections 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 --2- 172.21.15.86:0/4016984371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c07c350 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 -- 172.21.15.86:0/4016984371 >> 172.21.15.86:0/4016984371 conn(0x7fe26c103ae0 msgr2=0x7fe26c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 -- 172.21.15.86:0/4016984371 shutdown_connections 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.730+0000 7fe271000700 1 -- 172.21.15.86:0/4016984371 wait complete. 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.731+0000 7fe271000700 1 Processor -- start 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.731+0000 7fe271000700 1 -- start start 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.732+0000 7fe271000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c1a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.732+0000 7fe271000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10a640 con 0x7fe26c07dea0 2024-04-25T18:50:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.732+0000 7fe269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c1a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.732+0000 7fe269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c1a2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41378/0 (socket says 172.21.15.86:41378) 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.732+0000 7fe269e00700 1 -- 172.21.15.86:0/3366799212 learned_addr learned my addr 172.21.15.86:0/3366799212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.732+0000 7fe269e00700 1 -- 172.21.15.86:0/3366799212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1a3250 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.733+0000 7fe269e00700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c1a2d10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00b7b0 tx=0x7fe25c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.733+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c020020 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.733+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe25c003d10 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.733+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25c017040 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.733+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c1a34e0 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.733+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1a61c0 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.734+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe25c023020 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.734+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0640a0 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.735+0000 7fe263400700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fe250039600 0x7fe25003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.735+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe25c00f050 con 0x7fe26c07dea0 2024-04-25T18:50:10.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.735+0000 7fe261400700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fe250039600 0x7fe25003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.735+0000 7fe261400700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fe250039600 0x7fe25003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe254006fd0 tx=0x7fe254006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.740+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe25c02a020 con 0x7fe26c07dea0 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:09.966+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe26c02dff0 con 0x7fe250039600 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.345+0000 7fe263400700 1 -- 172.21.15.86:0/3366799212 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe26c02dff0 con 0x7fe250039600 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.349+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fe250039600 msgr2=0x7fe25003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.349+0000 7fe271000700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fe250039600 0x7fe25003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe254006fd0 tx=0x7fe254006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.349+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 msgr2=0x7fe26c1a2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.349+0000 7fe271000700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c1a2d10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00b7b0 tx=0x7fe25c00be90 comp rx=0 tx=0).stop 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.350+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 shutdown_connections 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.350+0000 7fe271000700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fe250039600 0x7fe25003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.350+0000 7fe271000700 1 --2- 172.21.15.86:0/3366799212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe26c07dea0 0x7fe26c1a2d10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.350+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 >> 172.21.15.86:0/3366799212 conn(0x7fe26c103ae0 msgr2=0x7fe26c1997d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.350+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 shutdown_connections 2024-04-25T18:50:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:10.350+0000 7fe271000700 1 -- 172.21.15.86:0/3366799212 wait complete. 2024-04-25T18:50:11.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:11 smithi086 ceph-mon[27783]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:11 smithi086 ceph-mon[27783]: Generating ssh key... 2024-04-25T18:50:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:11 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:11 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:12.489 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:12 smithi086 ceph-mon[27783]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:12.503 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.578+0000 7f38ea600700 1 Processor -- start 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.578+0000 7f38ea600700 1 -- start start 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.579+0000 7f38ea600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e410b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.579+0000 7f38ea600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38e407bea0 con 0x7f38e4108f20 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.579+0000 7f38e3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e410b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.579+0000 7f38e3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e410b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47002/0 (socket says 172.21.15.86:47002) 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.579+0000 7f38e3400700 1 -- 172.21.15.86:0/215383181 learned_addr learned my addr 172.21.15.86:0/215383181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.579+0000 7f38e3400700 1 -- 172.21.15.86:0/215383181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e410b850 con 0x7f38e4108f20 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.580+0000 7f38e3400700 1 --2- 172.21.15.86:0/215383181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e410b310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38d40098a0 tx=0x7f38d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70ef2197037c06c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.580+0000 7f38e2000700 1 -- 172.21.15.86:0/215383181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d4004160 con 0x7f38e4108f20 2024-04-25T18:50:12.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.580+0000 7f38e2000700 1 -- 172.21.15.86:0/215383181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f38d4004300 con 0x7f38e4108f20 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.580+0000 7f38e2000700 1 -- 172.21.15.86:0/215383181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d4011af0 con 0x7f38e4108f20 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 -- 172.21.15.86:0/215383181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 msgr2=0x7f38e410b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 --2- 172.21.15.86:0/215383181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e410b310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38d40098a0 tx=0x7f38d4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 -- 172.21.15.86:0/215383181 shutdown_connections 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 --2- 172.21.15.86:0/215383181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e410b310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 -- 172.21.15.86:0/215383181 >> 172.21.15.86:0/215383181 conn(0x7f38e41029b0 msgr2=0x7f38e4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 -- 172.21.15.86:0/215383181 shutdown_connections 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.581+0000 7f38ea600700 1 -- 172.21.15.86:0/215383181 wait complete. 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.582+0000 7f38ea600700 1 Processor -- start 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.583+0000 7f38ea600700 1 -- start start 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.583+0000 7f38ea600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e41087a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:12.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.583+0000 7f38ea600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38e410c7a0 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.583+0000 7f38e3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e41087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.583+0000 7f38e3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e41087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47004/0 (socket says 172.21.15.86:47004) 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.583+0000 7f38e3400700 1 -- 172.21.15.86:0/636560651 learned_addr learned my addr 172.21.15.86:0/636560651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38e3400700 1 -- 172.21.15.86:0/636560651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e4108ce0 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38e3400700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e41087a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38d4011ac0 tx=0x7f38d4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d4003970 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f38d4003b10 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d4024610 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e4106ed0 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.584+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e4107410 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.585+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f38d401a9e0 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.586+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e40640a0 con 0x7f38e4108f20 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.586+0000 7f38dbe00700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f38c8039600 0x7f38c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:12.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.586+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f38d404d910 con 0x7f38e4108f20 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.586+0000 7f38e2a00700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f38c8039600 0x7f38c803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.587+0000 7f38e2a00700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f38c8039600 0x7f38c803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38d0006fd0 tx=0x7f38d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.592+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38d400b3e0 con 0x7f38e4108f20 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.829+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f38e4103c20 con 0x7f38c8039600 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.830+0000 7f38dbe00700 1 -- 172.21.15.86:0/636560651 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f38e4103c20 con 0x7f38c8039600 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f38c8039600 msgr2=0x7f38c803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f38c8039600 0x7f38c803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38d0006fd0 tx=0x7f38d0006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 msgr2=0x7f38e41087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e41087a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38d4011ac0 tx=0x7f38d4011ba0 comp rx=0 tx=0).stop 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 shutdown_connections 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f38c8039600 0x7f38c803bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 --2- 172.21.15.86:0/636560651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4108f20 0x7f38e41087a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 >> 172.21.15.86:0/636560651 conn(0x7f38e41029b0 msgr2=0x7f38e4103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 shutdown_connections 2024-04-25T18:50:12.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:11.833+0000 7f38ea600700 1 -- 172.21.15.86:0/636560651 wait complete. 2024-04-25T18:50:12.513 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T18:50:12.513 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T18:50:12.517 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-04-25T18:50:14.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:13 smithi086 ceph-mon[27783]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:15.127 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:14 smithi086 ceph-mon[27783]: Deploying cephadm binary to smithi086 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.308+0000 7f4dc5800700 1 Processor -- start 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.308+0000 7f4dc5800700 1 -- start start 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.308+0000 7f4dc5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.309+0000 7f4dc5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc007c830 con 0x7f4dc007de40 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.309+0000 7f4dbea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:20.070 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.309+0000 7f4dbea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47010/0 (socket says 172.21.15.86:47010) 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.309+0000 7f4dbea00700 1 -- 172.21.15.86:0/3055362083 learned_addr learned my addr 172.21.15.86:0/3055362083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.310+0000 7f4dbea00700 1 -- 172.21.15.86:0/3055362083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc007c9b0 con 0x7f4dc007de40 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.310+0000 7f4dbea00700 1 --2- 172.21.15.86:0/3055362083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc007c2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4db00098a0 tx=0x7f4db0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ab957291163642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.310+0000 7f4dbd600700 1 -- 172.21.15.86:0/3055362083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db0004160 con 0x7f4dc007de40 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.310+0000 7f4dbd600700 1 -- 172.21.15.86:0/3055362083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4db0004300 con 0x7f4dc007de40 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.310+0000 7f4dbd600700 1 -- 172.21.15.86:0/3055362083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db0011af0 con 0x7f4dc007de40 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 -- 172.21.15.86:0/3055362083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 msgr2=0x7f4dc007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 --2- 172.21.15.86:0/3055362083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc007c2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4db00098a0 tx=0x7f4db0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 -- 172.21.15.86:0/3055362083 shutdown_connections 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 --2- 172.21.15.86:0/3055362083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc007c2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 -- 172.21.15.86:0/3055362083 >> 172.21.15.86:0/3055362083 conn(0x7f4dc0103ab0 msgr2=0x7f4dc0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 -- 172.21.15.86:0/3055362083 shutdown_connections 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.311+0000 7f4dc5800700 1 -- 172.21.15.86:0/3055362083 wait complete. 2024-04-25T18:50:20.071 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.312+0000 7f4dc5800700 1 Processor -- start 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.313+0000 7f4dc5800700 1 -- start start 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.313+0000 7f4dc5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.313+0000 7f4dc5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc010ea60 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.313+0000 7f4dbea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.313+0000 7f4dbea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47022/0 (socket says 172.21.15.86:47022) 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.313+0000 7f4dbea00700 1 -- 172.21.15.86:0/3714126835 learned_addr learned my addr 172.21.15.86:0/3714126835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4dbea00700 1 -- 172.21.15.86:0/3714126835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc019eea0 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4dbea00700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc019e960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4db0011ac0 tx=0x7f4db0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db0003970 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4db0003b10 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db0024610 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc019f130 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.314+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc01a1e20 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.316+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc00640a0 con 0x7f4dc007de40 2024-04-25T18:50:20.072 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.316+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4db001a9e0 con 0x7f4dc007de40 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.316+0000 7f4db7400700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f4da4039600 0x7f4da403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.316+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4db004d9b0 con 0x7f4dc007de40 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.316+0000 7f4dbe000700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f4da4039600 0x7f4da403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.317+0000 7f4dbe000700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f4da4039600 0x7f4da403bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006fd0 tx=0x7f4dac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.323+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4db000b3e0 con 0x7f4dc007de40 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:13.550+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) -- 0x7f4dc002dff0 con 0x7f4da4039600 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.468+0000 7f4db7400700 1 -- 172.21.15.86:0/3714126835 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4dc002dff0 con 0x7f4da4039600 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f4da4039600 msgr2=0x7f4da403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f4da4039600 0x7f4da403bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006fd0 tx=0x7f4dac006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 msgr2=0x7f4dc019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc019e960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4db0011ac0 tx=0x7f4db0011ba0 comp rx=0 tx=0).stop 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 shutdown_connections 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f4da4039600 0x7f4da403bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 --2- 172.21.15.86:0/3714126835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dc007de40 0x7f4dc019e960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:20.073 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.472+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 >> 172.21.15.86:0/3714126835 conn(0x7f4dc0103ab0 msgr2=0x7f4dc0104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:20.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.473+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 shutdown_connections 2024-04-25T18:50:20.074 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:19.473+0000 7f4dc5800700 1 -- 172.21.15.86:0/3714126835 wait complete. 2024-04-25T18:50:20.074 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-04-25T18:50:20.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:20 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:20.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:20 smithi086 ceph-mon[27783]: Added host smithi086 2024-04-25T18:50:20.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:20 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.660+0000 7fbd43600700 1 Processor -- start 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.660+0000 7fbd43600700 1 -- start start 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.660+0000 7fbd43600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.660+0000 7fbd43600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c07c830 con 0x7fbd3c07de40 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.661+0000 7fbd40c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.661+0000 7fbd40c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57672/0 (socket says 172.21.15.86:57672) 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.661+0000 7fbd40c00700 1 -- 172.21.15.86:0/917537755 learned_addr learned my addr 172.21.15.86:0/917537755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.662+0000 7fbd40c00700 1 -- 172.21.15.86:0/917537755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c07c9b0 con 0x7fbd3c07de40 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.662+0000 7fbd40c00700 1 --2- 172.21.15.86:0/917537755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c07c2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd300098a0 tx=0x7fbd30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20313fec81f3242f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.662+0000 7fbd3b400700 1 -- 172.21.15.86:0/917537755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30004160 con 0x7fbd3c07de40 2024-04-25T18:50:21.759 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.662+0000 7fbd3b400700 1 -- 172.21.15.86:0/917537755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd30004300 con 0x7fbd3c07de40 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.662+0000 7fbd3b400700 1 -- 172.21.15.86:0/917537755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30011af0 con 0x7fbd3c07de40 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 -- 172.21.15.86:0/917537755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 msgr2=0x7fbd3c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 --2- 172.21.15.86:0/917537755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c07c2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd300098a0 tx=0x7fbd30009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 -- 172.21.15.86:0/917537755 shutdown_connections 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 --2- 172.21.15.86:0/917537755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c07c2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 -- 172.21.15.86:0/917537755 >> 172.21.15.86:0/917537755 conn(0x7fbd3c103ab0 msgr2=0x7fbd3c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 -- 172.21.15.86:0/917537755 shutdown_connections 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.663+0000 7fbd43600700 1 -- 172.21.15.86:0/917537755 wait complete. 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.664+0000 7fbd43600700 1 Processor -- start 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.665+0000 7fbd43600700 1 -- start start 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.665+0000 7fbd43600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:21.760 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.665+0000 7fbd43600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c10ea60 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.665+0000 7fbd40c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.665+0000 7fbd40c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57686/0 (socket says 172.21.15.86:57686) 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.665+0000 7fbd40c00700 1 -- 172.21.15.86:0/1462917228 learned_addr learned my addr 172.21.15.86:0/1462917228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.666+0000 7fbd40c00700 1 -- 172.21.15.86:0/1462917228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c19eea0 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.666+0000 7fbd40c00700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c19e960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd30011ac0 tx=0x7fbd30011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.666+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30003970 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.666+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd30003b10 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.666+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c19f130 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.666+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30024610 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.667+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a1e20 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.668+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbd3001a430 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.668+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c0640a0 con 0x7fbd3c07de40 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.668+0000 7fbd39600700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fbd240395b0 0x7fbd2403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:21.761 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.668+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd3004d950 con 0x7fbd3c07de40 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.669+0000 7fbd3be00700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fbd240395b0 0x7fbd2403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.669+0000 7fbd3be00700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fbd240395b0 0x7fbd2403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006fd0 tx=0x7fbd28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.675+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd30016260 con 0x7fbd3c07de40 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.920+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbd3c02dff0 con 0x7fbd240395b0 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.928+0000 7fbd39600700 1 -- 172.21.15.86:0/1462917228 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbd3c02dff0 con 0x7fbd240395b0 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.931+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fbd240395b0 msgr2=0x7fbd2403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.931+0000 7fbd43600700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fbd240395b0 0x7fbd2403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006fd0 tx=0x7fbd28006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.931+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 msgr2=0x7fbd3c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.931+0000 7fbd43600700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c19e960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd30011ac0 tx=0x7fbd30011ba0 comp rx=0 tx=0).stop 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.932+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 shutdown_connections 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.932+0000 7fbd43600700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fbd240395b0 0x7fbd2403ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:21.762 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.932+0000 7fbd43600700 1 --2- 172.21.15.86:0/1462917228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c07de40 0x7fbd3c19e960 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:21.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.932+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 >> 172.21.15.86:0/1462917228 conn(0x7fbd3c103ab0 msgr2=0x7fbd3c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:21.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.932+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 shutdown_connections 2024-04-25T18:50:21.763 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:20.932+0000 7fbd43600700 1 -- 172.21.15.86:0/1462917228 wait complete. 2024-04-25T18:50:21.763 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-04-25T18:50:22.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:21 smithi086 ceph-mon[27783]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:22.036 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:21 smithi086 ceph-mon[27783]: Saving service mon spec with placement count:5 2024-04-25T18:50:22.036 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:21 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:23.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T18:50:23.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.558+0000 7f9177e00700 1 Processor -- start 2024-04-25T18:50:23.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.558+0000 7f9177e00700 1 -- start start 2024-04-25T18:50:23.870 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.559+0000 7f9177e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917010ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.559+0000 7f9177e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f917010b0d0 con 0x7f917010a6e0 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.559+0000 7f9175400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917010ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.559+0000 7f9175400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917010ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57698/0 (socket says 172.21.15.86:57698) 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.559+0000 7f9175400700 1 -- 172.21.15.86:0/2118359018 learned_addr learned my addr 172.21.15.86:0/2118359018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.560+0000 7f9175400700 1 -- 172.21.15.86:0/2118359018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917010b890 con 0x7f917010a6e0 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.560+0000 7f9175400700 1 --2- 172.21.15.86:0/2118359018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917010ab00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91640098a0 tx=0x7f9164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4152ec46511dece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.560+0000 7f916fe00700 1 -- 172.21.15.86:0/2118359018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9164004160 con 0x7f917010a6e0 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f916fe00700 1 -- 172.21.15.86:0/2118359018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9164004300 con 0x7f917010a6e0 2024-04-25T18:50:23.871 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f916fe00700 1 -- 172.21.15.86:0/2118359018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9164011af0 con 0x7f917010a6e0 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 -- 172.21.15.86:0/2118359018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 msgr2=0x7f917010ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 --2- 172.21.15.86:0/2118359018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917010ab00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91640098a0 tx=0x7f9164009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 -- 172.21.15.86:0/2118359018 shutdown_connections 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 --2- 172.21.15.86:0/2118359018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917010ab00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 -- 172.21.15.86:0/2118359018 >> 172.21.15.86:0/2118359018 conn(0x7f917007dc20 msgr2=0x7f91701084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 -- 172.21.15.86:0/2118359018 shutdown_connections 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.561+0000 7f9177e00700 1 -- 172.21.15.86:0/2118359018 wait complete. 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.562+0000 7f9177e00700 1 Processor -- start 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9177e00700 1 -- start start 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9177e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:23.872 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9177e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f917010ec40 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9175400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9175400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57700/0 (socket says 172.21.15.86:57700) 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9175400700 1 -- 172.21.15.86:0/295590370 learned_addr learned my addr 172.21.15.86:0/295590370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.563+0000 7f9175400700 1 -- 172.21.15.86:0/295590370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917019ee30 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.564+0000 7f9175400700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917019e8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9164000c00 tx=0x7f9164011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.564+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9164003b20 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.564+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f916401a8c0 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.564+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9164024940 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.564+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917019f0c0 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.564+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701a1db0 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.565+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91700640a0 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.566+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9164022020 con 0x7f917010a6e0 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.566+0000 7f916e000700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f9158039600 0x7f915803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:23.873 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.566+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f916404d7f0 con 0x7f917010a6e0 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.567+0000 7f9174a00700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f9158039600 0x7f915803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.567+0000 7f9174a00700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f9158039600 0x7f915803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f915c006fd0 tx=0x7f915c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.572+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91640212c0 con 0x7f917010a6e0 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.814+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f917002dff0 con 0x7f9158039600 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.825+0000 7f916e000700 1 -- 172.21.15.86:0/295590370 <== mgr.14120 v2:172.21.15.86:6800/4264123395 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f917002dff0 con 0x7f9158039600 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f9158039600 msgr2=0x7f915803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f9158039600 0x7f915803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f915c006fd0 tx=0x7f915c006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 msgr2=0x7f917019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917019e8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9164000c00 tx=0x7f9164011b70 comp rx=0 tx=0).stop 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 shutdown_connections 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f9158039600 0x7f915803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 --2- 172.21.15.86:0/295590370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917010a6e0 0x7f917019e8f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 >> 172.21.15.86:0/295590370 conn(0x7f917007dc20 msgr2=0x7f9170107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:23.874 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 shutdown_connections 2024-04-25T18:50:23.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:22.828+0000 7f9177e00700 1 -- 172.21.15.86:0/295590370 wait complete. 2024-04-25T18:50:24.183 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:23 smithi086 ceph-mon[27783]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:24.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:23 smithi086 ceph-mon[27783]: Saving service mgr spec with placement count:2 2024-04-25T18:50:24.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:23 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:24.184 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:23 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:25.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:25 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:25.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:25 smithi086 ceph-mon[27783]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:50:25.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:25 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2488501545' entity='client.admin' 2024-04-25T18:50:25.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.490+0000 7fac24200700 1 Processor -- start 2024-04-25T18:50:25.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.491+0000 7fac24200700 1 -- start start 2024-04-25T18:50:25.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.491+0000 7fac24200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.491+0000 7fac24200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c108ed0 con 0x7fac1c1084e0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.491+0000 7fac21800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.492+0000 7fac21800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57714/0 (socket says 172.21.15.86:57714) 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.492+0000 7fac21800700 1 -- 172.21.15.86:0/3290604180 learned_addr learned my addr 172.21.15.86:0/3290604180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.492+0000 7fac21800700 1 -- 172.21.15.86:0/3290604180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c109690 con 0x7fac1c1084e0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.492+0000 7fac21800700 1 --2- 172.21.15.86:0/3290604180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c108900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac100098a0 tx=0x7fac10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647e095b8b3a3646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac1be00700 1 -- 172.21.15.86:0/3290604180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10004160 con 0x7fac1c1084e0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac1be00700 1 -- 172.21.15.86:0/3290604180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac10004300 con 0x7fac1c1084e0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac1be00700 1 -- 172.21.15.86:0/3290604180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10011a20 con 0x7fac1c1084e0 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac24200700 1 -- 172.21.15.86:0/3290604180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 msgr2=0x7fac1c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac24200700 1 --2- 172.21.15.86:0/3290604180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c108900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac100098a0 tx=0x7fac10009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac24200700 1 -- 172.21.15.86:0/3290604180 shutdown_connections 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac24200700 1 --2- 172.21.15.86:0/3290604180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c108900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:25.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.493+0000 7fac24200700 1 -- 172.21.15.86:0/3290604180 >> 172.21.15.86:0/3290604180 conn(0x7fac1c103ae0 msgr2=0x7fac1c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.494+0000 7fac24200700 1 -- 172.21.15.86:0/3290604180 shutdown_connections 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.494+0000 7fac24200700 1 -- 172.21.15.86:0/3290604180 wait complete. 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.495+0000 7fac24200700 1 Processor -- start 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.495+0000 7fac24200700 1 -- start start 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.495+0000 7fac24200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.495+0000 7fac24200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c10ca40 con 0x7fac1c1084e0 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.496+0000 7fac21800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.496+0000 7fac21800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c19c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57720/0 (socket says 172.21.15.86:57720) 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.496+0000 7fac21800700 1 -- 172.21.15.86:0/2488501545 learned_addr learned my addr 172.21.15.86:0/2488501545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.496+0000 7fac21800700 1 -- 172.21.15.86:0/2488501545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c19cc70 con 0x7fac1c1084e0 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.496+0000 7fac21800700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c19c730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.496+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10003aa0 con 0x7fac1c1084e0 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.497+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1001b8f0 con 0x7fac1c1084e0 2024-04-25T18:50:25.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.497+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac10023890 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.497+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c19cf00 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.497+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c19fbf0 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.498+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fac1002c3f0 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.498+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc005360 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.498+0000 7fac1a000700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fac04039600 0x7fac0403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.499+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac1004de20 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.499+0000 7fac20e00700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fac04039600 0x7fac0403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.500+0000 7fac20e00700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fac04039600 0x7fac0403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac08006fd0 tx=0x7fac08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.505+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac1001f030 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.731+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fabfc005650 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.737+0000 7fac1a000700 1 -- 172.21.15.86:0/2488501545 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fac100202b0 con 0x7fac1c1084e0 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fac04039600 msgr2=0x7fac0403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fac04039600 0x7fac0403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac08006fd0 tx=0x7fac08006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:25.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 msgr2=0x7fac1c19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c19c730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac10011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 shutdown_connections 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fac04039600 0x7fac0403bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 --2- 172.21.15.86:0/2488501545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c1084e0 0x7fac1c19c730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.741+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 >> 172.21.15.86:0/2488501545 conn(0x7fac1c103ae0 msgr2=0x7fac1c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.742+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 shutdown_connections 2024-04-25T18:50:25.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:24.742+0000 7fac24200700 1 -- 172.21.15.86:0/2488501545 wait complete. 2024-04-25T18:50:27.234 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.993+0000 7fc632000700 1 Processor -- start 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.993+0000 7fc632000700 1 -- start start 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.994+0000 7fc632000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.994+0000 7fc632000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c108ed0 con 0x7fc62c1084e0 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.994+0000 7fc62b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.994+0000 7fc62b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57730/0 (socket says 172.21.15.86:57730) 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.994+0000 7fc62b400700 1 -- 172.21.15.86:0/3806715736 learned_addr learned my addr 172.21.15.86:0/3806715736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.995+0000 7fc62b400700 1 -- 172.21.15.86:0/3806715736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c109690 con 0x7fc62c1084e0 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.995+0000 7fc62b400700 1 --2- 172.21.15.86:0/3806715736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c108900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098a0 tx=0x7fc61c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f814fe9877a692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.995+0000 7fc62a000700 1 -- 172.21.15.86:0/3806715736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61c004160 con 0x7fc62c1084e0 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.995+0000 7fc62a000700 1 -- 172.21.15.86:0/3806715736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c004300 con 0x7fc62c1084e0 2024-04-25T18:50:27.235 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.995+0000 7fc62a000700 1 -- 172.21.15.86:0/3806715736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61c011a20 con 0x7fc62c1084e0 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 -- 172.21.15.86:0/3806715736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 msgr2=0x7fc62c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 --2- 172.21.15.86:0/3806715736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c108900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098a0 tx=0x7fc61c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 -- 172.21.15.86:0/3806715736 shutdown_connections 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 --2- 172.21.15.86:0/3806715736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c108900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 -- 172.21.15.86:0/3806715736 >> 172.21.15.86:0/3806715736 conn(0x7fc62c103ae0 msgr2=0x7fc62c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 -- 172.21.15.86:0/3806715736 shutdown_connections 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.996+0000 7fc632000700 1 -- 172.21.15.86:0/3806715736 wait complete. 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.997+0000 7fc632000700 1 Processor -- start 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.997+0000 7fc632000700 1 -- start start 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.998+0000 7fc632000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c07db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.998+0000 7fc632000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10ca40 con 0x7fc62c1084e0 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.998+0000 7fc62b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c07db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.998+0000 7fc62b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c07db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57742/0 (socket says 172.21.15.86:57742) 2024-04-25T18:50:27.236 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.998+0000 7fc62b400700 1 -- 172.21.15.86:0/296179200 learned_addr learned my addr 172.21.15.86:0/296179200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.998+0000 7fc62b400700 1 -- 172.21.15.86:0/296179200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c07e070 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.999+0000 7fc62b400700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c07db30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc61c000c00 tx=0x7fc61c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.999+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61c003aa0 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.999+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c01b8f0 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.999+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c07c2b0 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.999+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61c023890 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:25.999+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c07c790 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.001+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc61c02c3f0 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.001+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c0640a0 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.002+0000 7fc623e00700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fc610039600 0x7fc61003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.002+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc61c04dec0 con 0x7fc62c1084e0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.002+0000 7fc62aa00700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fc610039600 0x7fc61003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:27.237 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.003+0000 7fc62aa00700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fc610039600 0x7fc61003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc618006fd0 tx=0x7fc618006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.008+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc61c01f030 con 0x7fc62c1084e0 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.234+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc62c195e50 con 0x7fc62c1084e0 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.239+0000 7fc623e00700 1 -- 172.21.15.86:0/296179200 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc61c01bda0 con 0x7fc62c1084e0 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fc610039600 msgr2=0x7fc61003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fc610039600 0x7fc61003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc618006fd0 tx=0x7fc618006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 msgr2=0x7fc62c07db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c07db30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc61c000c00 tx=0x7fc61c011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 shutdown_connections 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fc610039600 0x7fc61003bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 --2- 172.21.15.86:0/296179200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc62c1084e0 0x7fc62c07db30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 >> 172.21.15.86:0/296179200 conn(0x7fc62c103ae0 msgr2=0x7fc62c104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 shutdown_connections 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:26.243+0000 7fc632000700 1 -- 172.21.15.86:0/296179200 wait complete. 2024-04-25T18:50:27.238 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-04-25T18:50:27.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:27 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/296179200' entity='client.admin' 2024-04-25T18:50:27.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:27 smithi086 ceph-mon[27783]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:50:27.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:27 smithi086 ceph-mon[27783]: from='mgr.14120 172.21.15.86:0/1766801185' entity='mgr.x' 2024-04-25T18:50:28.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:28 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1683989885' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T18:50:28.618 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: ignoring --setuser ceph since I am not root 2024-04-25T18:50:28.619 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: ignoring --setgroup ceph since I am not root 2024-04-25T18:50:28.619 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:28.583+0000 7fa1420b3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:50:28.913 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:28.831+0000 7fa1420b3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:50:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.779+0000 7f6cf0c00700 1 Processor -- start 2024-04-25T18:50:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.779+0000 7f6cf0c00700 1 -- start start 2024-04-25T18:50:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.779+0000 7f6cf0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.779+0000 7f6cf0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec108f60 con 0x7f6cec108570 2024-04-25T18:50:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.780+0000 7f6ce9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.780+0000 7f6ce9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57746/0 (socket says 172.21.15.86:57746) 2024-04-25T18:50:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.780+0000 7f6ce9e00700 1 -- 172.21.15.86:0/421382681 learned_addr learned my addr 172.21.15.86:0/421382681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.781+0000 7f6ce9e00700 1 -- 172.21.15.86:0/421382681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec109720 con 0x7f6cec108570 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.781+0000 7f6ce9e00700 1 --2- 172.21.15.86:0/421382681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec108990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098a0 tx=0x7f6cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dee402991b73cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.781+0000 7f6ce8a00700 1 -- 172.21.15.86:0/421382681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc004160 con 0x7f6cec108570 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.781+0000 7f6ce8a00700 1 -- 172.21.15.86:0/421382681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cdc004300 con 0x7f6cec108570 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.781+0000 7f6ce8a00700 1 -- 172.21.15.86:0/421382681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc011a20 con 0x7f6cec108570 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 -- 172.21.15.86:0/421382681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 msgr2=0x7f6cec108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 --2- 172.21.15.86:0/421382681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec108990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098a0 tx=0x7f6cdc009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 -- 172.21.15.86:0/421382681 shutdown_connections 2024-04-25T18:50:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 --2- 172.21.15.86:0/421382681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec108990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 -- 172.21.15.86:0/421382681 >> 172.21.15.86:0/421382681 conn(0x7f6cec103bd0 msgr2=0x7f6cec105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 -- 172.21.15.86:0/421382681 shutdown_connections 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.782+0000 7f6cf0c00700 1 -- 172.21.15.86:0/421382681 wait complete. 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.783+0000 7f6cf0c00700 1 Processor -- start 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.783+0000 7f6cf0c00700 1 -- start start 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.783+0000 7f6cf0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.783+0000 7f6cf0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec10cad0 con 0x7f6cec108570 2024-04-25T18:50:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.784+0000 7f6ce9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.784+0000 7f6ce9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57752/0 (socket says 172.21.15.86:57752) 2024-04-25T18:50:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.784+0000 7f6ce9e00700 1 -- 172.21.15.86:0/1683989885 learned_addr learned my addr 172.21.15.86:0/1683989885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.784+0000 7f6ce9e00700 1 -- 172.21.15.86:0/1683989885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec07e0c0 con 0x7f6cec108570 2024-04-25T18:50:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.784+0000 7f6ce9e00700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec07db80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0119f0 tx=0x7f6cdc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.785+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0039e0 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.785+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cdc01b8f0 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.785+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec07c2b0 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.785+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0237a0 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.785+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec07c7f0 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.786+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec0640a0 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.786+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6cdc01b440 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.787+0000 7f6ce2a00700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f6cd00395b0 0x7f6cd003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.787+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6cdc04de00 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.787+0000 7f6ce9400700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f6cd00395b0 0x7f6cd003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.788+0000 7f6ce9400700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f6cd00395b0 0x7f6cd003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006fd0 tx=0x7f6cd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:27.792+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6cdc01f030 con 0x7f6cec108570 2024-04-25T18:50:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.067+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f6cec07ca70 con 0x7f6cec108570 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.254+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6cdc018350 con 0x7f6cec108570 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.259+0000 7f6ce2a00700 1 -- 172.21.15.86:0/1683989885 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6cdc016ca0 con 0x7f6cec108570 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.259+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f6cd00395b0 msgr2=0x7f6cd003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.259+0000 7f6cf0c00700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f6cd00395b0 0x7f6cd003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006fd0 tx=0x7f6cd8006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.259+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 msgr2=0x7f6cec07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.259+0000 7f6cf0c00700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec07db80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0119f0 tx=0x7f6cdc011ad0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.260+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 shutdown_connections 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.260+0000 7f6cf0c00700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7f6cd00395b0 0x7f6cd003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.260+0000 7f6cf0c00700 1 --2- 172.21.15.86:0/1683989885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cec108570 0x7f6cec07db80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.260+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 >> 172.21.15.86:0/1683989885 conn(0x7f6cec103bd0 msgr2=0x7f6cec1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.261+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 shutdown_connections 2024-04-25T18:50:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:28.261+0000 7f6cf0c00700 1 -- 172.21.15.86:0/1683989885 wait complete. 2024-04-25T18:50:29.250 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:28.968+0000 7fa1420b3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:50:29.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:29 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1683989885' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T18:50:29.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:29 smithi086 ceph-mon[27783]: mgrmap e9: x(active, since 23s) 2024-04-25T18:50:29.550 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:29.372+0000 7fa1420b3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:50:29.550 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:29.488+0000 7fa1420b3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:50:30.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:30 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2650374243' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T18:50:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:50:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T18:50:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:50:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T18:50:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T18:50:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.548+0000 7efed1200700 1 Processor -- start 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.548+0000 7efed1200700 1 -- start start 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.548+0000 7efed1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc108910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.548+0000 7efed1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efecc108ee0 con 0x7efecc1084f0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.549+0000 7efec9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.549+0000 7efec9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57772/0 (socket says 172.21.15.86:57772) 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.549+0000 7efec9e00700 1 -- 172.21.15.86:0/2630366709 learned_addr learned my addr 172.21.15.86:0/2630366709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.549+0000 7efec9e00700 1 -- 172.21.15.86:0/2630366709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc1096a0 con 0x7efecc1084f0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.550+0000 7efec9e00700 1 --2- 172.21.15.86:0/2630366709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc108910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efebc0098a0 tx=0x7efebc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0966688a21d15d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.550+0000 7efec8a00700 1 -- 172.21.15.86:0/2630366709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efebc004160 con 0x7efecc1084f0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.550+0000 7efec8a00700 1 -- 172.21.15.86:0/2630366709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc004300 con 0x7efecc1084f0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.550+0000 7efec8a00700 1 -- 172.21.15.86:0/2630366709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efebc011a20 con 0x7efecc1084f0 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 -- 172.21.15.86:0/2630366709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 msgr2=0x7efecc108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 --2- 172.21.15.86:0/2630366709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc108910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efebc0098a0 tx=0x7efebc009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:30.457 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 -- 172.21.15.86:0/2630366709 shutdown_connections 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 --2- 172.21.15.86:0/2630366709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc108910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 -- 172.21.15.86:0/2630366709 >> 172.21.15.86:0/2630366709 conn(0x7efecc103ab0 msgr2=0x7efecc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 -- 172.21.15.86:0/2630366709 shutdown_connections 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.551+0000 7efed1200700 1 -- 172.21.15.86:0/2630366709 wait complete. 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.552+0000 7efed1200700 1 Processor -- start 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.552+0000 7efed1200700 1 -- start start 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.553+0000 7efed1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.553+0000 7efed1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efecc10ca50 con 0x7efecc1084f0 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.553+0000 7efec9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.553+0000 7efec9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57780/0 (socket says 172.21.15.86:57780) 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.553+0000 7efec9e00700 1 -- 172.21.15.86:0/2650374243 learned_addr learned my addr 172.21.15.86:0/2650374243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.553+0000 7efec9e00700 1 -- 172.21.15.86:0/2650374243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc19cc60 con 0x7efecc1084f0 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.554+0000 7efec9e00700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc19c720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:30.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.554+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efebc003aa0 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.554+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efebc01b8f0 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.554+0000 7efed1200700 1 -- 172.21.15.86:0/2650374243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc19cef0 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.554+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efebc023890 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.554+0000 7efed1200700 1 -- 172.21.15.86:0/2650374243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc19fbe0 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.555+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7efebc01b440 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.555+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc0640a0 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.556+0000 7efec2a00700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 0x7efeb003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.556+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efebc04ded0 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.556+0000 7efec9400700 1 -- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 msgr2=0x7efeb003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.556+0000 7efec9400700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 0x7efeb003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.566+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efebc01f030 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.757+0000 7efec9400700 1 -- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 msgr2=0x7efeb003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.757+0000 7efec9400700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 0x7efeb003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.851+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7efecc068a40 con 0x7efecc1084f0 2024-04-25T18:50:30.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.851+0000 7efec2a00700 1 -- 172.21.15.86:0/2650374243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7efebc016260 con 0x7efecc1084f0 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 msgr2=0x7efeb003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 0x7efeb003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 msgr2=0x7efecc19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc19c720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 shutdown_connections 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7efeb0039600 0x7efeb003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 --2- 172.21.15.86:0/2650374243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efecc1084f0 0x7efecc19c720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.854+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 >> 172.21.15.86:0/2650374243 conn(0x7efecc103ab0 msgr2=0x7efecc104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.855+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 shutdown_connections 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:29.855+0000 7efeb7e00700 1 -- 172.21.15.86:0/2650374243 wait complete. 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-25T18:50:30.460 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-04-25T18:50:31.290 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:31.061+0000 7fa1420b3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:50:31.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:31.288+0000 7fa1420b3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:50:31.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:31.504+0000 7fa1420b3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:50:32.005 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:31.883+0000 7fa1420b3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:50:32.255 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:32.002+0000 7fa1420b3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:50:32.508 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:32.253+0000 7fa1420b3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:50:32.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:32.507+0000 7fa1420b3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:50:33.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:33.277+0000 7fa1420b3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:50:34.215 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:33.822+0000 7fa1420b3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:50:34.215 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:33.948+0000 7fa1420b3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:50:34.215 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:34.213+0000 7fa1420b3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:50:35.319 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:34.873+0000 7fa1420b3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:50:35.319 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:34.997+0000 7fa1420b3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:50:35.319 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:35.317+0000 7fa1420b3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:50:35.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:35.495+0000 7fa1420b3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:50:36.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:35.623+0000 7fa1420b3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:50:36.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:35.774+0000 7fa1420b3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:50:36.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:36 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:36.141+0000 7fa1420b3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:50:37.173 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:36 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:36.888+0000 7fa1420b3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:50:37.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:50:37.171+0000 7fa1420b3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:50:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: Active manager daemon x restarted 2024-04-25T18:50:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: Activating manager daemon x 2024-04-25T18:50:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T18:50:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: mgrmap e10: x(active, starting, since 0.0109373s) 2024-04-25T18:50:37.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:37.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:50:37.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:50:37.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:50:37.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:50:37.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:37 smithi086 ceph-mon[27783]: Manager daemon x is now available 2024-04-25T18:50:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:50:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:50:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:38 smithi086 ceph-mon[27783]: mgrmap e11: x(active, since 1.01366s) 2024-04-25T18:50:38.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.026+0000 7fa41f600700 1 Processor -- start 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.026+0000 7fa41f600700 1 -- start start 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.026+0000 7fa41f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa4181077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.026+0000 7fa41f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa418107d80 con 0x7fa418107390 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.027+0000 7fa41cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa4181077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.027+0000 7fa41cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa4181077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54562/0 (socket says 172.21.15.86:54562) 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.027+0000 7fa41cc00700 1 -- 172.21.15.86:0/2908696627 learned_addr learned my addr 172.21.15.86:0/2908696627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.027+0000 7fa41cc00700 1 -- 172.21.15.86:0/2908696627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa418108590 con 0x7fa418107390 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.028+0000 7fa41cc00700 1 --2- 172.21.15.86:0/2908696627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa4181077b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0098a0 tx=0x7fa40c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a14cf41301c888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.028+0000 7fa417400700 1 -- 172.21.15.86:0/2908696627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40c004160 con 0x7fa418107390 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.028+0000 7fa417400700 1 -- 172.21.15.86:0/2908696627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40c004300 con 0x7fa418107390 2024-04-25T18:50:38.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.028+0000 7fa417400700 1 -- 172.21.15.86:0/2908696627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40c011a20 con 0x7fa418107390 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 -- 172.21.15.86:0/2908696627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 msgr2=0x7fa4181077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 --2- 172.21.15.86:0/2908696627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa4181077b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0098a0 tx=0x7fa40c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 -- 172.21.15.86:0/2908696627 shutdown_connections 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 --2- 172.21.15.86:0/2908696627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa4181077b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 -- 172.21.15.86:0/2908696627 >> 172.21.15.86:0/2908696627 conn(0x7fa4181029b0 msgr2=0x7fa418104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 -- 172.21.15.86:0/2908696627 shutdown_connections 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.029+0000 7fa41f600700 1 -- 172.21.15.86:0/2908696627 wait complete. 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.030+0000 7fa41f600700 1 Processor -- start 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.031+0000 7fa41f600700 1 -- start start 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.031+0000 7fa41f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa41807ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.031+0000 7fa41f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa41810b940 con 0x7fa418107390 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.031+0000 7fa41cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa41807ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.031+0000 7fa41cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa41807ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54564/0 (socket says 172.21.15.86:54564) 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.031+0000 7fa41cc00700 1 -- 172.21.15.86:0/4272461873 learned_addr learned my addr 172.21.15.86:0/4272461873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:38.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.032+0000 7fa41cc00700 1 -- 172.21.15.86:0/4272461873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41819f190 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.032+0000 7fa41cc00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa41807ba60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0119f0 tx=0x7fa40c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.032+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40c0039e0 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.032+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40c01b8f0 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.032+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40c0237a0 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.032+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41819f360 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.033+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41819f700 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.034+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa40c01b440 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.034+0000 7fa415600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.034+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 --> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa40003c180 con 0x7fa4000395b0 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.034+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa40c04de80 con 0x7fa418107390 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.034+0000 7fa417e00700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 msgr2=0x7fa40003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.034+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.235+0000 7fa417e00700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 msgr2=0x7fa40003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.235+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:50:38.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.636+0000 7fa417e00700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 msgr2=0x7fa40003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:31.636+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:32.437+0000 7fa417e00700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 msgr2=0x7fa40003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:32.437+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:34.039+0000 7fa417e00700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 msgr2=0x7fa40003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4264123395 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:34.039+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:37.181+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fa40c02c3f0 con 0x7fa418107390 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:37.182+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 msgr2=0x7fa40003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:37.182+0000 7fa415600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.183+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fa40c04fc20 con 0x7fa418107390 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.183+0000 7fa415600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa40003cfe0 0x7fa40003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.183+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa40003c180 con 0x7fa40003cfe0 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.184+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa40003cfe0 0x7fa40003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.185+0000 7fa417e00700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa40003cfe0 0x7fa40003f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa404003a10 tx=0x7fa4040092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.187+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa40003c180 con 0x7fa40003cfe0 2024-04-25T18:50:38.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.195+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa418069b90 con 0x7fa40003cfe0 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa415600700 1 -- 172.21.15.86:0/4272461873 <== mgr.14152 v2:172.21.15.86:6800/237697797 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa418069b90 con 0x7fa40003cfe0 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa40003cfe0 msgr2=0x7fa40003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa40003cfe0 0x7fa40003f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa404003a10 tx=0x7fa4040092b0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 msgr2=0x7fa41807ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa41807ba60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0119f0 tx=0x7fa40c011ad0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 shutdown_connections 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa40003cfe0 0x7fa40003f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418107390 0x7fa41807ba60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 --2- 172.21.15.86:0/4272461873 >> [v2:172.21.15.86:6800/4264123395,v1:172.21.15.86:6801/4264123395] conn(0x7fa4000395b0 0x7fa40003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 >> 172.21.15.86:0/4272461873 conn(0x7fa4181029b0 msgr2=0x7fa418103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.196+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 shutdown_connections 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:38.197+0000 7fa41f600700 1 -- 172.21.15.86:0/4272461873 wait complete. 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-04-25T18:50:38.992 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-04-25T18:50:39.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:39 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:38] ENGINE Bus STARTING 2024-04-25T18:50:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:40 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:38] ENGINE Serving on http://172.21.15.86:8765 2024-04-25T18:50:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:40 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:38] ENGINE Serving on https://172.21.15.86:7150 2024-04-25T18:50:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:40 smithi086 ceph-mon[27783]: [25/Apr/2024:18:50:38] ENGINE Bus STARTED 2024-04-25T18:50:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:40 smithi086 ceph-mon[27783]: mgrmap e12: x(active, since 2s) 2024-04-25T18:50:40.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:40.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T18:50:40.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.000+0000 7f4b1f400700 1 Processor -- start 2024-04-25T18:50:40.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.000+0000 7f4b1f400700 1 -- start start 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.001+0000 7f4b1f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.001+0000 7f4b1f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1807c900 con 0x7f4b1807de80 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.001+0000 7f4b1ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.001+0000 7f4b1ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59522/0 (socket says 172.21.15.86:59522) 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.001+0000 7f4b1ca00700 1 -- 172.21.15.86:0/1725015081 learned_addr learned my addr 172.21.15.86:0/1725015081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.002+0000 7f4b1ca00700 1 -- 172.21.15.86:0/1725015081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1807ca80 con 0x7f4b1807de80 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.002+0000 7f4b1ca00700 1 --2- 172.21.15.86:0/1725015081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1807c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098a0 tx=0x7f4b0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b53725779be14c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.002+0000 7f4b17400700 1 -- 172.21.15.86:0/1725015081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c004160 con 0x7f4b1807de80 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.002+0000 7f4b17400700 1 -- 172.21.15.86:0/1725015081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c004300 con 0x7f4b1807de80 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.002+0000 7f4b17400700 1 -- 172.21.15.86:0/1725015081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c011a20 con 0x7f4b1807de80 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 -- 172.21.15.86:0/1725015081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 msgr2=0x7f4b1807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 --2- 172.21.15.86:0/1725015081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1807c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098a0 tx=0x7f4b0c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 -- 172.21.15.86:0/1725015081 shutdown_connections 2024-04-25T18:50:40.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 --2- 172.21.15.86:0/1725015081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1807c330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 -- 172.21.15.86:0/1725015081 >> 172.21.15.86:0/1725015081 conn(0x7f4b18103ab0 msgr2=0x7f4b18105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 -- 172.21.15.86:0/1725015081 shutdown_connections 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.003+0000 7f4b1f400700 1 -- 172.21.15.86:0/1725015081 wait complete. 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.004+0000 7f4b1f400700 1 Processor -- start 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.004+0000 7f4b1f400700 1 -- start start 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.005+0000 7f4b1f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.005+0000 7f4b1f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1810ea70 con 0x7f4b1807de80 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.005+0000 7f4b1ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.005+0000 7f4b1ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59538/0 (socket says 172.21.15.86:59538) 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.005+0000 7f4b1ca00700 1 -- 172.21.15.86:0/627644470 learned_addr learned my addr 172.21.15.86:0/627644470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.005+0000 7f4b1ca00700 1 -- 172.21.15.86:0/627644470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1819ee30 con 0x7f4b1807de80 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.006+0000 7f4b1ca00700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1819e8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c004660 tx=0x7f4b0c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.006+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c003aa0 con 0x7f4b1807de80 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.006+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c01b8f0 con 0x7f4b1807de80 2024-04-25T18:50:40.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.006+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1819f0c0 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.006+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c023900 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.006+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b181a1db0 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.007+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4b0c023b70 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.007+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b180640a0 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.008+0000 7f4b15600700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b00039660 0x7f4b0003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.008+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b0c04e010 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.008+0000 7f4b17e00700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b00039660 0x7f4b0003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.009+0000 7f4b17e00700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b00039660 0x7f4b0003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b04006fd0 tx=0x7f4b04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.014+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b0c01f030 con 0x7f4b1807de80 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.258+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f4b18062e80 con 0x7f4b00039660 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.387+0000 7f4b15600700 1 -- 172.21.15.86:0/627644470 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f4b18062e80 con 0x7f4b00039660 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.389+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b00039660 msgr2=0x7f4b0003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.389+0000 7f4b1f400700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b00039660 0x7f4b0003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b04006fd0 tx=0x7f4b04006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.389+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 msgr2=0x7f4b1819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.996 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.389+0000 7f4b1f400700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1819e8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c004660 tx=0x7f4b0c011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.390+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 shutdown_connections 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.390+0000 7f4b1f400700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b00039660 0x7f4b0003bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.390+0000 7f4b1f400700 1 --2- 172.21.15.86:0/627644470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1807de80 0x7f4b1819e8f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.390+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 >> 172.21.15.86:0/627644470 conn(0x7f4b18103ab0 msgr2=0x7f4b18104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.390+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 shutdown_connections 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:40.390+0000 7f4b1f400700 1 -- 172.21.15.86:0/627644470 wait complete. 2024-04-25T18:50:40.997 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-04-25T18:50:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:41 smithi086 ceph-mon[27783]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:42.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ec5HJcovcz1uEiaG9.LvGeIB7XfNx925kOl1fUyIJcRjtUEfHnY/K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714071042, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T18:50:42.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.580+0000 7f96be400700 1 Processor -- start 2024-04-25T18:50:42.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.580+0000 7f96be400700 1 -- start start 2024-04-25T18:50:42.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.581+0000 7f96be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b81077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:42.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.581+0000 7f96be400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b8107d80 con 0x7f96b8107390 2024-04-25T18:50:42.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.581+0000 7f96b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b81077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:42.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.581+0000 7f96b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b81077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59554/0 (socket says 172.21.15.86:59554) 2024-04-25T18:50:42.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.581+0000 7f96b7400700 1 -- 172.21.15.86:0/3780437048 learned_addr learned my addr 172.21.15.86:0/3780437048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:42.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.582+0000 7f96b7400700 1 -- 172.21.15.86:0/3780437048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b8108590 con 0x7f96b8107390 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.582+0000 7f96b7400700 1 --2- 172.21.15.86:0/3780437048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b81077b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96a80098a0 tx=0x7f96a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25f8a0fd3479151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.582+0000 7f96b6000700 1 -- 172.21.15.86:0/3780437048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8004160 con 0x7f96b8107390 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.582+0000 7f96b6000700 1 -- 172.21.15.86:0/3780437048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a8004300 con 0x7f96b8107390 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.582+0000 7f96b6000700 1 -- 172.21.15.86:0/3780437048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8011a20 con 0x7f96b8107390 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.583+0000 7f96be400700 1 -- 172.21.15.86:0/3780437048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 msgr2=0x7f96b81077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.583+0000 7f96be400700 1 --2- 172.21.15.86:0/3780437048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b81077b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96a80098a0 tx=0x7f96a8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.583+0000 7f96be400700 1 -- 172.21.15.86:0/3780437048 shutdown_connections 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.583+0000 7f96be400700 1 --2- 172.21.15.86:0/3780437048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b81077b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.583+0000 7f96be400700 1 -- 172.21.15.86:0/3780437048 >> 172.21.15.86:0/3780437048 conn(0x7f96b81029b0 msgr2=0x7f96b8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.583+0000 7f96be400700 1 -- 172.21.15.86:0/3780437048 shutdown_connections 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.584+0000 7f96be400700 1 -- 172.21.15.86:0/3780437048 wait complete. 2024-04-25T18:50:42.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.584+0000 7f96be400700 1 Processor -- start 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.585+0000 7f96be400700 1 -- start start 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.585+0000 7f96be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.585+0000 7f96be400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b807afe0 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.585+0000 7f96b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.585+0000 7f96b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59568/0 (socket says 172.21.15.86:59568) 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.585+0000 7f96b7400700 1 -- 172.21.15.86:0/2916283796 learned_addr learned my addr 172.21.15.86:0/2916283796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.586+0000 7f96b7400700 1 -- 172.21.15.86:0/2916283796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b819a750 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.586+0000 7f96b7400700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b819a210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96a8000c00 tx=0x7f96a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.586+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8003aa0 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.586+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a801b8f0 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.586+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b819a9e0 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.586+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8023890 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.587+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b819d6d0 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.588+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b80640a0 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.589+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f96a801b440 con 0x7f96b8107390 2024-04-25T18:50:42.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.589+0000 7f96afe00700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f969c03dab0 0x7f969c03ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.589+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96a804dc10 con 0x7f96b8107390 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.589+0000 7f96b6a00700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f969c03dab0 0x7f969c03ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.590+0000 7f96b6a00700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f969c03dab0 0x7f969c03ff70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96a4006fd0 tx=0x7f96a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.595+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96a801f030 con 0x7f96b8107390 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:41.838+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f96b819d960 con 0x7f969c03dab0 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.076+0000 7f96afe00700 1 -- 172.21.15.86:0/2916283796 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f96b819d960 con 0x7f969c03dab0 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f969c03dab0 msgr2=0x7f969c03ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f969c03dab0 0x7f969c03ff70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96a4006fd0 tx=0x7f96a4006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 msgr2=0x7f96b819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b819a210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f96a8000c00 tx=0x7f96a8011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 shutdown_connections 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f969c03dab0 0x7f969c03ff70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 --2- 172.21.15.86:0/2916283796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b8107390 0x7f96b819a210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:42.754 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 >> 172.21.15.86:0/2916283796 conn(0x7f96b81029b0 msgr2=0x7f96b8190830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:42.755 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 shutdown_connections 2024-04-25T18:50:42.755 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:42.078+0000 7f96be400700 1 -- 172.21.15.86:0/2916283796 wait complete. 2024-04-25T18:50:42.755 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-04-25T18:50:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:42 smithi086 ceph-mon[27783]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:43 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/589837791' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T18:50:45.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T18:50:45.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.393+0000 7f5f23800700 1 Processor -- start 2024-04-25T18:50:45.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.393+0000 7f5f23800700 1 -- start start 2024-04-25T18:50:45.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.393+0000 7f5f23800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c107210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:45.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.393+0000 7f5f23800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c07be40 con 0x7f5f1c106df0 2024-04-25T18:50:45.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.394+0000 7f5f20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c107210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.394+0000 7f5f20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c107210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59574/0 (socket says 172.21.15.86:59574) 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.394+0000 7f5f20e00700 1 -- 172.21.15.86:0/3838197845 learned_addr learned my addr 172.21.15.86:0/3838197845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.395+0000 7f5f20e00700 1 -- 172.21.15.86:0/3838197845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c107750 con 0x7f5f1c106df0 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.395+0000 7f5f20e00700 1 --2- 172.21.15.86:0/3838197845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c107210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82580956ba77c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.395+0000 7f5f1b400700 1 -- 172.21.15.86:0/3838197845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10004160 con 0x7f5f1c106df0 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.396+0000 7f5f1b400700 1 -- 172.21.15.86:0/3838197845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f10004300 con 0x7f5f1c106df0 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.396+0000 7f5f1b400700 1 -- 172.21.15.86:0/3838197845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10011a20 con 0x7f5f1c106df0 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.396+0000 7f5f23800700 1 -- 172.21.15.86:0/3838197845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 msgr2=0x7f5f1c107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.396+0000 7f5f23800700 1 --2- 172.21.15.86:0/3838197845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c107210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.396+0000 7f5f23800700 1 -- 172.21.15.86:0/3838197845 shutdown_connections 2024-04-25T18:50:45.134 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.397+0000 7f5f23800700 1 --2- 172.21.15.86:0/3838197845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c107210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.397+0000 7f5f23800700 1 -- 172.21.15.86:0/3838197845 >> 172.21.15.86:0/3838197845 conn(0x7f5f1c102a70 msgr2=0x7f5f1c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.397+0000 7f5f23800700 1 -- 172.21.15.86:0/3838197845 shutdown_connections 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.397+0000 7f5f23800700 1 -- 172.21.15.86:0/3838197845 wait complete. 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.398+0000 7f5f23800700 1 Processor -- start 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.398+0000 7f5f23800700 1 -- start start 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.398+0000 7f5f23800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.398+0000 7f5f23800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1087c0 con 0x7f5f1c106df0 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.399+0000 7f5f20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.399+0000 7f5f20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59586/0 (socket says 172.21.15.86:59586) 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.399+0000 7f5f20e00700 1 -- 172.21.15.86:0/589837791 learned_addr learned my addr 172.21.15.86:0/589837791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.399+0000 7f5f20e00700 1 -- 172.21.15.86:0/589837791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c19ee10 con 0x7f5f1c106df0 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.400+0000 7f5f20e00700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c19e8d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000c00 tx=0x7f5f10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.400+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10003aa0 con 0x7f5f1c106df0 2024-04-25T18:50:45.135 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.400+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1001b8f0 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.400+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c19f0a0 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.400+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10023870 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.400+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c1a1d90 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.402+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f5f10023b20 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.402+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1c0640a0 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.402+0000 7f5f19600700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f04039660 0x7f5f0403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.402+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f1004e220 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.402+0000 7f5f1be00700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f04039660 0x7f5f0403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.403+0000 7f5f1be00700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f04039660 0x7f5f0403bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.409+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f1001f030 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.638+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5f1c02df80 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.639+0000 7f5f19600700 1 -- 172.21.15.86:0/589837791 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5f10023e10 con 0x7f5f1c106df0 2024-04-25T18:50:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.641+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f04039660 msgr2=0x7f5f0403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.641+0000 7f5f23800700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f04039660 0x7f5f0403bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.641+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 msgr2=0x7f5f1c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.641+0000 7f5f23800700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c19e8d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000c00 tx=0x7f5f10011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.642+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 shutdown_connections 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.642+0000 7f5f23800700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f04039660 0x7f5f0403bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.642+0000 7f5f23800700 1 --2- 172.21.15.86:0/589837791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c106df0 0x7f5f1c19e8d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.642+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 >> 172.21.15.86:0/589837791 conn(0x7f5f1c102a70 msgr2=0x7f5f1c195210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.642+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 shutdown_connections 2024-04-25T18:50:45.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:43.642+0000 7f5f23800700 1 -- 172.21.15.86:0/589837791 wait complete. 2024-04-25T18:50:45.160 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:50:45.160 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-25T18:50:45.181 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:50:45.181 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-25T18:50:45.181 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-25T18:50:45.181 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T18:50:45.185 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-04-25T18:50:45.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:45.185 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086.front.sepia.ceph.com:8443/ 2024-04-25T18:50:45.185 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-04-25T18:50:45.185 INFO:teuthology.orchestra.run.smithi086.stdout: Password: 2zn27uyvcq 2024-04-25T18:50:45.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:45.186 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config directory 2024-04-25T18:50:45.187 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling autotune for osd_memory_target 2024-04-25T18:50:45.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:44 smithi086 ceph-mon[27783]: mgrmap e13: x(active, since 6s) 2024-04-25T18:50:46.602 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.763+0000 7f05bd800700 1 Processor -- start 2024-04-25T18:50:46.602 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.763+0000 7f05bd800700 1 -- start start 2024-04-25T18:50:46.602 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.763+0000 7f05bd800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:46.602 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.763+0000 7f05bd800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8110a70 con 0x7f05b8110080 2024-04-25T18:50:46.602 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.764+0000 7f05b6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:46.606 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.764+0000 7f05b6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59600/0 (socket says 172.21.15.86:59600) 2024-04-25T18:50:46.606 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.764+0000 7f05b6a00700 1 -- 172.21.15.86:0/400641272 learned_addr learned my addr 172.21.15.86:0/400641272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:46.606 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.764+0000 7f05b6a00700 1 -- 172.21.15.86:0/400641272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b810ece0 con 0x7f05b8110080 2024-04-25T18:50:46.606 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.765+0000 7f05b6a00700 1 --2- 172.21.15.86:0/400641272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81104a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05a80098a0 tx=0x7f05a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73ebc5572883f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.765+0000 7f05b5600700 1 -- 172.21.15.86:0/400641272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a8004160 con 0x7f05b8110080 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.765+0000 7f05b5600700 1 -- 172.21.15.86:0/400641272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8004300 con 0x7f05b8110080 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.765+0000 7f05b5600700 1 -- 172.21.15.86:0/400641272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a8011a20 con 0x7f05b8110080 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 -- 172.21.15.86:0/400641272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 msgr2=0x7f05b81104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 --2- 172.21.15.86:0/400641272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81104a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05a80098a0 tx=0x7f05a8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 -- 172.21.15.86:0/400641272 shutdown_connections 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 --2- 172.21.15.86:0/400641272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81104a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 -- 172.21.15.86:0/400641272 >> 172.21.15.86:0/400641272 conn(0x7f05b807dc20 msgr2=0x7f05b8080040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 -- 172.21.15.86:0/400641272 shutdown_connections 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.766+0000 7f05bd800700 1 -- 172.21.15.86:0/400641272 wait complete. 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.767+0000 7f05bd800700 1 Processor -- start 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.767+0000 7f05bd800700 1 -- start start 2024-04-25T18:50:46.607 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05bd800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05bd800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81130a0 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05b6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05b6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59610/0 (socket says 172.21.15.86:59610) 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05b6a00700 1 -- 172.21.15.86:0/4038463292 learned_addr learned my addr 172.21.15.86:0/4038463292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05b6a00700 1 -- 172.21.15.86:0/4038463292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b81a31d0 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.768+0000 7f05b6a00700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81a2c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009520 tx=0x7f05a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.769+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a80039e0 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.769+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a801b8f0 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.769+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a8019770 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.769+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b81a3460 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.769+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a6150 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.770+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f05a801b440 con 0x7f05b8110080 2024-04-25T18:50:46.608 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.770+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b80640a0 con 0x7f05b8110080 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.770+0000 7f05af400700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f059c0396f0 0x7f059c03bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.770+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f05a804e140 con 0x7f05b8110080 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.771+0000 7f05b6000700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f059c0396f0 0x7f059c03bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.771+0000 7f05b6000700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f059c0396f0 0x7f059c03bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f05a4006fd0 tx=0x7f05a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:45.776+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05a801f030 con 0x7f05b8110080 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.002+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f05b802df80 con 0x7f05b8110080 2024-04-25T18:50:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.002+0000 7f05af400700 1 -- 172.21.15.86:0/4038463292 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f05a8010380 con 0x7f05b8110080 2024-04-25T18:50:46.610 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f059c0396f0 msgr2=0x7f059c03bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f059c0396f0 0x7f059c03bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f05a4006fd0 tx=0x7f05a4006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 msgr2=0x7f05b81a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81a2c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009520 tx=0x7f05a8011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 shutdown_connections 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f059c0396f0 0x7f059c03bbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 --2- 172.21.15.86:0/4038463292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b8110080 0x7f05b81a2c90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 >> 172.21.15.86:0/4038463292 conn(0x7f05b807dc20 msgr2=0x7f05b807f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 shutdown_connections 2024-04-25T18:50:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:46.006+0000 7f05bd800700 1 -- 172.21.15.86:0/4038463292 wait complete. 2024-04-25T18:50:48.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.198+0000 7f0f62e00700 1 Processor -- start 2024-04-25T18:50:48.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.198+0000 7f0f62e00700 1 -- start start 2024-04-25T18:50:48.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.199+0000 7f0f62e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c10ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:48.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.199+0000 7f0f62e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c10b0d0 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.200+0000 7f0f5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c10ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:48.258 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.200+0000 7f0f5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c10ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59618/0 (socket says 172.21.15.86:59618) 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.200+0000 7f0f5be00700 1 -- 172.21.15.86:0/4076161908 learned_addr learned my addr 172.21.15.86:0/4076161908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.200+0000 7f0f5be00700 1 -- 172.21.15.86:0/4076161908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c10b890 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.200+0000 7f0f5be00700 1 --2- 172.21.15.86:0/4076161908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c10ab00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0098a0 tx=0x7f0f4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a792dc1a973ea13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f5aa00700 1 -- 172.21.15.86:0/4076161908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c004160 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f5aa00700 1 -- 172.21.15.86:0/4076161908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f4c004300 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f5aa00700 1 -- 172.21.15.86:0/4076161908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c011a20 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f62e00700 1 -- 172.21.15.86:0/4076161908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 msgr2=0x7f0f5c10ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f62e00700 1 --2- 172.21.15.86:0/4076161908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c10ab00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0098a0 tx=0x7f0f4c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f62e00700 1 -- 172.21.15.86:0/4076161908 shutdown_connections 2024-04-25T18:50:48.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f62e00700 1 --2- 172.21.15.86:0/4076161908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c10ab00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.201+0000 7f0f62e00700 1 -- 172.21.15.86:0/4076161908 >> 172.21.15.86:0/4076161908 conn(0x7f0f5c07dc20 msgr2=0x7f0f5c1084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.202+0000 7f0f62e00700 1 -- 172.21.15.86:0/4076161908 shutdown_connections 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.202+0000 7f0f62e00700 1 -- 172.21.15.86:0/4076161908 wait complete. 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.202+0000 7f0f62e00700 1 Processor -- start 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.203+0000 7f0f62e00700 1 -- start start 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.203+0000 7f0f62e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.203+0000 7f0f62e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c10ec40 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.203+0000 7f0f5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.203+0000 7f0f5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59620/0 (socket says 172.21.15.86:59620) 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.203+0000 7f0f5be00700 1 -- 172.21.15.86:0/1554297630 learned_addr learned my addr 172.21.15.86:0/1554297630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.204+0000 7f0f5be00700 1 -- 172.21.15.86:0/1554297630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c19edd0 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.265 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.204+0000 7f0f5be00700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c19e890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009520 tx=0x7f0f4c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.204+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c0039e0 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.204+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f4c01b8f0 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.204+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c19f060 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.204+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c019770 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.205+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c1a1d50 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.205+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0f4c019a20 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.206+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c0640a0 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.206+0000 7f0f58c00700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0f400396f0 0x7f0f4003bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.206+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f4c04e1f0 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.206+0000 7f0f5b400700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0f400396f0 0x7f0f4003bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.207+0000 7f0f5b400700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0f400396f0 0x7f0f4003bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f48006fd0 tx=0x7f0f48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.212+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f4c01f030 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.559+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f0f5c02df80 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.565+0000 7f0f58c00700 1 -- 172.21.15.86:0/1554297630 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f0f4c010a10 con 0x7f0f5c10a6e0 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.567+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0f400396f0 msgr2=0x7f0f4003bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.567+0000 7f0f62e00700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0f400396f0 0x7f0f4003bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f48006fd0 tx=0x7f0f48006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.567+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 msgr2=0x7f0f5c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.567+0000 7f0f62e00700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c19e890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009520 tx=0x7f0f4c011aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.568+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 shutdown_connections 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.568+0000 7f0f62e00700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0f400396f0 0x7f0f4003bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.568+0000 7f0f62e00700 1 --2- 172.21.15.86:0/1554297630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c10a6e0 0x7f0f5c19e890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.568+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 >> 172.21.15.86:0/1554297630 conn(0x7f0f5c07dc20 msgr2=0x7f0f5c107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.568+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 shutdown_connections 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-25T18:50:47.568+0000 7f0f62e00700 1 -- 172.21.15.86:0/1554297630 wait complete. 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T18:50:48.267 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T18:50:48.268 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:50:48.269 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-04-25T18:50:48.368 INFO:tasks.cephadm:Fetching config... 2024-04-25T18:50:48.368 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:50:48.368 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T18:50:48.418 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T18:50:48.418 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:50:48.418 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T18:50:48.459 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T18:50:48.460 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:50:48.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T18:50:48.573 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T18:50:48.573 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:50:48.573 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T18:50:48.669 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T18:50:48.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:50:48.764 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:50:48.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:50:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:48 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1554297630' entity='client.admin' 2024-04-25T18:50:48.833 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:50:48.845 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:50:48.887 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwSwumC4WSh+zJB+kk5NSE57BvliowDrpFeHmJXwJHLj96rKeTD7IxzmjC7AlLZSTgA1E7+gRcq+nP1L2XNxsGXcLyub4g6Uw7OQyWcMSIUufwIyhOJ8HZURbvztIYeLazp8flfMQmJsXdo0m2+JtM3LsUdpQlpaS4ryG9GFMuK1QuRs1kJx+6FNp4nKhYzVWyUhU3K+5n/QXYMjjwoWwcEqd+Y6LFPMX2A+l2S1GPX3WpgWkgWCSLKaCvF+mzupC1hn8KdJrBhsC/bXTjygqmiEB7vYgSEnhR0nBXFg91tchC0RLFi6yJozrbVwFqE74goIsBXiSE4hzOLJuq9QwzNSPPZ5Vk286cluy79c42NRE+i35aFfDQBE/3x02OoB1BJ+1o9wmEqVme65utrafJ29Jb7/42Y43gh8TvCTMe8Vao/5aQkMUps9MCvjFYIJq22rO9Yse3y1MKH5uGZqht5ko3MVNzB4hb1aWoLP47cXdIrPOTSI3bLSTAM0tnatc= ceph-1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:50:48.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T18:50:49.309 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 -- 172.21.15.86:0/3373655853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 msgr2=0x7f8ebc1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 --2- 172.21.15.86:0/3373655853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc1011c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009910 tx=0x7f8eac009c20 comp rx=0 tx=0).stop 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 -- 172.21.15.86:0/3373655853 shutdown_connections 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 --2- 172.21.15.86:0/3373655853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc1011c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 -- 172.21.15.86:0/3373655853 >> 172.21.15.86:0/3373655853 conn(0x7f8ebc0fc940 msgr2=0x7f8ebc0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 -- 172.21.15.86:0/3373655853 shutdown_connections 2024-04-25T18:50:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.742+0000 7f8ec1600700 1 -- 172.21.15.86:0/3373655853 wait complete. 2024-04-25T18:50:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.743+0000 7f8ec1600700 1 Processor -- start 2024-04-25T18:50:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.744+0000 7f8ec1600700 1 -- start start 2024-04-25T18:50:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.744+0000 7f8ec1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.744+0000 7f8ec1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc10a5e0 con 0x7f8ebc100da0 2024-04-25T18:50:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.744+0000 7f8ebaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.744+0000 7f8ebaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39174/0 (socket says 172.21.15.86:39174) 2024-04-25T18:50:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.744+0000 7f8ebaa00700 1 -- 172.21.15.86:0/1485069772 learned_addr learned my addr 172.21.15.86:0/1485069772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.745+0000 7f8ebaa00700 1 -- 172.21.15.86:0/1485069772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc19a230 con 0x7f8ebc100da0 2024-04-25T18:50:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.745+0000 7f8ebaa00700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc199cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005190 tx=0x7f8eac004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.745+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eac027070 con 0x7f8ebc100da0 2024-04-25T18:50:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.745+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc19a4c0 con 0x7f8ebc100da0 2024-04-25T18:50:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.745+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac022470 con 0x7f8ebc100da0 2024-04-25T18:50:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.745+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eac00f5f0 con 0x7f8ebc100da0 2024-04-25T18:50:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.746+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc19d160 con 0x7f8ebc100da0 2024-04-25T18:50:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.747+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc10ad60 con 0x7f8ebc100da0 2024-04-25T18:50:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.747+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8eac02a070 con 0x7f8ebc100da0 2024-04-25T18:50:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.747+0000 7f8eb3400700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8ea0039650 0x7f8ea003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.747+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8eac01e030 con 0x7f8ebc100da0 2024-04-25T18:50:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.748+0000 7f8eba000700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8ea0039650 0x7f8ea003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.748+0000 7f8eba000700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8ea0039650 0x7f8ea003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8006fd0 tx=0x7f8ea8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:51.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.754+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8eac013070 con 0x7f8ebc100da0 2024-04-25T18:50:51.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.976+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8ebc02e090 con 0x7f8ebc100da0 2024-04-25T18:50:51.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.980+0000 7f8eb3400700 1 -- 172.21.15.86:0/1485069772 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8eac0310c0 con 0x7f8ebc100da0 2024-04-25T18:50:51.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.983+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8ea0039650 msgr2=0x7f8ea003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:51.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.983+0000 7f8ec1600700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8ea0039650 0x7f8ea003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8006fd0 tx=0x7f8ea8006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:51.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 msgr2=0x7f8ebc199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:51.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc199cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8eac005190 tx=0x7f8eac004f80 comp rx=0 tx=0).stop 2024-04-25T18:50:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 shutdown_connections 2024-04-25T18:50:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8ea0039650 0x7f8ea003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 --2- 172.21.15.86:0/1485069772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100da0 0x7f8ebc199cf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 >> 172.21.15.86:0/1485069772 conn(0x7f8ebc0fc940 msgr2=0x7f8ebc0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 shutdown_connections 2024-04-25T18:50:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:51.984+0000 7f8ec1600700 1 -- 172.21.15.86:0/1485069772 wait complete. 2024-04-25T18:50:52.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1485069772' entity='client.admin' 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:52.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:52.737 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T18:50:52.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T18:50:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.151+0000 7ff66b200700 1 -- 172.21.15.86:0/1386547279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 msgr2=0x7ff664103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.151+0000 7ff66b200700 1 --2- 172.21.15.86:0/1386547279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664103fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff654009910 tx=0x7ff654009c20 comp rx=0 tx=0).stop 2024-04-25T18:50:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.152+0000 7ff66b200700 1 -- 172.21.15.86:0/1386547279 shutdown_connections 2024-04-25T18:50:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.152+0000 7ff66b200700 1 --2- 172.21.15.86:0/1386547279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664103fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.152+0000 7ff66b200700 1 -- 172.21.15.86:0/1386547279 >> 172.21.15.86:0/1386547279 conn(0x7ff6640fda70 msgr2=0x7ff6640ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.152+0000 7ff66b200700 1 -- 172.21.15.86:0/1386547279 shutdown_connections 2024-04-25T18:50:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.152+0000 7ff66b200700 1 -- 172.21.15.86:0/1386547279 wait complete. 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.153+0000 7ff66b200700 1 Processor -- start 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.153+0000 7ff66b200700 1 -- start start 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.153+0000 7ff66b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.153+0000 7ff66b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff664108500 con 0x7ff664103b90 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.154+0000 7ff663e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.154+0000 7ff663e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39202/0 (socket says 172.21.15.86:39202) 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.154+0000 7ff663e00700 1 -- 172.21.15.86:0/1951992024 learned_addr learned my addr 172.21.15.86:0/1951992024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.154+0000 7ff663e00700 1 -- 172.21.15.86:0/1951992024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff664076c40 con 0x7ff664103b90 2024-04-25T18:50:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.154+0000 7ff663e00700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff654005950 tx=0x7ff654004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.155+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff654026070 con 0x7ff664103b90 2024-04-25T18:50:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.155+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff664074e30 con 0x7ff664103b90 2024-04-25T18:50:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.155+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65400bc00 con 0x7ff664103b90 2024-04-25T18:50:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.155+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff654003a10 con 0x7ff664103b90 2024-04-25T18:50:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.155+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff664075370 con 0x7ff664103b90 2024-04-25T18:50:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.156+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff654029070 con 0x7ff664103b90 2024-04-25T18:50:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.156+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664108c80 con 0x7ff664103b90 2024-04-25T18:50:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.156+0000 7ff660c00700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6480396b0 0x7ff64803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.156+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff65401e070 con 0x7ff664103b90 2024-04-25T18:50:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.157+0000 7ff663400700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6480396b0 0x7ff64803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.157+0000 7ff663400700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6480396b0 0x7ff64803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff650006fd0 tx=0x7ff650006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:55.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.162+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff654013070 con 0x7ff664103b90 2024-04-25T18:50:55.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.394+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff664190e70 con 0x7ff6480396b0 2024-04-25T18:50:55.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.402+0000 7ff660c00700 1 -- 172.21.15.86:0/1951992024 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff664190e70 con 0x7ff6480396b0 2024-04-25T18:50:55.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6480396b0 msgr2=0x7ff64803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6480396b0 0x7ff64803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff650006fd0 tx=0x7ff650006e40 comp rx=0 tx=0).stop 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 msgr2=0x7ff664076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff654005950 tx=0x7ff654004fa0 comp rx=0 tx=0).stop 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 shutdown_connections 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6480396b0 0x7ff64803bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 --2- 172.21.15.86:0/1951992024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff664103b90 0x7ff664076700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:55.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.405+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 >> 172.21.15.86:0/1951992024 conn(0x7ff6640fda70 msgr2=0x7ff6640fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:55.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.406+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 shutdown_connections 2024-04-25T18:50:55.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:55.406+0000 7ff66b200700 1 -- 172.21.15.86:0/1951992024 wait complete. 2024-04-25T18:50:56.208 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-04-25T18:50:56.209 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:50:56.209 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-04-25T18:50:56.227 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:50:56.227 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:50:56.285 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-04-25T18:50:56.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch host add smithi183 2024-04-25T18:50:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:56 smithi086 ceph-mon[27783]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:56 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:56 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:56 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:56 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.680 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:57.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:57 smithi086 ceph-mon[27783]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-25T18:50:57.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:57 smithi086 ceph-mon[27783]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:50:58.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 -- 172.21.15.86:0/2050846854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 msgr2=0x7fbbfc1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 --2- 172.21.15.86:0/2050846854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc1019d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00b0a0 tx=0x7fbbec00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:50:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 -- 172.21.15.86:0/2050846854 shutdown_connections 2024-04-25T18:50:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 --2- 172.21.15.86:0/2050846854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc1019d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 -- 172.21.15.86:0/2050846854 >> 172.21.15.86:0/2050846854 conn(0x7fbbfc076ea0 msgr2=0x7fbbfc0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 -- 172.21.15.86:0/2050846854 shutdown_connections 2024-04-25T18:50:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.573+0000 7fbc03000700 1 -- 172.21.15.86:0/2050846854 wait complete. 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.574+0000 7fbc03000700 1 Processor -- start 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.574+0000 7fbc03000700 1 -- start start 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.574+0000 7fbc03000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.574+0000 7fbc03000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc10b820 con 0x7fbbfc1015f0 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.575+0000 7fbbfbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.575+0000 7fbbfbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39214/0 (socket says 172.21.15.86:39214) 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.575+0000 7fbbfbe00700 1 -- 172.21.15.86:0/3339896367 learned_addr learned my addr 172.21.15.86:0/3339896367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:50:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.575+0000 7fbbfbe00700 1 -- 172.21.15.86:0/3339896367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc198380 con 0x7fbbfc1015f0 2024-04-25T18:50:58.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.575+0000 7fbbfbe00700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc197e40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00d010 tx=0x7fbbec003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:58.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.575+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbec021020 con 0x7fbbfc1015f0 2024-04-25T18:50:58.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.576+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbfc198610 con 0x7fbbfc1015f0 2024-04-25T18:50:58.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.576+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbec0044b0 con 0x7fbbfc1015f0 2024-04-25T18:50:58.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.576+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbec017040 con 0x7fbbfc1015f0 2024-04-25T18:50:58.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.576+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbfc19c6c0 con 0x7fbbfc1015f0 2024-04-25T18:50:58.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.577+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbbec024020 con 0x7fbbfc1015f0 2024-04-25T18:50:58.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.577+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbfc102a80 con 0x7fbbfc1015f0 2024-04-25T18:50:58.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.577+0000 7fbbf9600700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbe00396a0 0x7fbbe003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:58.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.577+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbbec00f050 con 0x7fbbfc1015f0 2024-04-25T18:50:58.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.578+0000 7fbbf3400700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbe00396a0 0x7fbbe003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:58.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.578+0000 7fbbf3400700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbe00396a0 0x7fbbe003bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.583+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbec02b020 con 0x7fbbfc1015f0 2024-04-25T18:50:58.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:50:58.813+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7fbbfc199150 con 0x7fbbe00396a0 2024-04-25T18:50:58.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:58 smithi086 ceph-mon[27783]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:50:58.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:58 smithi086 ceph-mon[27783]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:50:58.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:58 smithi086 ceph-mon[27783]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:50:58.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:58.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:50:58.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:50:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:00 smithi086 ceph-mon[27783]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:00 smithi086 ceph-mon[27783]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:01.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:01 smithi086 ceph-mon[27783]: Deploying cephadm binary to smithi183 2024-04-25T18:51:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:02 smithi086 ceph-mon[27783]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.648+0000 7fbbf9600700 1 -- 172.21.15.86:0/3339896367 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbbfc199150 con 0x7fbbe00396a0 2024-04-25T18:51:03.650 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-04-25T18:51:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbe00396a0 msgr2=0x7fbbe003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbe00396a0 0x7fbbe003bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4006fd0 tx=0x7fbbe4006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 msgr2=0x7fbbfc197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc197e40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00d010 tx=0x7fbbec003ce0 comp rx=0 tx=0).stop 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 shutdown_connections 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbe00396a0 0x7fbbe003bb60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 --2- 172.21.15.86:0/3339896367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbfc1015f0 0x7fbbfc197e40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 >> 172.21.15.86:0/3339896367 conn(0x7fbbfc076ea0 msgr2=0x7fbbfc069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.652+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 shutdown_connections 2024-04-25T18:51:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:03.653+0000 7fbc03000700 1 -- 172.21.15.86:0/3339896367 wait complete. 2024-04-25T18:51:03.725 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:03 smithi086 ceph-mon[27783]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:04.289 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T18:51:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:05.022 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:05.022 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:04 smithi086 ceph-mon[27783]: Added host smithi183 2024-04-25T18:51:05.022 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:05.022 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:05.813 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:05.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:05 smithi086 ceph-mon[27783]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:06.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 -- 172.21.15.86:0/2747988582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 msgr2=0x7ff23810aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 --2- 172.21.15.86:0/2747988582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23810aa30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009910 tx=0x7ff22c009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 -- 172.21.15.86:0/2747988582 shutdown_connections 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 --2- 172.21.15.86:0/2747988582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23810aa30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 -- 172.21.15.86:0/2747988582 >> 172.21.15.86:0/2747988582 conn(0x7ff238100160 msgr2=0x7ff238102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 -- 172.21.15.86:0/2747988582 shutdown_connections 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.571+0000 7ff240200700 1 -- 172.21.15.86:0/2747988582 wait complete. 2024-04-25T18:51:06.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.572+0000 7ff240200700 1 Processor -- start 2024-04-25T18:51:06.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.573+0000 7ff240200700 1 -- start start 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.573+0000 7ff240200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.573+0000 7ff240200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff23810eb40 con 0x7ff23810a650 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.573+0000 7ff23d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.573+0000 7ff23d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41772/0 (socket says 172.21.15.86:41772) 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.573+0000 7ff23d800700 1 -- 172.21.15.86:0/1242800207 learned_addr learned my addr 172.21.15.86:0/1242800207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.574+0000 7ff23d800700 1 -- 172.21.15.86:0/1242800207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23819a5d0 con 0x7ff23810a650 2024-04-25T18:51:06.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.574+0000 7ff23d800700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23819a090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff22c006010 tx=0x7ff22c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.575+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff22c026070 con 0x7ff23810a650 2024-04-25T18:51:06.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.575+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23819a8c0 con 0x7ff23810a650 2024-04-25T18:51:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.575+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22c00bba0 con 0x7ff23810a650 2024-04-25T18:51:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.575+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff22c003910 con 0x7ff23810a650 2024-04-25T18:51:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.575+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23819e960 con 0x7ff23810a650 2024-04-25T18:51:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.576+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff22c029070 con 0x7ff23810a650 2024-04-25T18:51:06.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.576+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238105d90 con 0x7ff23810a650 2024-04-25T18:51:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.577+0000 7ff236000700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff220039650 0x7ff22003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.577+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff22c01e070 con 0x7ff23810a650 2024-04-25T18:51:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.577+0000 7ff23ce00700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff220039650 0x7ff22003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:06.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.578+0000 7ff23ce00700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff220039650 0x7ff22003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff224006fd0 tx=0x7ff224006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.583+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff22c013070 con 0x7ff23810a650 2024-04-25T18:51:06.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.817+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff23819b3f0 con 0x7ff220039650 2024-04-25T18:51:06.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.820+0000 7ff236000700 1 -- 172.21.15.86:0/1242800207 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff23819b3f0 con 0x7ff220039650 2024-04-25T18:51:06.821 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:51:06.821 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.822+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff220039650 msgr2=0x7ff22003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.822+0000 7ff240200700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff220039650 0x7ff22003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff224006fd0 tx=0x7ff224006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 msgr2=0x7ff23819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23819a090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff22c006010 tx=0x7ff22c005780 comp rx=0 tx=0).stop 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 shutdown_connections 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff220039650 0x7ff22003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 --2- 172.21.15.86:0/1242800207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff23810a650 0x7ff23819a090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:06.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 >> 172.21.15.86:0/1242800207 conn(0x7ff238100160 msgr2=0x7ff23810c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 shutdown_connections 2024-04-25T18:51:06.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:06.823+0000 7ff240200700 1 -- 172.21.15.86:0/1242800207 wait complete. 2024-04-25T18:51:07.495 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:07.552 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2024-04-25T18:51:07.552 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:51:07.552 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2024-04-25T18:51:07.571 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:51:07.571 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:51:07.628 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2024-04-25T18:51:07.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch host add smithi194 2024-04-25T18:51:08.066 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:08.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:08 smithi086 ceph-mon[27783]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:51:08.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:08 smithi086 ceph-mon[27783]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:09.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.983+0000 7f492ce00700 1 -- 172.21.15.86:0/680065328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 msgr2=0x7f49280751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:09.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.983+0000 7f492ce00700 1 --2- 172.21.15.86:0/680065328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f49280751f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4918009910 tx=0x7f4918009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:09.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.984+0000 7f492ce00700 1 -- 172.21.15.86:0/680065328 shutdown_connections 2024-04-25T18:51:09.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.984+0000 7f492ce00700 1 --2- 172.21.15.86:0/680065328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f49280751f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:09.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.984+0000 7f492ce00700 1 -- 172.21.15.86:0/680065328 >> 172.21.15.86:0/680065328 conn(0x7f49280fdf80 msgr2=0x7f49281003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:09.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.984+0000 7f492ce00700 1 -- 172.21.15.86:0/680065328 shutdown_connections 2024-04-25T18:51:09.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.984+0000 7f492ce00700 1 -- 172.21.15.86:0/680065328 wait complete. 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.985+0000 7f492ce00700 1 Processor -- start 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.985+0000 7f492ce00700 1 -- start start 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.985+0000 7f492ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f492819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.985+0000 7f492ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f492810eba0 con 0x7f4928074e10 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.986+0000 7f4925e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f492819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.986+0000 7f4925e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f492819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51304/0 (socket says 172.21.15.86:51304) 2024-04-25T18:51:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.986+0000 7f4925e00700 1 -- 172.21.15.86:0/1850609732 learned_addr learned my addr 172.21.15.86:0/1850609732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:51:09.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.986+0000 7f4925e00700 1 -- 172.21.15.86:0/1850609732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492819a650 con 0x7f4928074e10 2024-04-25T18:51:09.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.986+0000 7f4925e00700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f492819a110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49180056d0 tx=0x7f49180057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:09.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.987+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4918026070 con 0x7f4928074e10 2024-04-25T18:51:09.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.987+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4918004370 con 0x7f4928074e10 2024-04-25T18:51:09.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.987+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492819a8e0 con 0x7f4928074e10 2024-04-25T18:51:09.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.987+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49180039b0 con 0x7f4928074e10 2024-04-25T18:51:09.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.987+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492819ad40 con 0x7f4928074e10 2024-04-25T18:51:09.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.988+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4918029070 con 0x7f4928074e10 2024-04-25T18:51:09.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.988+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49280764c0 con 0x7f4928074e10 2024-04-25T18:51:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.988+0000 7f491f400700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f490c0396b0 0x7f490c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.988+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f491801e070 con 0x7f4928074e10 2024-04-25T18:51:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.989+0000 7f491d400700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f490c0396b0 0x7f490c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.989+0000 7f491d400700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f490c0396b0 0x7f490c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4910006fd0 tx=0x7f4910006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:09.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:09.994+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4918013070 con 0x7f4928074e10 2024-04-25T18:51:10.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:10.225+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) -- 0x7f492819b510 con 0x7f490c0396b0 2024-04-25T18:51:10.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:10.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:10.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:10.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:10.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:10.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:10.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:10.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:11.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:12 smithi086 ceph-mon[27783]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:51:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:12 smithi086 ceph-mon[27783]: Deploying cephadm binary to smithi194 2024-04-25T18:51:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:12 smithi086 ceph-mon[27783]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:14 smithi086 ceph-mon[27783]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:14.913 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi194' with addr '172.21.15.194' 2024-04-25T18:51:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.911+0000 7f491f400700 1 -- 172.21.15.86:0/1850609732 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f492819b510 con 0x7f490c0396b0 2024-04-25T18:51:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f490c0396b0 msgr2=0x7f490c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f490c0396b0 0x7f490c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4910006fd0 tx=0x7f4910006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 msgr2=0x7f492819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f492819a110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49180056d0 tx=0x7f49180057b0 comp rx=0 tx=0).stop 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 shutdown_connections 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f490c0396b0 0x7f490c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 --2- 172.21.15.86:0/1850609732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928074e10 0x7f492819a110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.915+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 >> 172.21.15.86:0/1850609732 conn(0x7f49280fdf80 msgr2=0x7f4928108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.916+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 shutdown_connections 2024-04-25T18:51:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:14.916+0000 7f492ce00700 1 -- 172.21.15.86:0/1850609732 wait complete. 2024-04-25T18:51:15.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T18:51:16.051 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:16.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:16.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:15 smithi086 ceph-mon[27783]: Added host smithi194 2024-04-25T18:51:16.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:16.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:15 smithi086 ceph-mon[27783]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:16.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:17.171 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:16 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 -- 172.21.15.86:0/2529424688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 msgr2=0x7f39a410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 --2- 172.21.15.86:0/2529424688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a410aa40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3994009910 tx=0x7f3994009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 -- 172.21.15.86:0/2529424688 shutdown_connections 2024-04-25T18:51:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 --2- 172.21.15.86:0/2529424688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a410aa40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 -- 172.21.15.86:0/2529424688 >> 172.21.15.86:0/2529424688 conn(0x7f39a4100190 msgr2=0x7f39a41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 -- 172.21.15.86:0/2529424688 shutdown_connections 2024-04-25T18:51:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.975+0000 7f39aac00700 1 -- 172.21.15.86:0/2529424688 wait complete. 2024-04-25T18:51:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.976+0000 7f39aac00700 1 Processor -- start 2024-04-25T18:51:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.977+0000 7f39aac00700 1 -- start start 2024-04-25T18:51:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.977+0000 7f39aac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.977+0000 7f39aac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39a410eb50 con 0x7f39a410a660 2024-04-25T18:51:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.977+0000 7f39a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.977+0000 7f39a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51334/0 (socket says 172.21.15.86:51334) 2024-04-25T18:51:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.977+0000 7f39a3e00700 1 -- 172.21.15.86:0/70121474 learned_addr learned my addr 172.21.15.86:0/70121474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:51:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.978+0000 7f39a3e00700 1 -- 172.21.15.86:0/70121474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a419a5f0 con 0x7f39a410a660 2024-04-25T18:51:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.978+0000 7f39a3e00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a419a0b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3994006010 tx=0x7f3994005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.978+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3994026070 con 0x7f39a410a660 2024-04-25T18:51:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.978+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f399400bba0 con 0x7f39a410a660 2024-04-25T18:51:17.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.978+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a419a8e0 con 0x7f39a410a660 2024-04-25T18:51:17.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.978+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3994003910 con 0x7f39a410a660 2024-04-25T18:51:17.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.979+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a419e980 con 0x7f39a410a660 2024-04-25T18:51:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.980+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3994029070 con 0x7f39a410a660 2024-04-25T18:51:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.980+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a4105da0 con 0x7f39a410a660 2024-04-25T18:51:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.980+0000 7f39a1600700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f39880396b0 0x7f398803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.980+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f399401e070 con 0x7f39a410a660 2024-04-25T18:51:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.980+0000 7f399be00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f39880396b0 0x7f398803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.981+0000 7f399be00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f39880396b0 0x7f398803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f398c006fd0 tx=0x7f398c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:17.986+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3994013070 con 0x7f39a410a660 2024-04-25T18:51:18.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:18 smithi086 ceph-mon[27783]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:18.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:18.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.217+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39a419b410 con 0x7f39880396b0 2024-04-25T18:51:18.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.220+0000 7f39a1600700 1 -- 172.21.15.86:0/70121474 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f39a419b410 con 0x7f39880396b0 2024-04-25T18:51:18.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:51:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2024-04-25T18:51:18.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f39880396b0 msgr2=0x7f398803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:18.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f39880396b0 0x7f398803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f398c006fd0 tx=0x7f398c006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:18.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 msgr2=0x7f39a419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:18.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a419a0b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3994006010 tx=0x7f3994005780 comp rx=0 tx=0).stop 2024-04-25T18:51:18.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 shutdown_connections 2024-04-25T18:51:18.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f39880396b0 0x7f398803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:18.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 --2- 172.21.15.86:0/70121474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a410a660 0x7f39a419a0b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:18.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 >> 172.21.15.86:0/70121474 conn(0x7f39a4100190 msgr2=0x7f39a410c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:18.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 shutdown_connections 2024-04-25T18:51:18.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:18.223+0000 7f39aac00700 1 -- 172.21.15.86:0/70121474 wait complete. 2024-04-25T18:51:18.864 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T18:51:18.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T18:51:19.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:19 smithi086 ceph-mon[27783]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:51:19.291 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:20.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:20 smithi086 ceph-mon[27783]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 -- 172.21.15.86:0/2749919427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 msgr2=0x7fbb1c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 --2- 172.21.15.86:0/2749919427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c10a960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009910 tx=0x7fbb10009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 -- 172.21.15.86:0/2749919427 shutdown_connections 2024-04-25T18:51:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 --2- 172.21.15.86:0/2749919427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c10a960 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 -- 172.21.15.86:0/2749919427 >> 172.21.15.86:0/2749919427 conn(0x7fbb1c0fffc0 msgr2=0x7fbb1c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 -- 172.21.15.86:0/2749919427 shutdown_connections 2024-04-25T18:51:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.243+0000 7fbb22400700 1 -- 172.21.15.86:0/2749919427 wait complete. 2024-04-25T18:51:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.244+0000 7fbb22400700 1 Processor -- start 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.245+0000 7fbb22400700 1 -- start start 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.245+0000 7fbb22400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.245+0000 7fbb22400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c10e9f0 con 0x7fbb1c10a580 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.245+0000 7fbb21000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.245+0000 7fbb21000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45792/0 (socket says 172.21.15.86:45792) 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.245+0000 7fbb21000700 1 -- 172.21.15.86:0/2298326443 learned_addr learned my addr 172.21.15.86:0/2298326443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:51:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb21000700 1 -- 172.21.15.86:0/2298326443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c19a3d0 con 0x7fbb1c10a580 2024-04-25T18:51:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb21000700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c199e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb100056d0 tx=0x7fbb100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb10026070 con 0x7fbb1c10a580 2024-04-25T18:51:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c19a660 con 0x7fbb1c10a580 2024-04-25T18:51:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c19e760 con 0x7fbb1c10a580 2024-04-25T18:51:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1000bc00 con 0x7fbb1c10a580 2024-04-25T18:51:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.246+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb100037f0 con 0x7fbb1c10a580 2024-04-25T18:51:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.247+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbb10029070 con 0x7fbb1c10a580 2024-04-25T18:51:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.247+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c105bf0 con 0x7fbb1c10a580 2024-04-25T18:51:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.248+0000 7fbb19600700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbb04039650 0x7fbb0403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.248+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb1001e070 con 0x7fbb1c10a580 2024-04-25T18:51:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.248+0000 7fbb1be00700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbb04039650 0x7fbb0403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.249+0000 7fbb1be00700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbb04039650 0x7fbb0403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006fd0 tx=0x7fbb08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:21.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.254+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb10013070 con 0x7fbb1c10a580 2024-04-25T18:51:21.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.488+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbb1c19b1f0 con 0x7fbb1c10a580 2024-04-25T18:51:21.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.606+0000 7fbb19600700 1 -- 172.21.15.86:0/2298326443 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fbb10004390 con 0x7fbb1c10a580 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbb04039650 msgr2=0x7fbb0403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbb04039650 0x7fbb0403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006fd0 tx=0x7fbb08006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 msgr2=0x7fbb1c199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c199e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb100056d0 tx=0x7fbb100057b0 comp rx=0 tx=0).stop 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 shutdown_connections 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbb04039650 0x7fbb0403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 --2- 172.21.15.86:0/2298326443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb1c10a580 0x7fbb1c199e90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:21.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.612+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 >> 172.21.15.86:0/2298326443 conn(0x7fbb1c0fffc0 msgr2=0x7fbb1c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.613+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 shutdown_connections 2024-04-25T18:51:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:21.613+0000 7fbb22400700 1 -- 172.21.15.86:0/2298326443 wait complete. 2024-04-25T18:51:21.616 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-25T18:51:21.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:21.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:51:21.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:21.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:21 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2298326443' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T18:51:22.336 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-04-25T18:51:22.336 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-04-25T18:51:22.336 INFO:tasks.cephadm:Adding mon.c on smithi194 2024-04-25T18:51:22.336 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c' 2024-04-25T18:51:22.566 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:22 smithi086 ceph-mon[27783]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:51:23.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:22 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2298326443' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T18:51:23.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:22 smithi086 ceph-mon[27783]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:23.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:23.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:23.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:23.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 -- 172.21.15.194:0/1489962604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 msgr2=0x7f3e801088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:23.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 --2- 172.21.15.194:0/1489962604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e801088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009910 tx=0x7f3e74009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:23.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 -- 172.21.15.194:0/1489962604 shutdown_connections 2024-04-25T18:51:23.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 --2- 172.21.15.194:0/1489962604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e801088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 -- 172.21.15.194:0/1489962604 >> 172.21.15.194:0/1489962604 conn(0x7f3e800fdf80 msgr2=0x7f3e801003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 -- 172.21.15.194:0/1489962604 shutdown_connections 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.903+0000 7f3e86a83700 1 -- 172.21.15.194:0/1489962604 wait complete. 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.904+0000 7f3e86a83700 1 Processor -- start 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.904+0000 7f3e86a83700 1 -- start start 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.904+0000 7f3e86a83700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e80074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:23.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.904+0000 7f3e86a83700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010c940 con 0x7f3e801084e0 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.905+0000 7f3e8481f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e80074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.905+0000 7f3e8481f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e80074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.194:35306/0 (socket says 172.21.15.194:35306) 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.905+0000 7f3e8481f700 1 -- 172.21.15.194:0/4027597419 learned_addr learned my addr 172.21.15.194:0/4027597419 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.905+0000 7f3e8481f700 1 -- 172.21.15.194:0/4027597419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e80076c30 con 0x7f3e801084e0 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.906+0000 7f3e8481f700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e80074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006010 tx=0x7f3e74005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.906+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e74026070 con 0x7f3e801084e0 2024-04-25T18:51:23.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.906+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e800752e0 con 0x7f3e801084e0 2024-04-25T18:51:23.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.906+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e800757a0 con 0x7f3e801084e0 2024-04-25T18:51:23.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.906+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7400bba0 con 0x7f3e801084e0 2024-04-25T18:51:23.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.906+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e740038a0 con 0x7f3e801084e0 2024-04-25T18:51:23.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.907+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e80103b90 con 0x7f3e801084e0 2024-04-25T18:51:23.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.908+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3e74029070 con 0x7f3e801084e0 2024-04-25T18:51:23.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.908+0000 7f3e7dffb700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3e70039650 0x7f3e7003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:23.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.908+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3e7401e070 con 0x7f3e801084e0 2024-04-25T18:51:23.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.908+0000 7f3e7ffff700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3e70039650 0x7f3e7003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:23.911 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.909+0000 7f3e7ffff700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3e70039650 0x7f3e7003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c006fd0 tx=0x7f3e6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:23.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:23.924+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e74013070 con 0x7f3e801084e0 2024-04-25T18:51:24.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:23 smithi086 ceph-mon[27783]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:24.136 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.134+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}) -- 0x7f3e800762e0 con 0x7f3e70039650 2024-04-25T18:51:24.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.150+0000 7f3e7dffb700 1 -- 172.21.15.194:0/4027597419 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3e800762e0 con 0x7f3e70039650 2024-04-25T18:51:24.151 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mon update... 2024-04-25T18:51:24.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3e70039650 msgr2=0x7f3e7003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:24.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3e70039650 0x7f3e7003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c006fd0 tx=0x7f3e6c006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:24.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 msgr2=0x7f3e80074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:24.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e80074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006010 tx=0x7f3e74005780 comp rx=0 tx=0).stop 2024-04-25T18:51:24.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 shutdown_connections 2024-04-25T18:51:24.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3e70039650 0x7f3e7003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 --2- 172.21.15.194:0/4027597419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e801084e0 0x7f3e80074d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 >> 172.21.15.194:0/4027597419 conn(0x7f3e800fdf80 msgr2=0x7f3e8010a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:24.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 shutdown_connections 2024-04-25T18:51:24.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:24.151+0000 7f3e86a83700 1 -- 172.21.15.194:0/4027597419 wait complete. 2024-04-25T18:51:24.747 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T18:51:24.750 DEBUG:teuthology.orchestra.run.smithi194:mon.c> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T18:51:24.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:51:24.752 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c;count:3 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:25.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:25 smithi086 ceph-mon[27783]: Deploying daemon mon.c on smithi194 2024-04-25T18:51:26.444 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:26 smithi086 ceph-mon[27783]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:26.459 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.352+0000 7f8aaf7fe700 1 -- 172.21.15.194:0/458042228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4005460 con 0x7f8ab0101590 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 -- 172.21.15.194:0/458042228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 msgr2=0x7f8ab0101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 --2- 172.21.15.194:0/458042228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab0101970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009910 tx=0x7f8aa4009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 -- 172.21.15.194:0/458042228 shutdown_connections 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 --2- 172.21.15.194:0/458042228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab0101970 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 -- 172.21.15.194:0/458042228 >> 172.21.15.194:0/458042228 conn(0x7f8ab00fce40 msgr2=0x7f8ab00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:28.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 -- 172.21.15.194:0/458042228 shutdown_connections 2024-04-25T18:51:28.357 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.353+0000 7f8ab7034700 1 -- 172.21.15.194:0/458042228 wait complete. 2024-04-25T18:51:28.357 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab7034700 1 Processor -- start 2024-04-25T18:51:28.357 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab7034700 1 -- start start 2024-04-25T18:51:28.357 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab7034700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab010a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:28.357 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab7034700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010c910 con 0x7f8ab0101590 2024-04-25T18:51:28.357 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab4dd0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:28.358 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab4dd0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.194:35320/0 (socket says 172.21.15.194:35320) 2024-04-25T18:51:28.358 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.354+0000 7f8ab4dd0700 1 -- 172.21.15.194:0/1097934408 learned_addr learned my addr 172.21.15.194:0/1097934408 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:51:28.358 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.355+0000 7f8ab4dd0700 1 -- 172.21.15.194:0/1097934408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab010c320 con 0x7f8ab0101590 2024-04-25T18:51:28.358 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.356+0000 7f8ab4dd0700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab010a430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40056d0 tx=0x7f8aa40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:28.358 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.356+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4026070 con 0x7f8ab0101590 2024-04-25T18:51:28.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.356+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab010a9d0 con 0x7f8ab0101590 2024-04-25T18:51:28.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.356+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab010ae90 con 0x7f8ab0101590 2024-04-25T18:51:28.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.356+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa400b6f0 con 0x7f8ab0101590 2024-04-25T18:51:28.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.356+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4030060 con 0x7f8ab0101590 2024-04-25T18:51:28.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.357+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94005330 con 0x7f8ab0101590 2024-04-25T18:51:28.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.358+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8aa4029070 con 0x7f8ab0101590 2024-04-25T18:51:28.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.358+0000 7f8aadffb700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8aa0039630 0x7f8aa003baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:28.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.358+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8aa401e070 con 0x7f8ab0101590 2024-04-25T18:51:28.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.359+0000 7f8aaffff700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8aa0039630 0x7f8aa003baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:28.361 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.359+0000 7f8aaffff700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8aa0039630 0x7f8aa003baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006fd0 tx=0x7f8a9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:28.376 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.374+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8aa4013070 con 0x7f8ab0101590 2024-04-25T18:51:28.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:28 smithi086 ceph-mon[27783]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:28.680 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.677+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8a940058f0 con 0x7f8ab0101590 2024-04-25T18:51:28.681 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.679+0000 7f8aadffb700 1 -- 172.21.15.194:0/1097934408 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f8aa401f310 con 0x7f8ab0101590 2024-04-25T18:51:28.681 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:51:28.681 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","modified":"2024-04-25T18:49:19.080254Z","created":"2024-04-25T18:49:19.080254Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T18:51:28.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8aa0039630 msgr2=0x7f8aa003baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:28.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8aa0039630 0x7f8aa003baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006fd0 tx=0x7f8a9c006e40 comp rx=0 tx=0).stop 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 msgr2=0x7f8ab010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab010a430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40056d0 tx=0x7f8aa40057b0 comp rx=0 tx=0).stop 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 shutdown_connections 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f8aa0039630 0x7f8aa003baf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 --2- 172.21.15.194:0/1097934408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ab0101590 0x7f8ab010a430 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 >> 172.21.15.194:0/1097934408 conn(0x7f8ab00fce40 msgr2=0x7f8ab0102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 shutdown_connections 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:28.682+0000 7f8ab7034700 1 -- 172.21.15.194:0/1097934408 wait complete. 2024-04-25T18:51:28.685 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-25T18:51:29.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:29 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/1097934408' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:51:29.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:29 smithi194 systemd[1]: Started Ceph mon.c for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:51:30.295 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:51:30.295 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:51:30.530 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:51:35.020 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 -- 172.21.15.194:0/2226951634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b00035c0 msgr2=0x7f58b0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2226951634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b00035c0 0x7f58b0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58c406a4a0 tx=0x7f58b400feb0 comp rx=0 tx=0).stop 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 -- 172.21.15.194:0/2226951634 shutdown_connections 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2226951634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b00035c0 0x7f58b0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 -- 172.21.15.194:0/2226951634 >> 172.21.15.194:0/2226951634 conn(0x7f58c40fcf20 msgr2=0x7f58c40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 -- 172.21.15.194:0/2226951634 shutdown_connections 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.017+0000 7f58c96cb700 1 -- 172.21.15.194:0/2226951634 wait complete. 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c96cb700 1 Processor -- start 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c96cb700 1 -- start start 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c96cb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:35.021 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c96cb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58c4101890 0x7f58c4101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c96cb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410a480 con 0x7f58c4101890 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c96cb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410a300 con 0x7f58b00035c0 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c2ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c2ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:33854/0 (socket says 172.21.15.194:33854) 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c2ffd700 1 -- 172.21.15.194:0/2931900360 learned_addr learned my addr 172.21.15.194:0/2931900360 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.018+0000 7f58c2ffd700 1 -- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 msgr2=0x7f58c4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58c2ffd700 1 -- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 msgr2=0x7f58c4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58c2ffd700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58c2ffd700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58ba7fc700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58c4101890 0x7f58c4101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:35.022 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58ba7fc700 1 -- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 msgr2=0x7f58c4101350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:51:35.023 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58ba7fc700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:35.023 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.019+0000 7f58ba7fc700 1 -- 172.21.15.194:0/2931900360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c4103270 con 0x7f58c4101890 2024-04-25T18:51:35.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.020+0000 7f58ba7fc700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58c4101890 0x7f58c4101d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00b4d0 tx=0x7f58ac00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:35.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.020+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f58ac011630 con 0x7f58c4101890 2024-04-25T18:51:35.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.020+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c41a4370 con 0x7f58c4101890 2024-04-25T18:51:35.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.020+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c41a48b0 con 0x7f58c4101890 2024-04-25T18:51:35.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.020+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58ac011c80 con 0x7f58c4101890 2024-04-25T18:51:35.024 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.020+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f58ac01adc0 con 0x7f58c4101890 2024-04-25T18:51:35.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.021+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c406aca0 con 0x7f58c4101890 2024-04-25T18:51:35.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.021+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f58ac018020 con 0x7f58c4101890 2024-04-25T18:51:35.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.022+0000 7f58c0ff9700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f58a40393f0 0x7f58a403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:35.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.022+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f58ac04c210 con 0x7f58c4101890 2024-04-25T18:51:35.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.022+0000 7f58c2ffd700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f58a40393f0 0x7f58a403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:35.025 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.023+0000 7f58c2ffd700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f58a40393f0 0x7f58a403b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f58b400fa60 tx=0x7f58b4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:35.038 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.037+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58ac01f020 con 0x7f58c4101890 2024-04-25T18:51:35.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:51:35.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: mon.a calling monitor election 2024-04-25T18:51:35.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: mon.c calling monitor election 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: monmap epoch 2 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: last_changed 2024-04-25T18:51:29.761966+0000 2024-04-25T18:51:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: min_mon_release 19 (squid) 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: election_strategy: 1 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: fsmap 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: mgrmap e13: x(active, since 57s) 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: overall HEALTH_OK 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:35.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:35.316 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.315+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f58c41a4d20 con 0x7f58c4101890 2024-04-25T18:51:35.317 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.316+0000 7f58c0ff9700 1 -- 172.21.15.194:0/2931900360 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f58ac017dd0 con 0x7f58c4101890 2024-04-25T18:51:35.317 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:51:35.318 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","modified":"2024-04-25T18:51:29.761966Z","created":"2024-04-25T18:49:19.080254Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:51:35.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f58a40393f0 msgr2=0x7f58a403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f58a40393f0 0x7f58a403b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f58b400fa60 tx=0x7f58b4005a90 comp rx=0 tx=0).stop 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58c4101890 msgr2=0x7f58c4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58c4101890 0x7f58c4101d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00b4d0 tx=0x7f58ac00b890 comp rx=0 tx=0).stop 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 shutdown_connections 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f58a40393f0 0x7f58a403b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58b00035c0 0x7f58c4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 --2- 172.21.15.194:0/2931900360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58c4101890 0x7f58c4101d10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 >> 172.21.15.194:0/2931900360 conn(0x7f58c40fcf20 msgr2=0x7f58c40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 shutdown_connections 2024-04-25T18:51:35.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:35.318+0000 7f58c96cb700 1 -- 172.21.15.194:0/2931900360 wait complete. 2024-04-25T18:51:35.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:51:35.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: mon.a calling monitor election 2024-04-25T18:51:35.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: mon.c calling monitor election 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: monmap epoch 2 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: last_changed 2024-04-25T18:51:29.761966+0000 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:35.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: min_mon_release 19 (squid) 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: election_strategy: 1 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: fsmap 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: mgrmap e13: x(active, since 57s) 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: overall HEALTH_OK 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:35.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:35.324 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-04-25T18:51:35.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:35 smithi194 ceph-mon[32740]: Deploying daemon mon.b on smithi183 2024-04-25T18:51:35.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:35 smithi194 ceph-mon[32740]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:35.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:35 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/2931900360' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:51:35.944 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:36.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:35 smithi086 ceph-mon[27783]: Deploying daemon mon.b on smithi183 2024-04-25T18:51:36.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:35 smithi086 ceph-mon[27783]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:36.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:35 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/2931900360' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:51:36.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:36.969 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:51:36.970 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:51:37.231 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:51:38.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:38 smithi194 ceph-mon[32740]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:38 smithi086 ceph-mon[27783]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:38.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/4123977400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8108620 msgr2=0x7fe4f8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:38.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/4123977400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8108620 0x7fe4f8108a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009910 tx=0x7fe4ec009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:38.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/4123977400 shutdown_connections 2024-04-25T18:51:38.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/4123977400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8108620 0x7fe4f8108a00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:38.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/4123977400 >> 172.21.15.194:0/4123977400 conn(0x7fe4f80fe040 msgr2=0x7fe4f8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:38.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/4123977400 shutdown_connections 2024-04-25T18:51:38.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.618+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/4123977400 wait complete. 2024-04-25T18:51:38.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4ff0a0700 1 Processor -- start 2024-04-25T18:51:38.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4ff0a0700 1 -- start start 2024-04-25T18:51:38.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4ff0a0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:38.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4ff0a0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8198630 0x7fe4f819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:38.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4ff0a0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810ca90 con 0x7fe4f8198630 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4ff0a0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810c910 con 0x7fe4f8108620 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:33864/0 (socket says 172.21.15.194:33864) 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 -- 172.21.15.194:0/614357397 learned_addr learned my addr 172.21.15.194:0/614357397 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 -- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 msgr2=0x7fe4f81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 -- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 msgr2=0x7fe4f81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4fce3c700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.619+0000 7fe4f7fff700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8198630 0x7fe4f819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.620+0000 7fe4f7fff700 1 -- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 msgr2=0x7fe4f81980f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.620+0000 7fe4f7fff700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:38.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.620+0000 7fe4f7fff700 1 -- 172.21.15.194:0/614357397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f8198c70 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.621+0000 7fe4f7fff700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8198630 0x7fe4f819c9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b4d0 tx=0x7fe4e400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.621+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe4e4011630 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.621+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f819cf00 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.621+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4e4011c80 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.621+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe4e401adc0 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.621+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f819d4e0 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.622+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f8103c90 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.622+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe4e4018020 con 0x7fe4f8198630 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.623+0000 7fe4f5ffb700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe4e80393f0 0x7fe4e803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:38.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.623+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe4e404c270 con 0x7fe4f8198630 2024-04-25T18:51:38.625 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.623+0000 7fe4fce3c700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe4e80393f0 0x7fe4e803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:38.625 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.624+0000 7fe4fce3c700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe4e80393f0 0x7fe4e803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009750 tx=0x7fe4ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:38.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.639+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4e401f020 con 0x7fe4f8198630 2024-04-25T18:51:38.791 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.790+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40177d0 con 0x7fe4f8198630 2024-04-25T18:51:38.921 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:38.920+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe4f819d7a0 con 0x7fe4f8198630 2024-04-25T18:51:39.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:38 smithi183 systemd[1]: Started Ceph mon.b for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:51:43.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.809+0000 7fe4f5ffb700 1 -- 172.21.15.194:0/614357397 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fe4e4049520 con 0x7fe4f8198630 2024-04-25T18:51:43.811 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-25T18:51:43.811 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":3,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","modified":"2024-04-25T18:51:38.788202Z","created":"2024-04-25T18:49:19.080254Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.812+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe4e80393f0 msgr2=0x7fe4e803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.812+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe4e80393f0 0x7fe4e803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009750 tx=0x7fe4ec01a040 comp rx=0 tx=0).stop 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.812+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8198630 msgr2=0x7fe4f819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.812+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8198630 0x7fe4f819c9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b4d0 tx=0x7fe4e400b890 comp rx=0 tx=0).stop 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 shutdown_connections 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe4e80393f0 0x7fe4e803b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f8108620 0x7fe4f81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 --2- 172.21.15.194:0/614357397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f8198630 0x7fe4f819c9c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:43.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 >> 172.21.15.194:0/614357397 conn(0x7fe4f80fe040 msgr2=0x7fe4f80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:43.815 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 shutdown_connections 2024-04-25T18:51:43.815 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:43.813+0000 7fe4ff0a0700 1 -- 172.21.15.194:0/614357397 wait complete. 2024-04-25T18:51:43.815 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 3 2024-04-25T18:51:44.129 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: mon.c calling monitor election 2024-04-25T18:51:44.129 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:51:44.129 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: mon.a calling monitor election 2024-04-25T18:51:44.129 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.129 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/614357397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: monmap epoch 3 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: last_changed 2024-04-25T18:51:38.788202+0000 2024-04-25T18:51:44.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: min_mon_release 19 (squid) 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: election_strategy: 1 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: fsmap 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: mgrmap e13: x(active, since 66s) 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: overall HEALTH_OK 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: mon.c calling monitor election 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: mon.a calling monitor election 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/614357397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:51:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: monmap epoch 3 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: last_changed 2024-04-25T18:51:38.788202+0000 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: min_mon_release 19 (squid) 2024-04-25T18:51:44.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: election_strategy: 1 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: fsmap 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: mgrmap e13: x(active, since 66s) 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: overall HEALTH_OK 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:44.433 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T18:51:44.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T18:51:44.842 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:44.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:44 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:44.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:44 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:45.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:44 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:45.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:44 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: mon.c calling monitor election 2024-04-25T18:51:46.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:51:46.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: mon.a calling monitor election 2024-04-25T18:51:46.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/614357397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: monmap epoch 3 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:46.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: last_changed 2024-04-25T18:51:38.788202+0000 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: min_mon_release 19 (squid) 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: election_strategy: 1 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: fsmap 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: mgrmap e13: x(active, since 66s) 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: overall HEALTH_OK 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:46.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:46.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: mon.b calling monitor election 2024-04-25T18:51:46.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: mon.b calling monitor election 2024-04-25T18:51:46.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: mon.c calling monitor election 2024-04-25T18:51:46.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: mon.a calling monitor election 2024-04-25T18:51:46.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:51:46.533 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: monmap epoch 3 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: last_changed 2024-04-25T18:51:38.788202+0000 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: min_mon_release 19 (squid) 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: election_strategy: 1 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: fsmap 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: mgrmap e13: x(active, since 68s) 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: overall HEALTH_OK 2024-04-25T18:51:46.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: mon.b calling monitor election 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: mon.b calling monitor election 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: mon.c calling monitor election 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: mon.a calling monitor election 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: monmap epoch 3 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: last_changed 2024-04-25T18:51:38.788202+0000 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: min_mon_release 19 (squid) 2024-04-25T18:51:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: election_strategy: 1 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: fsmap 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: mgrmap e13: x(active, since 68s) 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: overall HEALTH_OK 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 -- 172.21.15.86:0/2932444045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10701084e0 msgr2=0x7f10701088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 --2- 172.21.15.86:0/2932444045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10701084e0 0x7f10701088c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1060009910 tx=0x7f1060009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 -- 172.21.15.86:0/2932444045 shutdown_connections 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 --2- 172.21.15.86:0/2932444045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10701084e0 0x7f10701088c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 -- 172.21.15.86:0/2932444045 >> 172.21.15.86:0/2932444045 conn(0x7f10700fdf80 msgr2=0x7f10701003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 -- 172.21.15.86:0/2932444045 shutdown_connections 2024-04-25T18:51:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.690+0000 7f1076200700 1 -- 172.21.15.86:0/2932444045 wait complete. 2024-04-25T18:51:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.691+0000 7f1076200700 1 Processor -- start 2024-04-25T18:51:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 -- start start 2024-04-25T18:51:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 0x7f107019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f107019a890 0x7f107019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107019f160 0x7f107019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f107010c940 con 0x7f107019f160 2024-04-25T18:51:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f107010c7c0 con 0x7f10701084e0 2024-04-25T18:51:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f1076200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f107010cac0 con 0x7f107019a890 2024-04-25T18:51:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f106f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 0x7f107019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f106f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 0x7f107019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48784/0 (socket says 172.21.15.86:48784) 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f106f400700 1 -- 172.21.15.86:0/278041098 learned_addr learned my addr 172.21.15.86:0/278041098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.692+0000 7f106ea00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f107019a890 0x7f107019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107019f160 0x7f107019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 -- 172.21.15.86:0/278041098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f107019a890 msgr2=0x7f107019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f107019a890 0x7f107019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 -- 172.21.15.86:0/278041098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 msgr2=0x7f107019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 0x7f107019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 -- 172.21.15.86:0/278041098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10701a6ed0 con 0x7f107019f160 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106ea00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f107019a890 0x7f107019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106f400700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 0x7f107019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.693+0000 7f106fe00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107019f160 0x7f107019f610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f106800bd50 tx=0x7f106800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.694+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068019070 con 0x7f107019f160 2024-04-25T18:51:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.694+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f106800cbb0 con 0x7f107019f160 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.694+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10701a7100 con 0x7f107019f160 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.694+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680125a0 con 0x7f107019f160 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.694+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701a7690 con 0x7f107019f160 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.695+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f106801c070 con 0x7f107019f160 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.695+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10700640a0 con 0x7f107019f160 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.696+0000 7f1067e00700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1054039ae0 0x7f105403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:46.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.696+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f106804c120 con 0x7f107019f160 2024-04-25T18:51:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.696+0000 7f106f400700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1054039ae0 0x7f105403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.697+0000 7f106f400700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1054039ae0 0x7f105403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1060009750 tx=0x7f106001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.702+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10680133a0 con 0x7f107019f160 2024-04-25T18:51:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: mon.b calling monitor election 2024-04-25T18:51:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: mon.b calling monitor election 2024-04-25T18:51:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: mon.c calling monitor election 2024-04-25T18:51:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: mon.a calling monitor election 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: monmap epoch 3 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: last_changed 2024-04-25T18:51:38.788202+0000 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: created 2024-04-25T18:49:19.080254+0000 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: min_mon_release 19 (squid) 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: election_strategy: 1 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: fsmap 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: mgrmap e13: x(active, since 68s) 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: overall HEALTH_OK 2024-04-25T18:51:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.938+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f10701a7950 con 0x7f107019f160 2024-04-25T18:51:46.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.938+0000 7f1067e00700 1 -- 172.21.15.86:0/278041098 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f1068012c60 con 0x7f107019f160 2024-04-25T18:51:46.941 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:46.941 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-25T18:51:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T18:51:46.941 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2024-04-25T18:51:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1054039ae0 msgr2=0x7f105403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1054039ae0 0x7f105403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1060009750 tx=0x7f106001a040 comp rx=0 tx=0).stop 2024-04-25T18:51:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107019f160 msgr2=0x7f107019f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107019f160 0x7f107019f610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f106800bd50 tx=0x7f106800bd80 comp rx=0 tx=0).stop 2024-04-25T18:51:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 shutdown_connections 2024-04-25T18:51:46.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1054039ae0 0x7f105403bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107019f160 0x7f107019f610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701084e0 0x7f107019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 --2- 172.21.15.86:0/278041098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f107019a890 0x7f107019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:46.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.941+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 >> 172.21.15.86:0/278041098 conn(0x7f10700fdf80 msgr2=0x7f1070109ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:46.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.942+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 shutdown_connections 2024-04-25T18:51:46.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:46.942+0000 7f1076200700 1 -- 172.21.15.86:0/278041098 wait complete. 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/278041098' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:47.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.621 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T18:51:47.621 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:51:47.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:51:47.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/278041098' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.693 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:51:47.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:51:47.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:51:47.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/278041098' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:47.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:47.813 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:51:47.813 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:51:47.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:51:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:51:47.787+0000 7fa0fe600700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T18:51:47.840 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:51:47.840 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:51:47.905 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:51:47.906 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:51:47.936 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:51:47.936 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:51:48.004 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-04-25T18:51:48.004 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-04-25T18:51:48.004 INFO:tasks.cephadm:Adding mgr.z on smithi194 2024-04-25T18:51:48.004 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi086=x;smithi183=y;smithi194=z' 2024-04-25T18:51:48.278 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:51:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:48 smithi183 ceph-mon[32344]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:51:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:48 smithi183 ceph-mon[32344]: Reconfiguring daemon mon.a on smithi086 2024-04-25T18:51:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:48 smithi183 ceph-mon[32344]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:48.685 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:48 smithi194 ceph-mon[32740]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:51:48.685 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:48 smithi194 ceph-mon[32740]: Reconfiguring daemon mon.a on smithi086 2024-04-25T18:51:48.685 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:48 smithi194 ceph-mon[32740]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:48.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:48 smithi086 ceph-mon[27783]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:51:48.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:48 smithi086 ceph-mon[27783]: Reconfiguring daemon mon.a on smithi086 2024-04-25T18:51:48.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:48 smithi086 ceph-mon[27783]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 -- 172.21.15.194:0/1916523466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 msgr2=0x7f6df8100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 --2- 172.21.15.194:0/1916523466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8100cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009910 tx=0x7f6df4009c20 comp rx=0 tx=0).stop 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 -- 172.21.15.194:0/1916523466 shutdown_connections 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 --2- 172.21.15.194:0/1916523466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8100cd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 -- 172.21.15.194:0/1916523466 >> 172.21.15.194:0/1916523466 conn(0x7f6df80fc1f0 msgr2=0x7f6df80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 -- 172.21.15.194:0/1916523466 shutdown_connections 2024-04-25T18:51:49.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.769+0000 7f6dffb59700 1 -- 172.21.15.194:0/1916523466 wait complete. 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 Processor -- start 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 -- start start 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df8198230 0x7f6df819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 0x7f6df819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81084b0 con 0x7f6df81008b0 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8108330 con 0x7f6df8198230 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dffb59700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8108630 con 0x7f6df81986b0 2024-04-25T18:51:49.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dfe0f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 0x7f6df819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dfe0f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 0x7f6df819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:45714/0 (socket says 172.21.15.194:45714) 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dfd0f4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df8198230 0x7f6df819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.770+0000 7f6dfe0f6700 1 -- 172.21.15.194:0/3767665245 learned_addr learned my addr 172.21.15.194:0/3767665245 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfd8f5700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfe0f6700 1 -- 172.21.15.194:0/3767665245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df8198230 msgr2=0x7f6df819d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfe0f6700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df8198230 0x7f6df819d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfe0f6700 1 -- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 msgr2=0x7f6df8197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfe0f6700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfe0f6700 1 -- 172.21.15.194:0/3767665245 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df81a0110 con 0x7f6df81986b0 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfd8f5700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dfe0f6700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 0x7f6df819f810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6df000ba50 tx=0x7f6df000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0019070 con 0x7f6df81986b0 2024-04-25T18:51:49.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df0014440 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df81a0400 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df000ce60 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.771+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df81a0970 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.772+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6df001c070 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.772+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df80640a0 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.772+0000 7f6deeffd700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6de4039ae0 0x7f6de403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.772+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6df004be10 con 0x7f6df81986b0 2024-04-25T18:51:49.776 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.773+0000 7f6dfd8f5700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6de4039ae0 0x7f6de403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:49.777 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.774+0000 7f6dfd8f5700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6de4039ae0 0x7f6de403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6df400b5c0 tx=0x7f6df4005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:49.779 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.778+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6df0014910 con 0x7f6df81986b0 2024-04-25T18:51:49.989 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:49.987+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}) -- 0x7f6df8062f10 con 0x7f6de4039ae0 2024-04-25T18:51:50.002 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.001+0000 7f6deeffd700 1 -- 172.21.15.194:0/3767665245 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6df8062f10 con 0x7f6de4039ae0 2024-04-25T18:51:50.002 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mgr update... 2024-04-25T18:51:50.004 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6de4039ae0 msgr2=0x7f6de403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:50.004 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6de4039ae0 0x7f6de403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6df400b5c0 tx=0x7f6df4005260 comp rx=0 tx=0).stop 2024-04-25T18:51:50.004 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 msgr2=0x7f6df819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:50.004 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 0x7f6df819f810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6df000ba50 tx=0x7f6df000bd60 comp rx=0 tx=0).stop 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 shutdown_connections 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6de4039ae0 0x7f6de403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81986b0 0x7f6df819f810 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df81008b0 0x7f6df8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 --2- 172.21.15.194:0/3767665245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6df8198230 0x7f6df819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 >> 172.21.15.194:0/3767665245 conn(0x7f6df80fc1f0 msgr2=0x7f6df80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 shutdown_connections 2024-04-25T18:51:50.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:51:50.003+0000 7f6dffb59700 1 -- 172.21.15.194:0/3767665245 wait complete. 2024-04-25T18:51:50.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: Reconfiguring daemon mon.b on smithi183 2024-04-25T18:51:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:49 smithi183 ceph-mon[32344]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: Reconfiguring daemon mon.b on smithi183 2024-04-25T18:51:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:49 smithi194 ceph-mon[32740]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:50.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:50.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:50.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:51:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: Reconfiguring daemon mon.b on smithi183 2024-04-25T18:51:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:49 smithi086 ceph-mon[27783]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:50.676 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T18:51:50.679 DEBUG:teuthology.orchestra.run.smithi194:mgr.z> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T18:51:50.681 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T18:51:50.681 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:51:50.681 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:51:50.741 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:51:50.741 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-25T18:51:50.794 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:51:50.794 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:50.794 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T18:51:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 18:50:49.920022837 +0000 2024-04-25T18:51:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 18:43:17.728788462 +0000 2024-04-25T18:51:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 18:43:17.728788462 +0000 2024-04-25T18:51:50.795 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 18:43:17.728788462 +0000 2024-04-25T18:51:50.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:51:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T18:51:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T18:51:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000404254 s, 1.3 MB/s 2024-04-25T18:51:50.876 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:51:50.937 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-25T18:51:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:51:50.984 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:50.984 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-25T18:51:50.984 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:50.984 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:50.985 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 18:50:50.035020438 +0000 2024-04-25T18:51:50.985 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 18:43:18.326776076 +0000 2024-04-25T18:51:50.985 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 18:43:18.326776076 +0000 2024-04-25T18:51:50.985 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 18:43:18.326776076 +0000 2024-04-25T18:51:50.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T18:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T18:51:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000330419 s, 1.5 MB/s 2024-04-25T18:51:51.054 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:51:51.105 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-25T18:51:51.149 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:51:51.149 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:51.149 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-25T18:51:51.149 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:51.149 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:51.149 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 18:50:50.139018268 +0000 2024-04-25T18:51:51.150 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 18:43:18.929763585 +0000 2024-04-25T18:51:51.150 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 18:43:18.929763585 +0000 2024-04-25T18:51:51.150 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 18:43:18.929763585 +0000 2024-04-25T18:51:51.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: Saving service mgr spec with placement smithi086=x;smithi183=y;smithi194=z;count:3 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:51.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:51.262 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:51.262 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: Saving service mgr spec with placement smithi086=x;smithi183=y;smithi194=z;count:3 2024-04-25T18:51:51.262 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.262 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.262 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.263 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:51.263 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:51.263 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T18:51:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T18:51:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000509594 s, 1.0 MB/s 2024-04-25T18:51:51.275 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:51:51.329 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 18:50:50.218016620 +0000 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 18:43:19.520751344 +0000 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 18:43:19.520751344 +0000 2024-04-25T18:51:51.377 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 18:43:19.520751344 +0000 2024-04-25T18:51:51.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:51:51.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:51.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: Saving service mgr spec with placement smithi086=x;smithi183=y;smithi194=z;count:3 2024-04-25T18:51:51.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:51.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:51:51.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:51:51.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T18:51:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T18:51:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000364252 s, 1.4 MB/s 2024-04-25T18:51:51.458 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:51:51.515 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-25T18:51:51.515 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:51:51.532 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:51:51.532 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-04-25T18:51:51.587 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:51:51.587 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:51.587 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T18:51:51.587 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:51.588 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:51.588 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 18:51:38.253265033 +0000 2024-04-25T18:51:51.588 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 18:42:13.128317220 +0000 2024-04-25T18:51:51.588 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 18:42:13.128317220 +0000 2024-04-25T18:51:51.588 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 18:42:13.128317220 +0000 2024-04-25T18:51:51.588 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:51:51.650 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T18:51:51.650 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T18:51:51.650 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000221183 s, 2.3 MB/s 2024-04-25T18:51:51.651 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:51:51.708 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 18:51:38.253265033 +0000 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 18:42:13.465307807 +0000 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 18:42:13.465307807 +0000 2024-04-25T18:51:51.763 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 18:42:13.465307807 +0000 2024-04-25T18:51:51.763 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:51:51.825 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T18:51:51.825 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T18:51:51.825 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000188355 s, 2.7 MB/s 2024-04-25T18:51:51.826 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:51:51.882 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 18:51:38.254265006 +0000 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 18:42:13.808298226 +0000 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 18:42:13.808298226 +0000 2024-04-25T18:51:51.937 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 18:42:13.808298226 +0000 2024-04-25T18:51:51.938 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:51:51.999 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T18:51:52.000 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T18:51:52.000 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000195712 s, 2.6 MB/s 2024-04-25T18:51:52.001 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:51:52.069 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-04-25T18:51:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:51:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:52.128 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T18:51:52.129 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:52.129 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:52.129 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-25 18:51:38.254265006 +0000 2024-04-25T18:51:52.129 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-25 18:42:14.146288786 +0000 2024-04-25T18:51:52.129 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-25 18:42:14.146288786 +0000 2024-04-25T18:51:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-25 18:42:14.146288786 +0000 2024-04-25T18:51:52.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:51:52.191 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-25T18:51:52.191 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-25T18:51:52.191 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000218753 s, 2.3 MB/s 2024-04-25T18:51:52.192 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:51:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:52 smithi194 ceph-mon[32740]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:51:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:52 smithi194 ceph-mon[32740]: Reconfiguring daemon mon.c on smithi194 2024-04-25T18:51:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:52 smithi194 ceph-mon[32740]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:52.249 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-25T18:51:52.249 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:51:52.265 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:51:52.265 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-25 18:51:28.539901524 +0000 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-25 18:42:15.405816645 +0000 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-25 18:42:15.405816645 +0000 2024-04-25T18:51:52.320 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-25 18:42:15.405816645 +0000 2024-04-25T18:51:52.321 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:51:52.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:52 smithi086 ceph-mon[27783]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:51:52.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:52 smithi086 ceph-mon[27783]: Reconfiguring daemon mon.c on smithi194 2024-04-25T18:51:52.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:52 smithi086 ceph-mon[27783]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:52.384 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-25T18:51:52.384 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-25T18:51:52.385 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000273351 s, 1.9 MB/s 2024-04-25T18:51:52.386 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:51:52.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:52 smithi183 ceph-mon[32344]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:51:52.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:52 smithi183 ceph-mon[32344]: Reconfiguring daemon mon.c on smithi194 2024-04-25T18:51:52.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:52 smithi183 ceph-mon[32344]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:52.441 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-25 18:51:28.539901524 +0000 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-25 18:42:15.746809611 +0000 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-25 18:42:15.746809611 +0000 2024-04-25T18:51:52.497 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-25 18:42:15.746809611 +0000 2024-04-25T18:51:52.498 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:51:52.560 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-25T18:51:52.560 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-25T18:51:52.560 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000176467 s, 2.9 MB/s 2024-04-25T18:51:52.561 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:51:52.616 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-25 18:51:28.540901504 +0000 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-25 18:42:16.092802475 +0000 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-25 18:42:16.092802475 +0000 2024-04-25T18:51:52.671 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-25 18:42:16.092802475 +0000 2024-04-25T18:51:52.672 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:51:52.734 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-25T18:51:52.735 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-25T18:51:52.735 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000182486 s, 2.8 MB/s 2024-04-25T18:51:52.736 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:51:52.793 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-25 18:51:28.540901504 +0000 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-25 18:42:16.425795607 +0000 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-25 18:42:16.425795607 +0000 2024-04-25T18:51:52.849 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-04-25 18:42:16.425795607 +0000 2024-04-25T18:51:52.850 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:51:52.911 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-25T18:51:52.911 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-25T18:51:52.911 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000180785 s, 2.8 MB/s 2024-04-25T18:51:52.912 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:51:52.967 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-04-25T18:51:52.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:51:53.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:53.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:53 smithi086 ceph-mon[27783]: Deploying daemon mgr.z on smithi194 2024-04-25T18:51:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:53.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:53.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:53 smithi183 ceph-mon[32344]: Deploying daemon mgr.z on smithi194 2024-04-25T18:51:53.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:53 smithi194 ceph-mon[32740]: Deploying daemon mgr.z on smithi194 2024-04-25T18:51:54.162 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 systemd[1]: Starting Ceph mgr.z for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:51:54.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:54 smithi183 ceph-mon[32344]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:54.452 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:54 smithi194 ceph-mon[32740]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:54.453 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 podman[34588]: 2024-04-25 18:51:54.174134825 +0000 UTC m=+0.018126579 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:51:54.453 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 podman[34588]: 2024-04-25 18:51:54.274803639 +0000 UTC m=+0.118795420 container create c92e9a25aeb63552e80c119ad222896220c92c19dae65b6ff19a332a3e510e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T18:51:54.453 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 podman[34588]: 2024-04-25 18:51:54.452210033 +0000 UTC m=+0.296201772 container init c92e9a25aeb63552e80c119ad222896220c92c19dae65b6ff19a332a3e510e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T18:51:54.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:54 smithi086 ceph-mon[27783]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:54.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 podman[34588]: 2024-04-25 18:51:54.455079531 +0000 UTC m=+0.299071278 container start c92e9a25aeb63552e80c119ad222896220c92c19dae65b6ff19a332a3e510e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T18:51:54.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 ceph-mgr[34607]: -- 172.21.15.194:0/400583432 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c109845a0 con 0x558c10959000 2024-04-25T18:51:54.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 bash[34588]: c92e9a25aeb63552e80c119ad222896220c92c19dae65b6ff19a332a3e510e9b 2024-04-25T18:51:54.732 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 systemd[1]: Started Ceph mgr.z for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:51:54.732 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:54.607+0000 7fe58cf95200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:51:55.200 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:54.792+0000 7fe58cf95200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:51:55.201 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:54.893+0000 7fe58cf95200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:51:55.201 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:55.199+0000 7fe58cf95200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:51:55.481 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:55.283+0000 7fe58cf95200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: Deploying daemon mgr.y on smithi183 2024-04-25T18:51:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:55 smithi183 ceph-mon[32344]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:55.961 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:51:55.961 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:51:55.961 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-25T18:51:55.962 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-25T18:51:55.962 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0361356 s, 290 MB/s 2024-04-25T18:51:55.962 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:55.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: Deploying daemon mgr.y on smithi183 2024-04-25T18:51:55.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:55 smithi086 ceph-mon[27783]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:55.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: Deploying daemon mgr.y on smithi183 2024-04-25T18:51:55.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:55 smithi194 ceph-mon[32740]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:56.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-04-25T18:51:56.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:56.478+0000 7fe58cf95200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:51:56.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:56.652+0000 7fe58cf95200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:51:56.879 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:56 smithi183 systemd[1]: Starting Ceph mgr.y for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:51:57.112 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:56.824+0000 7fe58cf95200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:51:57.177 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:56 smithi183 podman[33711]: 2024-04-25 18:51:56.781130316 +0000 UTC m=+0.017694887 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:51:57.177 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:56 smithi183 podman[33711]: 2024-04-25 18:51:56.894633358 +0000 UTC m=+0.131197903 container create f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T18:51:57.178 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 podman[33711]: 2024-04-25 18:51:57.130409983 +0000 UTC m=+0.366974527 container init f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:51:57.178 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:57.409 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:57.111+0000 7fe58cf95200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:51:57.409 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:57.203+0000 7fe58cf95200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:51:57.409 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:57.407+0000 7fe58cf95200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:51:57.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 podman[33711]: 2024-04-25 18:51:57.132927311 +0000 UTC m=+0.369491857 container start f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:51:57.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 ceph-mgr[33729]: -- 172.21.15.183:0/2919240046 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3dc52c5a0 con 0x55b3dc501000 2024-04-25T18:51:57.417 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 bash[33711]: f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f 2024-04-25T18:51:57.417 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 systemd[1]: Started Ceph mgr.y for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:51:57.417 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:57.288+0000 7f747a4e8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:51:57.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:57.602+0000 7fe58cf95200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:51:57.905 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:57.473+0000 7f747a4e8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:51:57.905 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:57.577+0000 7f747a4e8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:51:57.906 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:57.904+0000 7f747a4e8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:51:58.166 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:57.991+0000 7f747a4e8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:51:58.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:58 smithi086 ceph-mon[27783]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:58.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:58.234+0000 7fe58cf95200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-mon[32740]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.666 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:58 smithi183 ceph-mon[32344]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:51:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:51:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:58.971 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:58.664+0000 7fe58cf95200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:51:58.971 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:58.765+0000 7fe58cf95200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:51:58.971 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:58.969+0000 7fe58cf95200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:51:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 -- 172.21.15.86:0/4121226913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c102ab0 msgr2=0x7f711c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 --2- 172.21.15.86:0/4121226913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c102ab0 0x7f711c102ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71100099c0 tx=0x7f7110009cd0 comp rx=0 tx=0).stop 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 -- 172.21.15.86:0/4121226913 shutdown_connections 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 --2- 172.21.15.86:0/4121226913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c104650 0x7f711c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 --2- 172.21.15.86:0/4121226913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c102ab0 0x7f711c102ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 --2- 172.21.15.86:0/4121226913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f711c103cb0 0x7f711c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 -- 172.21.15.86:0/4121226913 >> 172.21.15.86:0/4121226913 conn(0x7f711c0fe050 msgr2=0x7f711c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 -- 172.21.15.86:0/4121226913 shutdown_connections 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.021+0000 7f7123e00700 1 -- 172.21.15.86:0/4121226913 wait complete. 2024-04-25T18:51:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 Processor -- start 2024-04-25T18:51:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 -- start start 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f711c102ab0 0x7f711c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 0x7f711c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c104650 0x7f711c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f711c10e570 con 0x7f711c102ab0 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f711c10e3f0 con 0x7f711c104650 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7123e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f711c10e6f0 con 0x7f711c103cb0 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7120a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 0x7f711c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7121400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f711c102ab0 0x7f711c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7120a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 0x7f711c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36134/0 (socket says 172.21.15.86:36134) 2024-04-25T18:51:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.023+0000 7f7120a00700 1 -- 172.21.15.86:0/3036082072 learned_addr learned my addr 172.21.15.86:0/3036082072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:51:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7121e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c104650 0x7f711c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7120a00700 1 -- 172.21.15.86:0/3036082072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c104650 msgr2=0x7f711c1a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7120a00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c104650 0x7f711c1a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7120a00700 1 -- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f711c102ab0 msgr2=0x7f711c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7120a00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f711c102ab0 0x7f711c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7120a00700 1 -- 172.21.15.86:0/3036082072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c1a2a60 con 0x7f711c103cb0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f7120a00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 0x7f711c19b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9d0 tx=0x7f710800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.024+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710800cbd0 con 0x7f711c103cb0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.025+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710800ee70 con 0x7f711c103cb0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.025+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108018680 con 0x7f711c103cb0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.025+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c1a2d50 con 0x7f711c103cb0 2024-04-25T18:51:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.025+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1a31e0 con 0x7f711c103cb0 2024-04-25T18:51:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.026+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7108010430 con 0x7f711c103cb0 2024-04-25T18:51:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.026+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c1a3410 con 0x7f711c103cb0 2024-04-25T18:51:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.026+0000 7f711a000700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7104039890 0x7f710403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.026+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f710804cc70 con 0x7f711c103cb0 2024-04-25T18:51:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.027+0000 7f7121400700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7104039890 0x7f710403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.027+0000 7f7121400700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7104039890 0x7f710403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f711000b580 tx=0x7f7110005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.032+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7108018d40 con 0x7f711c103cb0 2024-04-25T18:51:59.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:51:59.287+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f711c076390 con 0x7f7104039890 2024-04-25T18:51:59.575 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:59.236+0000 7f747a4e8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:51:59.575 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:59.403+0000 7f747a4e8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:51:59.819 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:59.492+0000 7fe58cf95200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:51:59.819 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:59.581+0000 7fe58cf95200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:51:59.829 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:59.573+0000 7f747a4e8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:52:00.106 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.106 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.106 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: Reconfiguring daemon mgr.x on smithi086 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:00.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:51:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.154 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:59.857+0000 7f747a4e8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:52:00.154 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:51:59.949+0000 7f747a4e8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:52:00.154 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:00.153+0000 7f747a4e8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:52:00.155 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: Reconfiguring daemon mgr.x on smithi086 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:51:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:59.818+0000 7fe58cf95200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:52:00.156 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:51:59.949+0000 7fe58cf95200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:52:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:52:00.041+0000 7fe58cf95200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:52:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:52:00.153+0000 7fe58cf95200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:52:00.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: Reconfiguring daemon mgr.x on smithi086 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:00.158 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:51:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:00.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:00.343+0000 7f747a4e8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:52:00.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:52:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:52:00.454+0000 7fe58cf95200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:52:01.352 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:00.931+0000 7f747a4e8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:52:01.481 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:52:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:52:01.044+0000 7fe58cf95200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:52:01.481 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 18:52:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T18:52:01.256+0000 7fe58cf95200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:52:01.645 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:01.350+0000 7f747a4e8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:52:01.645 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:01.446+0000 7f747a4e8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:52:01.645 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:01.643+0000 7f747a4e8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:52:02.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:02.047+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f7108018820 con 0x7f711c103cb0 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: Standby manager daemon z started 2024-04-25T18:52:02.110 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:52:02.111 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:52:02.111 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:52:02.111 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:02 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:52:02.263 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:02.171+0000 7f747a4e8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:52:02.263 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:02.261+0000 7f747a4e8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: Standby manager daemon z started 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:52:02.264 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:52:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: Standby manager daemon z started 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:52:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:02 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/911232242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:52:02.629 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:02.498+0000 7f747a4e8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:52:02.629 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:02.628+0000 7f747a4e8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:52:02.916 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:02.722+0000 7f747a4e8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:52:02.916 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:02.835+0000 7f747a4e8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:52:03.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:03 smithi086 ceph-mon[27783]: mgrmap e14: x(active, since 84s), standbys: z 2024-04-25T18:52:03.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:52:03.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:03.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:03.140+0000 7f747a4e8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:52:03.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:03 smithi183 ceph-mon[32344]: mgrmap e14: x(active, since 84s), standbys: z 2024-04-25T18:52:03.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:52:03.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:03.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:03 smithi194 ceph-mon[32740]: mgrmap e14: x(active, since 84s), standbys: z 2024-04-25T18:52:03.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:52:03.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:04.055 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:03.728+0000 7f747a4e8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:52:04.056 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 18:52:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T18:52:03.942+0000 7f747a4e8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:52:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/4150909378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]: dispatch 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]: dispatch 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]': finished 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: Standby manager daemon y started 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:52:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:04 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:52:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/4150909378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]: dispatch 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]: dispatch 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]': finished 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: Standby manager daemon y started 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:52:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:04 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:52:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/4150909378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]: dispatch 2024-04-25T18:52:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]: dispatch 2024-04-25T18:52:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "983ae9a3-f0a7-436b-996b-e9916864d30e"}]': finished 2024-04-25T18:52:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:52:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: Standby manager daemon y started 2024-04-25T18:52:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:52:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:52:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:52:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:04 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/3843897344' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:52:04.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:04.944+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mon.1 v2:172.21.15.194:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f7108010720 con 0x7f711c103cb0 2024-04-25T18:52:05.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:05 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2738060415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:05.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:05 smithi183 ceph-mon[32344]: mgrmap e15: x(active, since 87s), standbys: z, y 2024-04-25T18:52:05.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:52:05.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:05 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2738060415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:05.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:05 smithi086 ceph-mon[27783]: mgrmap e15: x(active, since 87s), standbys: z, y 2024-04-25T18:52:05.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:52:05.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:05 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2738060415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:05.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:05 smithi194 ceph-mon[32740]: mgrmap e15: x(active, since 87s), standbys: z, y 2024-04-25T18:52:05.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:52:06.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:06 smithi183 ceph-mon[32344]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:06.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:06 smithi194 ceph-mon[32740]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:06 smithi086 ceph-mon[27783]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:08.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:08 smithi086 ceph-mon[27783]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:08.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:08 smithi183 ceph-mon[32344]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:08 smithi194 ceph-mon[32740]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:10.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:10 smithi086 ceph-mon[27783]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:10.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:10 smithi183 ceph-mon[32344]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:10 smithi194 ceph-mon[32740]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:12 smithi086 ceph-mon[27783]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:12 smithi183 ceph-mon[32344]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:12 smithi194 ceph-mon[32740]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:52:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:13.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:52:13.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:13.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:52:13.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:14.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:14 smithi086 ceph-mon[27783]: Deploying daemon osd.0 on smithi086 2024-04-25T18:52:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:14 smithi086 ceph-mon[27783]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:14 smithi183 ceph-mon[32344]: Deploying daemon osd.0 on smithi086 2024-04-25T18:52:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:14 smithi183 ceph-mon[32344]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:14 smithi194 ceph-mon[32740]: Deploying daemon osd.0 on smithi086 2024-04-25T18:52:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:14 smithi194 ceph-mon[32740]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:16.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:16 smithi086 ceph-mon[27783]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:16.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:16 smithi183 ceph-mon[32344]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:16 smithi194 ceph-mon[32740]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:18.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:18 smithi086 ceph-mon[27783]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:18.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:18 smithi183 ceph-mon[32344]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:18 smithi194 ceph-mon[32740]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:20.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:20 smithi086 ceph-mon[27783]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:20.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:20 smithi183 ceph-mon[32344]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:20 smithi194 ceph-mon[32740]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:21.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:21.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:21.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:21.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:21 smithi086 ceph-mon[27783]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:21 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:21 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:21 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:21 smithi194 ceph-mon[32740]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:21.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:21 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:21.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:21 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:21.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:21 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:21.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:21 smithi183 ceph-mon[32344]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:24.445 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:24 smithi086 ceph-mon[27783]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:24.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:24.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:24.446 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:24 smithi183 ceph-mon[32344]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:24.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:24 smithi194 ceph-mon[32740]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:24.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.920+0000 7f711a000700 1 -- 172.21.15.86:0/3036082072 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f711c076390 con 0x7f7104039890 2024-04-25T18:52:24.922 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-04-25T18:52:24.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7104039890 msgr2=0x7f710403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:24.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7104039890 0x7f710403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f711000b580 tx=0x7f7110005fb0 comp rx=0 tx=0).stop 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 msgr2=0x7f711c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 0x7f711c19b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9d0 tx=0x7f710800ece0 comp rx=0 tx=0).stop 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 shutdown_connections 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7104039890 0x7f710403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c104650 0x7f711c1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f711c102ab0 0x7f711c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 --2- 172.21.15.86:0/3036082072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f711c103cb0 0x7f711c19b340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 >> 172.21.15.86:0/3036082072 conn(0x7f711c0fe050 msgr2=0x7f711c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.924+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 shutdown_connections 2024-04-25T18:52:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:52:24.925+0000 7f7123e00700 1 -- 172.21.15.86:0/3036082072 wait complete. 2024-04-25T18:52:25.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:25 smithi086 ceph-mon[27783]: from='osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:52:25.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:25 smithi086 ceph-mon[27783]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:52:25.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:25.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:25.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:25.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:25 smithi183 ceph-mon[32344]: from='osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:52:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:25 smithi183 ceph-mon[32344]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:52:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:25 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:25 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:25 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:25.699 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T18:52:25.703 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-04-25T18:52:25.703 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:52:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:25 smithi194 ceph-mon[32740]: from='osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:52:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:25 smithi194 ceph-mon[32740]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:52:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:25 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:25 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:25 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:25.944 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:52:26.362 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:26 smithi183 ceph-mon[32344]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:26.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:26 smithi183 ceph-mon[32344]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:52:26.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:26 smithi183 ceph-mon[32344]: from='osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:52:26.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:26 smithi183 ceph-mon[32344]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:52:26.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:26 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:26.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:26 smithi183 ceph-mon[32344]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:52:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-mon[27783]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-mon[27783]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:52:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-mon[27783]: from='osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:52:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-mon[27783]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:52:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-mon[27783]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:52:26.571 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0[40174]: 2024-04-25T18:52:26.316+0000 7ff95b200700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T18:52:26.571 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:52:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0[40174]: 2024-04-25T18:52:26.327+0000 7ff956000700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:52:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:26 smithi194 ceph-mon[32740]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:26 smithi194 ceph-mon[32740]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:52:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:26 smithi194 ceph-mon[32740]: from='osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:52:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:26 smithi194 ceph-mon[32740]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:52:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:26 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:26 smithi194 ceph-mon[32740]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:52:27.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:27 smithi086 ceph-mon[27783]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:52:27.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:27 smithi086 ceph-mon[27783]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:52:27.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:27 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:27.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:27 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:27.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:27 smithi183 ceph-mon[32344]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:52:27.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:27 smithi183 ceph-mon[32344]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:52:27.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:27 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:27.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:27 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:27 smithi194 ceph-mon[32740]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:52:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:27 smithi194 ceph-mon[32740]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:52:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:27 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:27 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:27.836 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:52:27.944 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:52:27.945 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T18:52:27.945 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T18:52:27.945 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121057 s, 866 MB/s 2024-04-25T18:52:27.945 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] boot 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:52:28.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:28 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:28.450 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-04-25T18:52:28.691 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] boot 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:52:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:28 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: osd.0 [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] boot 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:52:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:28 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:52:29.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:29 smithi183 ceph-mon[32344]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:52:29.713 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:29 smithi086 ceph-mon[27783]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:52:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:29 smithi194 ceph-mon[32740]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:52:30.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/2894987367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8102620 msgr2=0x7ff4f8109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:30.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/2894987367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8102620 0x7ff4f8109ee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000b0a0 tx=0x7ff4f000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:52:30.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/2894987367 shutdown_connections 2024-04-25T18:52:30.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/2894987367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8102620 0x7ff4f8109ee0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/2894987367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f8102f40 0x7ff4f8103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/2894987367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8101c80 0x7ff4f81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/2894987367 >> 172.21.15.183:0/2894987367 conn(0x7ff4f80fc6b0 msgr2=0x7ff4f80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/2894987367 shutdown_connections 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.282+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/2894987367 wait complete. 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 Processor -- start 2024-04-25T18:52:30.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 -- start start 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 0x7ff4f8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8102620 0x7ff4f8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f8102f40 0x7ff4f81a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810d430 con 0x7ff4f8102620 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810d2b0 con 0x7ff4f8101c80 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4fcbbc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810d5b0 con 0x7ff4f8102f40 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4f659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 0x7ff4f8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4f659c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 0x7ff4f8198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:58152/0 (socket says 172.21.15.183:58152) 2024-04-25T18:52:30.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.283+0000 7ff4f659c700 1 -- 172.21.15.183:0/4293421229 learned_addr learned my addr 172.21.15.183:0/4293421229 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f6d9d700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f8102f40 0x7ff4f81a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f5d9b700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8102620 0x7ff4f8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f659c700 1 -- 172.21.15.183:0/4293421229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f8102f40 msgr2=0x7ff4f81a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f659c700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f8102f40 0x7ff4f81a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f659c700 1 -- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8102620 msgr2=0x7ff4f8199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f659c700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8102620 0x7ff4f8199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f659c700 1 -- 172.21.15.183:0/4293421229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f81a0760 con 0x7ff4f8101c80 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f5d9b700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8102620 0x7ff4f8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4f659c700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 0x7ff4f8198b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c990 tx=0x7ff4ec00cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec007b00 con 0x7ff4f8101c80 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a0a50 con 0x7ff4f8101c80 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4ec007ca0 con 0x7ff4f8101c80 2024-04-25T18:52:30.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec018660 con 0x7ff4f8101c80 2024-04-25T18:52:30.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.284+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a0f90 con 0x7ff4f8101c80 2024-04-25T18:52:30.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.285+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f8069730 con 0x7ff4f8101c80 2024-04-25T18:52:30.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.286+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff4ec00f450 con 0x7ff4f8101c80 2024-04-25T18:52:30.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.287+0000 7ff4e77fe700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff4e00a3bb0 0x7ff4e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:30.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.287+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff4ec0ce840 con 0x7ff4f8101c80 2024-04-25T18:52:30.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.287+0000 7ff4f5d9b700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff4e00a3bb0 0x7ff4e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:30.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.288+0000 7ff4f5d9b700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff4e00a3bb0 0x7ff4e00a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005950 tx=0x7ff4e8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:30.292 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.291+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4ec014030 con 0x7ff4f8101c80 2024-04-25T18:52:30.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:30.501+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff4f8101280 con 0x7ff4e00a3bb0 2024-04-25T18:52:30.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:30 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:30.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:30.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:30 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:30.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:30.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:30.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:30.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:30 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:31.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:31 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi086 2024-04-25T18:52:31.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:31 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi086 to 17089M 2024-04-25T18:52:31.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:31 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:31.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:31 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:31.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:31 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:31.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:31 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi086 2024-04-25T18:52:31.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:31 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi086 to 17089M 2024-04-25T18:52:31.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:31 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:31.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:31 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:31.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:31 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:31.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:31 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi086 2024-04-25T18:52:31.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:31 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi086 to 17089M 2024-04-25T18:52:31.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:31 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:31.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:31 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:31.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:31 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:32.362 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:32 smithi183 ceph-mon[32344]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:32.362 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:32 smithi183 ceph-mon[32344]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:32 smithi194 ceph-mon[32740]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:32 smithi194 ceph-mon[32740]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:32 smithi086 ceph-mon[27783]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:32 smithi086 ceph-mon[27783]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:33.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:33 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/911208462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]: dispatch 2024-04-25T18:52:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:33 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]: dispatch 2024-04-25T18:52:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:33 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]': finished 2024-04-25T18:52:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:33 smithi183 ceph-mon[32344]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:52:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:33 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/911208462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]: dispatch 2024-04-25T18:52:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:33 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]: dispatch 2024-04-25T18:52:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:33 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]': finished 2024-04-25T18:52:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:33 smithi194 ceph-mon[32740]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:52:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:33.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:33 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/911208462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]: dispatch 2024-04-25T18:52:33.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:33 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]: dispatch 2024-04-25T18:52:33.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:33 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf82beae-9a79-46f2-9ea8-59032823bc00"}]': finished 2024-04-25T18:52:33.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:33 smithi086 ceph-mon[27783]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:52:33.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:34 smithi183 ceph-mon[32344]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:34 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/1371707093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:34 smithi194 ceph-mon[32740]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:34 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/1371707093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:34 smithi086 ceph-mon[27783]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:34 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/1371707093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:36.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:36 smithi183 ceph-mon[32344]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:36 smithi194 ceph-mon[32740]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:36.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:36 smithi086 ceph-mon[27783]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:38 smithi183 ceph-mon[32344]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:38 smithi194 ceph-mon[32740]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:38 smithi086 ceph-mon[27783]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:40 smithi183 ceph-mon[32344]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:40 smithi194 ceph-mon[32740]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:40.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:40 smithi086 ceph-mon[27783]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:41.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:41 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:52:41.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:41 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:41 smithi183 ceph-mon[32344]: Deploying daemon osd.1 on smithi183 2024-04-25T18:52:41.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:41 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:52:41.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:41 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:41 smithi194 ceph-mon[32740]: Deploying daemon osd.1 on smithi183 2024-04-25T18:52:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:52:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:41 smithi086 ceph-mon[27783]: Deploying daemon osd.1 on smithi183 2024-04-25T18:52:42.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:42 smithi183 ceph-mon[32344]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:42 smithi194 ceph-mon[32740]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:42 smithi086 ceph-mon[27783]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:44.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:44 smithi086 ceph-mon[27783]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:44.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:44 smithi183 ceph-mon[32344]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:44 smithi194 ceph-mon[32740]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:46.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:46 smithi183 ceph-mon[32344]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:46.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:46.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:46.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:46.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:46 smithi086 ceph-mon[27783]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:46.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:46.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:46.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:46 smithi194 ceph-mon[32740]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:48.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:48 smithi183 ceph-mon[32344]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:48 smithi194 ceph-mon[32740]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:48 smithi086 ceph-mon[27783]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:49.286 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-04-25T18:52:49.286 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.284+0000 7ff4e77fe700 1 -- 172.21.15.183:0/4293421229 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4f8101280 con 0x7ff4e00a3bb0 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff4e00a3bb0 msgr2=0x7ff4e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff4e00a3bb0 0x7ff4e00a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005950 tx=0x7ff4e8005f50 comp rx=0 tx=0).stop 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 msgr2=0x7ff4f8198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 0x7ff4f8198b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c990 tx=0x7ff4ec00cca0 comp rx=0 tx=0).stop 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 shutdown_connections 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff4e00a3bb0 0x7ff4e00a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:49.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f8102f40 0x7ff4f81a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:49.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8101c80 0x7ff4f8198b00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:49.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.286+0000 7ff4fcbbc700 1 --2- 172.21.15.183:0/4293421229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f8102620 0x7ff4f8199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:49.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.287+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 >> 172.21.15.183:0/4293421229 conn(0x7ff4f80fc6b0 msgr2=0x7ff4f8100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:49.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.287+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 shutdown_connections 2024-04-25T18:52:49.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:52:49.287+0000 7ff4fcbbc700 1 -- 172.21.15.183:0/4293421229 wait complete. 2024-04-25T18:52:50.031 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T18:52:50.033 INFO:tasks.cephadm:Deploying osd.2 on smithi194 with /dev/vg_nvme/lv_4... 2024-04-25T18:52:50.034 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:52:50.275 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:52:50.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:50.322 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.322 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.322 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.322 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.322 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: from='osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:52:50.322 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:50 smithi183 ceph-mon[32344]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:52:50.397 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:50.398 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.398 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.398 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.398 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.398 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: from='osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:52:50.398 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:50 smithi194 ceph-mon[32740]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:52:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: from='osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:52:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:50 smithi086 ceph-mon[27783]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:52:51.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-mon[32344]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:52:51.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-mon[32344]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:52:51.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:51.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-mon[32344]: from='osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:52:51.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-mon[32344]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:52:51.496 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1[36922]: 2024-04-25T18:52:51.414+0000 7f4d5b186700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T18:52:51.496 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:52:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1[36922]: 2024-04-25T18:52:51.418+0000 7f4d56f79700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:52:51.719 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:51 smithi194 ceph-mon[32740]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:52:51.719 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:51 smithi194 ceph-mon[32740]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:52:51.719 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:51.720 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:51 smithi194 ceph-mon[32740]: from='osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:52:51.720 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:51 smithi194 ceph-mon[32740]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:52:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:51 smithi086 ceph-mon[27783]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:52:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:51 smithi086 ceph-mon[27783]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:52:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:51 smithi086 ceph-mon[27783]: from='osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:52:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:51 smithi086 ceph-mon[27783]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112942 s, 928 MB/s 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 2024-04-25T18:52:52.312 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-25T18:52:52.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:52 smithi194 ceph-mon[32740]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:52.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:52 smithi194 ceph-mon[32740]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:52:52.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:52 smithi194 ceph-mon[32740]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:52:52.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:52.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:52.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:52 smithi183 ceph-mon[32344]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:52.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:52 smithi183 ceph-mon[32344]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:52:52.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:52 smithi183 ceph-mon[32344]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:52:52.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:52.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:52 smithi086 ceph-mon[27783]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:52:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:52 smithi086 ceph-mon[27783]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:52:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:52 smithi086 ceph-mon[27783]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:52:52.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:52.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:52.939 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2024-04-25T18:52:53.177 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] boot 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:53.574 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:53.575 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.575 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:53.575 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:53.575 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:53.575 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] boot 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:53.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:53.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:53.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:53.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:52:53.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: osd.1 [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] boot 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:52:54.625 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:54 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi183 2024-04-25T18:52:54.626 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:54 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-25T18:52:54.626 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:54 smithi194 ceph-mon[32740]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:54.626 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:54 smithi194 ceph-mon[32740]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:52:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:54 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi183 2024-04-25T18:52:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:54 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-25T18:52:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:54 smithi086 ceph-mon[27783]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:54 smithi086 ceph-mon[27783]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:52:54.885 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 -- 172.21.15.194:0/287716834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 msgr2=0x7fdce0108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:54.885 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- 172.21.15.194:0/287716834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce0108260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009990 tx=0x7fdcd0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:52:54.885 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 -- 172.21.15.194:0/287716834 shutdown_connections 2024-04-25T18:52:54.885 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- 172.21.15.194:0/287716834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce01087a0 0x7fdce010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:54.885 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- 172.21.15.194:0/287716834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0103540 0x7fdce0105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- 172.21.15.194:0/287716834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce0108260 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 -- 172.21.15.194:0/287716834 >> 172.21.15.194:0/287716834 conn(0x7fdce00fcf70 msgr2=0x7fdce00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 -- 172.21.15.194:0/287716834 shutdown_connections 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 -- 172.21.15.194:0/287716834 wait complete. 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 Processor -- start 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 -- start start 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0103540 0x7fdce0102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce01014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.882+0000 7fdce76d4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce01087a0 0x7fdce0101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce76d4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdce010e630 con 0x7fdce01087a0 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce76d4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdce010e4b0 con 0x7fdce0103540 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce76d4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdce010e7b0 con 0x7fdce0105e70 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce01014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:54.886 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce01014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49238/0 (socket says 172.21.15.194:49238) 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 -- 172.21.15.194:0/1168863003 learned_addr learned my addr 172.21.15.194:0/1168863003 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce5470700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0103540 0x7fdce0102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 -- 172.21.15.194:0/1168863003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0103540 msgr2=0x7fdce0102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0103540 0x7fdce0102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 -- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce01087a0 msgr2=0x7fdce0101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce01087a0 0x7fdce0101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 -- 172.21.15.194:0/1168863003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce01022e0 con 0x7fdce0105e70 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce4c6f700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce01014f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0027070 con 0x7fdce0105e70 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce01a2eb0 con 0x7fdce0105e70 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd0022470 con 0x7fdce0105e70 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000f6c0 con 0x7fdce0105e70 2024-04-25T18:52:54.887 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.883+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01a3320 con 0x7fdce0105e70 2024-04-25T18:52:54.888 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.884+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce002df80 con 0x7fdce0105e70 2024-04-25T18:52:54.889 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.885+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fdcd002a070 con 0x7fdce0105e70 2024-04-25T18:52:54.889 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.886+0000 7fdcd67fc700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdccc0a3bb0 0x7fdccc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:54.889 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.886+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fdcd00cfa90 con 0x7fdce0105e70 2024-04-25T18:52:54.889 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.886+0000 7fdce5470700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdccc0a3bb0 0x7fdccc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:54.889 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.887+0000 7fdce5470700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdccc0a3bb0 0x7fdccc0a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc006fd0 tx=0x7fdcdc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:54.892 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:54.891+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdcd0013070 con 0x7fdce0105e70 2024-04-25T18:52:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:54 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi183 2024-04-25T18:52:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:54 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-25T18:52:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:54 smithi183 ceph-mon[32344]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:54 smithi183 ceph-mon[32344]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:52:55.095 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:52:55.094+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdce0062e80 con 0x7fdccc0a3bb0 2024-04-25T18:52:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:55 smithi194 ceph-mon[32740]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:55 smithi194 ceph-mon[32740]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:55 smithi086 ceph-mon[27783]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:55 smithi086 ceph-mon[27783]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:55.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:55 smithi183 ceph-mon[32344]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:55 smithi183 ceph-mon[32344]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:58.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:58 smithi194 ceph-mon[32740]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:58.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:58 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/2017710977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]: dispatch 2024-04-25T18:52:58.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:58 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]: dispatch 2024-04-25T18:52:58.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:58 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]': finished 2024-04-25T18:52:58.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:58 smithi194 ceph-mon[32740]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:52:58.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:52:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:58 smithi086 ceph-mon[27783]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:58 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/2017710977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]: dispatch 2024-04-25T18:52:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:58 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]: dispatch 2024-04-25T18:52:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:58 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]': finished 2024-04-25T18:52:58.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:58 smithi086 ceph-mon[27783]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:52:58.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:52:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:58 smithi183 ceph-mon[32344]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:52:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:58 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/2017710977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]: dispatch 2024-04-25T18:52:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:58 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]: dispatch 2024-04-25T18:52:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:58 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "768f0112-52c8-452e-b56c-1d19c065d486"}]': finished 2024-04-25T18:52:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:58 smithi183 ceph-mon[32344]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:52:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:52:59.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:52:59 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/2515739289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:59.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:52:59 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/2515739289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:52:59 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/2515739289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:00.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:00 smithi086 ceph-mon[27783]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:00 smithi183 ceph-mon[32344]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:00 smithi194 ceph-mon[32740]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:02 smithi086 ceph-mon[27783]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:02.638 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:02 smithi194 ceph-mon[32740]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:02 smithi183 ceph-mon[32344]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:04 smithi086 ceph-mon[27783]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:04.596 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:04 smithi194 ceph-mon[32740]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:04 smithi183 ceph-mon[32344]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:05.563 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:53:05.563 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:05.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:53:05.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:05.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:53:05.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:06.566 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:06 smithi194 ceph-mon[32740]: Deploying daemon osd.2 on smithi194 2024-04-25T18:53:06.566 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:06 smithi194 ceph-mon[32740]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:06.567 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:06 smithi183 ceph-mon[32344]: Deploying daemon osd.2 on smithi194 2024-04-25T18:53:06.567 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:06 smithi183 ceph-mon[32344]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:06 smithi086 ceph-mon[27783]: Deploying daemon osd.2 on smithi194 2024-04-25T18:53:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:06 smithi086 ceph-mon[27783]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:08.557 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:08 smithi194 ceph-mon[32740]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:08 smithi086 ceph-mon[27783]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:08.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:08 smithi183 ceph-mon[32344]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:10 smithi194 ceph-mon[32740]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:10.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:10 smithi183 ceph-mon[32344]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:10 smithi086 ceph-mon[27783]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:10.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:12.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:12 smithi183 ceph-mon[32344]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:12.677 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:12 smithi194 ceph-mon[32740]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:12 smithi086 ceph-mon[27783]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:14.080 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.079+0000 7fdcd67fc700 1 -- 172.21.15.194:0/1168863003 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdce0062e80 con 0x7fdccc0a3bb0 2024-04-25T18:53:14.081 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 2 on host 'smithi194' 2024-04-25T18:53:14.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdccc0a3bb0 msgr2=0x7fdccc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:14.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdccc0a3bb0 0x7fdccc0a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc006fd0 tx=0x7fdcdc008040 comp rx=0 tx=0).stop 2024-04-25T18:53:14.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 msgr2=0x7fdce01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:14.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce01014f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd00047b0 comp rx=0 tx=0).stop 2024-04-25T18:53:14.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 shutdown_connections 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdccc0a3bb0 0x7fdccc0a6070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdce01087a0 0x7fdce0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdce0103540 0x7fdce0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 --2- 172.21.15.194:0/1168863003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce0105e70 0x7fdce01014f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 >> 172.21.15.194:0/1168863003 conn(0x7fdce00fcf70 msgr2=0x7fdce0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 shutdown_connections 2024-04-25T18:53:14.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:53:14.082+0000 7fdce76d4700 1 -- 172.21.15.194:0/1168863003 wait complete. 2024-04-25T18:53:14.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:14.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:14.756 DEBUG:teuthology.orchestra.run.smithi194:osd.2> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T18:53:14.758 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-04-25T18:53:14.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:53:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:53:15.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:15 smithi086 ceph-mon[27783]: from='osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:53:15.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:15 smithi086 ceph-mon[27783]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:53:15.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:15 smithi183 ceph-mon[32344]: from='osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:53:15.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:15 smithi183 ceph-mon[32344]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:53:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:15 smithi194 ceph-mon[32740]: from='osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:53:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:15 smithi194 ceph-mon[32740]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:53:16.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-mon[32740]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:53:16.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-mon[32740]: from='osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:53:16.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-mon[32740]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:53:16.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:16.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-mon[32740]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:53:16.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-mon[32740]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:16.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:16 smithi086 ceph-mon[27783]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:53:16.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:16 smithi086 ceph-mon[27783]: from='osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:53:16.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:16 smithi086 ceph-mon[27783]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:53:16.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:16 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:16.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:16 smithi086 ceph-mon[27783]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:53:16.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:16 smithi086 ceph-mon[27783]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:16.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:16 smithi183 ceph-mon[32344]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:53:16.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:16 smithi183 ceph-mon[32344]: from='osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:53:16.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:16 smithi183 ceph-mon[32344]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:53:16.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:16 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:16.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:16 smithi183 ceph-mon[32344]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:53:16.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:16 smithi183 ceph-mon[32344]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:16.929 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2[37881]: 2024-04-25T18:53:16.633+0000 7fc6acac5700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T18:53:16.929 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2[37881]: 2024-04-25T18:53:16.637+0000 7fc6a40af700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:53:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:17 smithi194 ceph-mon[32740]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:53:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:17 smithi194 ceph-mon[32740]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:53:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:17 smithi194 ceph-mon[32740]: from='osd.2 ' entity='osd.2' 2024-04-25T18:53:17.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:17 smithi086 ceph-mon[27783]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:53:17.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:17 smithi086 ceph-mon[27783]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:53:17.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:17.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:17.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:17 smithi086 ceph-mon[27783]: from='osd.2 ' entity='osd.2' 2024-04-25T18:53:17.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:17 smithi183 ceph-mon[32344]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:53:17.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:17 smithi183 ceph-mon[32344]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:53:17.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:17.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:17.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:17 smithi183 ceph-mon[32344]: from='osd.2 ' entity='osd.2' 2024-04-25T18:53:17.803 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:53:17.803 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:53:17.803 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-25T18:53:17.803 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-25T18:53:17.803 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0370291 s, 283 MB/s 2024-04-25T18:53:17.803 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] boot 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:18.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:18.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:18.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] boot 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:18.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:53:18.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: osd.2 [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] boot 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:18.540 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:18.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-04-25T18:53:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:53:19.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:19 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi194 2024-04-25T18:53:19.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:19 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi194 to 17088M 2024-04-25T18:53:19.431 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:19 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi194 2024-04-25T18:53:19.431 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:19 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi194 to 17088M 2024-04-25T18:53:19.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:19 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi194 2024-04-25T18:53:19.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:19 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi194 to 17088M 2024-04-25T18:53:20.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:20 smithi086 ceph-mon[27783]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:53:20.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:20 smithi086 ceph-mon[27783]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:20.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:20 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:53:20.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:20 smithi194 ceph-mon[32740]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:53:20.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:20 smithi194 ceph-mon[32740]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:20.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:20 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:53:20.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:20 smithi183 ceph-mon[32344]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:53:20.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:20 smithi183 ceph-mon[32344]: pgmap v90: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:20.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:20 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:53:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 -- 172.21.15.86:0/177755924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc105ee0 msgr2=0x7f0adc106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 --2- 172.21.15.86:0/177755924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc105ee0 0x7f0adc106340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80098f0 tx=0x7f0ac8009c00 comp rx=0 tx=0).stop 2024-04-25T18:53:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 -- 172.21.15.86:0/177755924 shutdown_connections 2024-04-25T18:53:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 --2- 172.21.15.86:0/177755924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc106880 0x7f0adc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 --2- 172.21.15.86:0/177755924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc104ce0 0x7f0adc105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 --2- 172.21.15.86:0/177755924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc105ee0 0x7f0adc106340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 -- 172.21.15.86:0/177755924 >> 172.21.15.86:0/177755924 conn(0x7f0adc1002a0 msgr2=0x7f0adc1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.058+0000 7f0ae4400700 1 -- 172.21.15.86:0/177755924 shutdown_connections 2024-04-25T18:53:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.059+0000 7f0ae4400700 1 -- 172.21.15.86:0/177755924 wait complete. 2024-04-25T18:53:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 Processor -- start 2024-04-25T18:53:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 -- start start 2024-04-25T18:53:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc104ce0 0x7f0adc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc105ee0 0x7f0adc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 0x7f0adc1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc110780 con 0x7f0adc105ee0 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc110600 con 0x7f0adc104ce0 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae4400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc110900 con 0x7f0adc106880 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc105ee0 0x7f0adc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.060+0000 7f0ae2400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 0x7f0adc1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 0x7f0adc1a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35188/0 (socket says 172.21.15.86:35188) 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 -- 172.21.15.86:0/65198199 learned_addr learned my addr 172.21.15.86:0/65198199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae1a00700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc104ce0 0x7f0adc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 -- 172.21.15.86:0/65198199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc104ce0 msgr2=0x7f0adc19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc104ce0 0x7f0adc19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 -- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc105ee0 msgr2=0x7f0adc19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc105ee0 0x7f0adc19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 -- 172.21.15.86:0/65198199 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc1a2a80 con 0x7f0adc106880 2024-04-25T18:53:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae1a00700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc104ce0 0x7f0adc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:53:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae2400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 0x7f0adc1a23e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400ef60 tx=0x7f0ad400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40092e0 con 0x7f0adc106880 2024-04-25T18:53:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad4004bb0 con 0x7f0adc106880 2024-04-25T18:53:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.061+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc1a2d10 con 0x7f0adc106880 2024-04-25T18:53:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.062+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4004280 con 0x7f0adc106880 2024-04-25T18:53:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.062+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc1a3240 con 0x7f0adc106880 2024-04-25T18:53:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.063+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc069730 con 0x7f0adc106880 2024-04-25T18:53:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.064+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0ad401b030 con 0x7f0adc106880 2024-04-25T18:53:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.066+0000 7f0ada000700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0ac40a3c30 0x7f0ac40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.066+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f0ad40ce990 con 0x7f0adc106880 2024-04-25T18:53:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.066+0000 7f0ae1a00700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0ac40a3c30 0x7f0ac40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.067+0000 7f0ae1a00700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0ac40a3c30 0x7f0ac40a60f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009750 tx=0x7f0ad0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.070+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ad4022020 con 0x7f0adc106880 2024-04-25T18:53:21.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:21.333+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0adc062e80 con 0x7f0ac40a3c30 2024-04-25T18:53:21.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:53:21.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 ceph-mon[32740]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:53:21.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:53:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:53:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 ceph-mon[32344]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:53:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:53:21.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:53:21.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 ceph-mon[27783]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:53:21.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:53:21.571 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:53:21.571 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.571 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.572 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44815]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.572 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44840]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:53:21.572 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44840]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.572 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44840]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.572 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44840]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41049]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41049]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41049]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41049]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41052]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41052]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.777 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41052]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41783]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:53:21.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41783]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41783]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41783]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:53:21.982 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.982 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41787]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.982 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41787]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41790]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41790]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41793]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:21 smithi194 sudo[41793]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44877]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44877]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44880]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44880]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44880]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:21 smithi086 sudo[44880]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:22.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41055]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:53:22.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41055]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:22.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41055]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:22.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41055]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:22.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41058]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:53:22.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41058]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:53:22.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41058]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:53:22.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:21 smithi183 sudo[41058]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:22 smithi194 ceph-mon[32740]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:22.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:22 smithi086 ceph-mon[27783]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:53:22.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:22 smithi183 ceph-mon[32344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:53:23.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:23 smithi194 ceph-mon[32740]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:53:23.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:23 smithi086 ceph-mon[27783]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:53:23.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:23 smithi183 ceph-mon[32344]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:53:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:24.238+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ad4012050 con 0x7f0adc106880 2024-04-25T18:53:24.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:24 smithi086 ceph-mon[27783]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:24.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:24 smithi183 ceph-mon[32344]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:24 smithi194 ceph-mon[32740]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:25.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:25 smithi086 ceph-mon[27783]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:53:25.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:25 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1219518766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91c123c-a363-482a-9d1a-c6eb6f121b33"}]: dispatch 2024-04-25T18:53:25.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:25 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1219518766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a91c123c-a363-482a-9d1a-c6eb6f121b33"}]': finished 2024-04-25T18:53:25.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:25 smithi086 ceph-mon[27783]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:53:25.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:25 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:25.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:25 smithi183 ceph-mon[32344]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:53:25.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:25 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1219518766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91c123c-a363-482a-9d1a-c6eb6f121b33"}]: dispatch 2024-04-25T18:53:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:25 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1219518766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a91c123c-a363-482a-9d1a-c6eb6f121b33"}]': finished 2024-04-25T18:53:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:25 smithi183 ceph-mon[32344]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:53:25.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:25 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:25 smithi194 ceph-mon[32740]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:53:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:25 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1219518766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a91c123c-a363-482a-9d1a-c6eb6f121b33"}]: dispatch 2024-04-25T18:53:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:25 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1219518766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a91c123c-a363-482a-9d1a-c6eb6f121b33"}]': finished 2024-04-25T18:53:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:25 smithi194 ceph-mon[32740]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:53:25.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:25 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:26 smithi086 ceph-mon[27783]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:26 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3719717679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:26.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:26 smithi183 ceph-mon[32344]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:26.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:26 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3719717679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:26 smithi194 ceph-mon[32740]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:26 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3719717679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:28.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:28 smithi086 ceph-mon[27783]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:28.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:28 smithi183 ceph-mon[32344]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:28 smithi194 ceph-mon[32740]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:30 smithi086 ceph-mon[27783]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:30.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:30 smithi183 ceph-mon[32344]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:30 smithi194 ceph-mon[32740]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:32.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:32 smithi086 ceph-mon[27783]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:32 smithi183 ceph-mon[32344]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:32 smithi194 ceph-mon[32740]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:34 smithi086 ceph-mon[27783]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:53:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:34 smithi183 ceph-mon[32344]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:34 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:53:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:34 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:34 smithi194 ceph-mon[32740]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:53:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:35.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:35 smithi086 ceph-mon[27783]: Deploying daemon osd.3 on smithi086 2024-04-25T18:53:35.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:35 smithi183 ceph-mon[32344]: Deploying daemon osd.3 on smithi086 2024-04-25T18:53:35.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:35 smithi194 ceph-mon[32740]: Deploying daemon osd.3 on smithi086 2024-04-25T18:53:36.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:36 smithi086 ceph-mon[27783]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:36.567 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:36 smithi183 ceph-mon[32344]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:36 smithi194 ceph-mon[32740]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:38.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:38 smithi086 ceph-mon[27783]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:38 smithi183 ceph-mon[32344]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:38 smithi194 ceph-mon[32740]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:40.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:40 smithi086 ceph-mon[27783]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:40 smithi183 ceph-mon[32344]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:40 smithi194 ceph-mon[32740]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:42 smithi086 ceph-mon[27783]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:42 smithi183 ceph-mon[32344]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:42 smithi194 ceph-mon[32740]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:44.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:44 smithi086 ceph-mon[27783]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:44.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:44 smithi183 ceph-mon[32344]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:44 smithi194 ceph-mon[32740]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:46.300 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.300 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:46.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:46.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:46.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:46 smithi086 ceph-mon[27783]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:53:46.379 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-04-25T18:53:46.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.376+0000 7f0ada000700 1 -- 172.21.15.86:0/65198199 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0adc062e80 con 0x7f0ac40a3c30 2024-04-25T18:53:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.380+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0ac40a3c30 msgr2=0x7f0ac40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.380+0000 7f0ae4400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0ac40a3c30 0x7f0ac40a60f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009750 tx=0x7f0ad0009310 comp rx=0 tx=0).stop 2024-04-25T18:53:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.380+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 msgr2=0x7f0adc1a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.380+0000 7f0ae4400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 0x7f0adc1a23e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400ef60 tx=0x7f0ad400ef90 comp rx=0 tx=0).stop 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 shutdown_connections 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0ac40a3c30 0x7f0ac40a60f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0adc106880 0x7f0adc1a23e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0adc104ce0 0x7f0adc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 --2- 172.21.15.86:0/65198199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0adc105ee0 0x7f0adc19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 >> 172.21.15.86:0/65198199 conn(0x7f0adc1002a0 msgr2=0x7f0adc109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.381+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 shutdown_connections 2024-04-25T18:53:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:53:46.382+0000 7f0ae4400700 1 -- 172.21.15.86:0/65198199 wait complete. 2024-04-25T18:53:46.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:46 smithi183 ceph-mon[32344]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:53:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:46 smithi194 ceph-mon[32740]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:53:47.197 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T18:53:47.203 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-04-25T18:53:47.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:53:47.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:53:47.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:53:47.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:53:47.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:47.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:47.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:47.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:47.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:53:47.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:53:47.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:53:47.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:47.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:47.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:47.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:47 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:47.451 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:53:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:53:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:53:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:53:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:47.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:47.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:47.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:47 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:48.071 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3[47072]: 2024-04-25T18:53:47.703+0000 7fbc71a00700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T18:53:48.072 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 18:53:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3[47072]: 2024-04-25T18:53:47.720+0000 7fbc67800700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:53:48.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] boot 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T18:53:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] boot 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T18:53:48.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:53:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:53:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:48.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:53:48.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: from='osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843]' entity='osd.3' 2024-04-25T18:53:48.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: osd.3 [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] boot 2024-04-25T18:53:48.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T18:53:48.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:53:49.472 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:53:49.472 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:53:49.472 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T18:53:49.472 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T18:53:49.472 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122884 s, 853 MB/s 2024-04-25T18:53:49.472 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T18:53:50.163 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-04-25T18:53:50.283 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:50 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:53:50.283 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:50 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:53:50.284 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:50 smithi194 ceph-mon[32740]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:53:50.284 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:50 smithi194 ceph-mon[32740]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:53:50.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:50 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:53:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:50 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:53:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:50 smithi086 ceph-mon[27783]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:53:50.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:50 smithi086 ceph-mon[27783]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:53:50.404 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:53:50.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:50 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:53:50.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:50 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:53:50.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:50 smithi183 ceph-mon[32344]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:53:50.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:50 smithi183 ceph-mon[32344]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:53:51.031 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:51 smithi183 ceph-mon[32344]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:53:51.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:51 smithi086 ceph-mon[27783]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:53:51.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:51 smithi194 ceph-mon[32740]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:53:52.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:53:52.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:53:52.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:52.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:52.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:52.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:52.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:52 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.093 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/3138266387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8102aa0 msgr2=0x7fd9f8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:52.093 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/3138266387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8102aa0 0x7fd9f8102ec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0099c0 tx=0x7fd9ec009cd0 comp rx=0 tx=0).stop 2024-04-25T18:53:52.093 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/3138266387 shutdown_connections 2024-04-25T18:53:52.093 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/3138266387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9f8104640 0x7fd9f810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:52.093 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/3138266387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8102aa0 0x7fd9f8102ec0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/3138266387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/3138266387 >> 172.21.15.183:0/3138266387 conn(0x7fd9f80fe040 msgr2=0x7fd9f8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/3138266387 shutdown_connections 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/3138266387 wait complete. 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 Processor -- start 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 -- start start 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f8102aa0 0x7fd9f80767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.090+0000 7fd9ff1a9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9ff1a9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9f8104640 0x7fd9f8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9ff1a9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810e610 con 0x7fd9f8102aa0 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9ff1a9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810e490 con 0x7fd9f8103ca0 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9ff1a9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810e790 con 0x7fd9f8104640 2024-04-25T18:53:52.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:56008/0 (socket says 172.21.15.183:56008) 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 -- 172.21.15.183:0/367905390 learned_addr learned my addr 172.21.15.183:0/367905390 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9fd746700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9f8104640 0x7fd9f8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 -- 172.21.15.183:0/367905390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9f8104640 msgr2=0x7fd9f8075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9f8104640 0x7fd9f8075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 -- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f8102aa0 msgr2=0x7fd9f80767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f8102aa0 0x7fd9f80767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 -- 172.21.15.183:0/367905390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f8075c20 con 0x7fd9f8103ca0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f7fff700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8074e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400b530 tx=0x7fd9e400b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4015020 con 0x7fd9f8103ca0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81a2e70 con 0x7fd9f8103ca0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e4011ce0 con 0x7fd9f8103ca0 2024-04-25T18:53:52.095 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.091+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4010d20 con 0x7fd9f8103ca0 2024-04-25T18:53:52.096 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.092+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81a3350 con 0x7fd9f8103ca0 2024-04-25T18:53:52.096 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.093+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f8069730 con 0x7fd9f8103ca0 2024-04-25T18:53:52.096 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.094+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9e4018020 con 0x7fd9f8103ca0 2024-04-25T18:53:52.096 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.094+0000 7fd9f5ffb700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd9e80a3c40 0x7fd9e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:52.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.095+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7fd9e40ce9e0 con 0x7fd9f8103ca0 2024-04-25T18:53:52.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.095+0000 7fd9fcf45700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd9e80a3c40 0x7fd9e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:52.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.096+0000 7fd9fcf45700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd9e80a3c40 0x7fd9e80a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec003790 tx=0x7fd9ec009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:52.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.099+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9e401f020 con 0x7fd9f8103ca0 2024-04-25T18:53:52.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:53:52.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:52 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:53:52.310+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd9f8062e80 con 0x7fd9e80a3c40 2024-04-25T18:53:52.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:52.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:52.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:52.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:52 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi086 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:53.282 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:53 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:53.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi086 2024-04-25T18:53:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-25T18:53:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:53:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:53.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:53 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:53.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi086 2024-04-25T18:53:53.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-25T18:53:53.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:53:53.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:53.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:53 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:54.187 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:54 smithi183 ceph-mon[32344]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T18:53:54.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:54 smithi086 ceph-mon[27783]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T18:53:54.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:54 smithi194 ceph-mon[32740]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T18:53:55.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:55 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/2246084523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]: dispatch 2024-04-25T18:53:55.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:55 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]: dispatch 2024-04-25T18:53:55.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:55 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]': finished 2024-04-25T18:53:55.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:55 smithi194 ceph-mon[32740]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T18:53:55.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:53:55.508 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:55 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/2246084523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]: dispatch 2024-04-25T18:53:55.508 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:55 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]: dispatch 2024-04-25T18:53:55.508 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:55 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]': finished 2024-04-25T18:53:55.508 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:55 smithi183 ceph-mon[32344]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T18:53:55.508 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:53:55.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:55 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/2246084523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]: dispatch 2024-04-25T18:53:55.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:55 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]: dispatch 2024-04-25T18:53:55.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:55 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "deb085ac-6e63-4429-a71a-e9e35e2c27f4"}]': finished 2024-04-25T18:53:55.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:55 smithi086 ceph-mon[27783]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T18:53:55.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:53:56.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:56 smithi194 ceph-mon[32740]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T18:53:56.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:56 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/1490518072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:56 smithi086 ceph-mon[27783]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T18:53:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:56 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/1490518072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:56 smithi183 ceph-mon[32344]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T18:53:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:56 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/1490518072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:53:58 smithi086 ceph-mon[27783]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:53:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:53:58 smithi183 ceph-mon[32344]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:53:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:53:58 smithi194 ceph-mon[32740]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:54:00.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:00 smithi183 ceph-mon[32344]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:54:00.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:00 smithi086 ceph-mon[27783]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:54:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:00 smithi194 ceph-mon[32740]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:54:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:02 smithi086 ceph-mon[27783]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T18:54:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:02 smithi183 ceph-mon[32344]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T18:54:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:02 smithi194 ceph-mon[32740]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T18:54:03.563 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:54:03.563 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:54:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:03.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:54:03.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:04 smithi086 ceph-mon[27783]: Deploying daemon osd.4 on smithi183 2024-04-25T18:54:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:04 smithi086 ceph-mon[27783]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:04.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:04 smithi183 ceph-mon[32344]: Deploying daemon osd.4 on smithi183 2024-04-25T18:54:04.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:04 smithi183 ceph-mon[32344]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:04 smithi194 ceph-mon[32740]: Deploying daemon osd.4 on smithi183 2024-04-25T18:54:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:04 smithi194 ceph-mon[32740]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:06.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:06 smithi183 ceph-mon[32344]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:06 smithi086 ceph-mon[27783]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:06 smithi194 ceph-mon[32740]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:08.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:08 smithi183 ceph-mon[32344]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:08 smithi194 ceph-mon[32740]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:08 smithi086 ceph-mon[27783]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:09 smithi183 ceph-mon[32344]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:09 smithi194 ceph-mon[32740]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:10.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:10.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:10.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:10.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:09 smithi086 ceph-mon[27783]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:12.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:12 smithi183 ceph-mon[32344]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:12.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:12.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:12.419 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.556+0000 7fd9f5ffb700 1 -- 172.21.15.183:0/367905390 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd9f8062e80 con 0x7fd9e80a3c40 2024-04-25T18:54:12.557 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-04-25T18:54:12.559 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd9e80a3c40 msgr2=0x7fd9e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.559 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd9e80a3c40 0x7fd9e80a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec003790 tx=0x7fd9ec009f10 comp rx=0 tx=0).stop 2024-04-25T18:54:12.559 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 msgr2=0x7fd9f8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.559 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8074e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400b530 tx=0x7fd9e400b8f0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 shutdown_connections 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd9e80a3c40 0x7fd9e80a6100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9f8104640 0x7fd9f8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f8102aa0 0x7fd9f80767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 --2- 172.21.15.183:0/367905390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9f8103ca0 0x7fd9f8074e30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 >> 172.21.15.183:0/367905390 conn(0x7fd9f80fe040 msgr2=0x7fd9f81070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 shutdown_connections 2024-04-25T18:54:12.560 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:54:12.558+0000 7fd9ff1a9700 1 -- 172.21.15.183:0/367905390 wait complete. 2024-04-25T18:54:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:12 smithi086 ceph-mon[27783]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:12 smithi194 ceph-mon[32740]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.249 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T18:54:13.252 INFO:tasks.cephadm:Deploying osd.5 on smithi194 with /dev/vg_nvme/lv_3... 2024-04-25T18:54:13.252 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:54:13.497 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:54:13.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:13 smithi183 ceph-mon[32344]: from='osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:54:13.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:13 smithi183 ceph-mon[32344]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:54:13.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:13 smithi086 ceph-mon[27783]: from='osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:54:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:13 smithi086 ceph-mon[27783]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:54:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:13.619 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:13 smithi194 ceph-mon[32740]: from='osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:54:13.619 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:13 smithi194 ceph-mon[32740]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:54:13.619 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.619 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:13.619 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:14.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:14 smithi086 ceph-mon[27783]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:14.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:14 smithi086 ceph-mon[27783]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:54:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:14 smithi086 ceph-mon[27783]: from='osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:54:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:14 smithi086 ceph-mon[27783]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:54:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:14 smithi086 ceph-mon[27783]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:54:14.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-mon[32344]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:14.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-mon[32344]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:54:14.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-mon[32344]: from='osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:54:14.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-mon[32344]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:54:14.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:14.584 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-mon[32344]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:54:14.584 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T18:54:14.328+0000 7ff65c541700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T18:54:14.584 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 18:54:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T18:54:14.333+0000 7ff658334700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:54:14.601 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:14 smithi194 ceph-mon[32740]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:14.601 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:14 smithi194 ceph-mon[32740]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:54:14.601 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:14 smithi194 ceph-mon[32740]: from='osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:54:14.601 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:14 smithi194 ceph-mon[32740]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:54:14.601 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:14.601 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:14 smithi194 ceph-mon[32740]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:54:15.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:15 smithi194 ceph-mon[32740]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:54:15.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:15 smithi194 ceph-mon[32740]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:54:15.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:15.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:15.521 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:54:15.521 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:54:15.521 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-25T18:54:15.521 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-25T18:54:15.522 INFO:teuthology.orchestra.run.smithi194.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115826 s, 905 MB/s 2024-04-25T18:54:15.522 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-25T18:54:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:15 smithi183 ceph-mon[32344]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:54:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:15 smithi183 ceph-mon[32344]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:54:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:15.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:15 smithi086 ceph-mon[27783]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:54:15.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:15 smithi086 ceph-mon[27783]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:54:15.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:15.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:16.169 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2024-04-25T18:54:16.407 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:54:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:16 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:54:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:16 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:54:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:16 smithi194 ceph-mon[32740]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:16 smithi194 ceph-mon[32740]: osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] boot 2024-04-25T18:54:16.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:16 smithi194 ceph-mon[32740]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T18:54:16.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:16 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:16.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:16 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:54:16.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:16 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:54:16.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:16 smithi183 ceph-mon[32344]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:16.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:16 smithi183 ceph-mon[32344]: osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] boot 2024-04-25T18:54:16.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:16 smithi183 ceph-mon[32344]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T18:54:16.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:16 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:16 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:54:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:16 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:54:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:16 smithi086 ceph-mon[27783]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:54:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:16 smithi086 ceph-mon[27783]: osd.4 [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] boot 2024-04-25T18:54:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:16 smithi086 ceph-mon[27783]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T18:54:16.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:16 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi183 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-25T18:54:17.554 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.555 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:17.555 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:17.555 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:54:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi183 2024-04-25T18:54:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:17.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-25T18:54:17.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:17.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:17.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi183 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:17.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.100+0000 7f3c7984c700 1 -- 172.21.15.194:0/911631412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74103cd0 msgr2=0x7f3c74104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.100+0000 7f3c7984c700 1 --2- 172.21.15.194:0/911631412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74103cd0 0x7f3c74104130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009990 tx=0x7f3c68009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.100+0000 7f3c7984c700 1 -- 172.21.15.194:0/911631412 shutdown_connections 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.100+0000 7f3c7984c700 1 --2- 172.21.15.194:0/911631412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74104670 0x7f3c7410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.100+0000 7f3c7984c700 1 --2- 172.21.15.194:0/911631412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74102ad0 0x7f3c74102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.100+0000 7f3c7984c700 1 --2- 172.21.15.194:0/911631412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74103cd0 0x7f3c74104130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- 172.21.15.194:0/911631412 >> 172.21.15.194:0/911631412 conn(0x7f3c740fe090 msgr2=0x7f3c741004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- 172.21.15.194:0/911631412 shutdown_connections 2024-04-25T18:54:18.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- 172.21.15.194:0/911631412 wait complete. 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 Processor -- start 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- start start 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 0x7f3c740767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74103cd0 0x7f3c74074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74104670 0x7f3c74075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410e5e0 con 0x7f3c74103cd0 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410e460 con 0x7f3c74104670 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.101+0000 7f3c7984c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410e760 con 0x7f3c74102ad0 2024-04-25T18:54:18.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 0x7f3c740767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 0x7f3c740767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:35042/0 (socket says 172.21.15.194:35042) 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 -- 172.21.15.194:0/448606272 learned_addr learned my addr 172.21.15.194:0/448606272 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c737fe700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74104670 0x7f3c74075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 -- 172.21.15.194:0/448606272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74104670 msgr2=0x7f3c74075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74104670 0x7f3c74075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 -- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74103cd0 msgr2=0x7f3c74074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74103cd0 0x7f3c74074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 -- 172.21.15.194:0/448606272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c74075c20 con 0x7f3c74102ad0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c737fe700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74104670 0x7f3c74075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c72ffd700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 0x7f3c740767e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400d980 tx=0x7f3c6400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64017070 con 0x7f3c74102ad0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c64010460 con 0x7f3c74102ad0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400de60 con 0x7f3c74102ad0 2024-04-25T18:54:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c741a2dd0 con 0x7f3c74102ad0 2024-04-25T18:54:18.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.102+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c741a3170 con 0x7f3c74102ad0 2024-04-25T18:54:18.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.103+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c74069730 con 0x7f3c74102ad0 2024-04-25T18:54:18.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.104+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c64009b40 con 0x7f3c74102ad0 2024-04-25T18:54:18.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.105+0000 7f3c7884a700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3c5c0a3c40 0x7f3c5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:18.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.105+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f3c640cef40 con 0x7f3c74102ad0 2024-04-25T18:54:18.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.105+0000 7f3c727fc700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3c5c0a3c40 0x7f3c5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:18.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.106+0000 7f3c727fc700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3c5c0a3c40 0x7f3c5c0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800b5c0 tx=0x7f3c68003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:18.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.110+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c6409c990 con 0x7f3c74102ad0 2024-04-25T18:54:18.316 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:18.315+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3c74062e80 con 0x7f3c5c0a3c40 2024-04-25T18:54:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:18 smithi194 ceph-mon[32740]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:54:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:54:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:18 smithi086 ceph-mon[27783]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:54:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:54:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:18 smithi183 ceph-mon[32344]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:54:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:54:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:19 smithi194 ceph-mon[32740]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:19.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:19 smithi086 ceph-mon[27783]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:19.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:19 smithi183 ceph-mon[32344]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:20.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:20 smithi194 ceph-mon[32740]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:20 smithi086 ceph-mon[27783]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:20 smithi183 ceph-mon[32344]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:21 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/287053961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]: dispatch 2024-04-25T18:54:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:21 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]: dispatch 2024-04-25T18:54:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:21 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]': finished 2024-04-25T18:54:21.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:21 smithi194 ceph-mon[32740]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T18:54:21.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:21 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:21 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/287053961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]: dispatch 2024-04-25T18:54:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:21 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]: dispatch 2024-04-25T18:54:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:21 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]': finished 2024-04-25T18:54:21.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:21 smithi086 ceph-mon[27783]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T18:54:21.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:21 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:21.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:21 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/287053961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]: dispatch 2024-04-25T18:54:21.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:21 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]: dispatch 2024-04-25T18:54:21.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:21 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74a891b6-ac08-4b3a-bd2a-5b8d735a77de"}]': finished 2024-04-25T18:54:21.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:21 smithi183 ceph-mon[32344]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T18:54:21.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:21 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:22 smithi194 ceph-mon[32740]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:22 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/3550564443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:54:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:22 smithi086 ceph-mon[27783]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:22 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/3550564443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:54:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:22 smithi183 ceph-mon[32344]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:22 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/3550564443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:54:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:24 smithi194 ceph-mon[32740]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:24 smithi086 ceph-mon[27783]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:24 smithi183 ceph-mon[32344]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:26 smithi194 ceph-mon[32740]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:26 smithi086 ceph-mon[27783]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:26.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:26 smithi183 ceph-mon[32344]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:27.711 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:27 smithi194 ceph-mon[32740]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:27.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:27 smithi086 ceph-mon[27783]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:27.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:27 smithi183 ceph-mon[32344]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:28 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:54:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:28 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:28 smithi194 ceph-mon[32740]: Deploying daemon osd.5 on smithi194 2024-04-25T18:54:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:28 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:54:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:28 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:28 smithi086 ceph-mon[27783]: Deploying daemon osd.5 on smithi194 2024-04-25T18:54:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:28 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:54:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:28 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:28 smithi183 ceph-mon[32344]: Deploying daemon osd.5 on smithi194 2024-04-25T18:54:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:29 smithi194 ceph-mon[32740]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:29.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:29 smithi086 ceph-mon[27783]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:29.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:29 smithi183 ceph-mon[32344]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:32.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:32 smithi086 ceph-mon[27783]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:32 smithi183 ceph-mon[32344]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:32 smithi194 ceph-mon[32740]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:34.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:34 smithi194 ceph-mon[32740]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:34.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:34.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:34.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:34 smithi086 ceph-mon[27783]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:34.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:34 smithi183 ceph-mon[32344]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:34 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:34 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:34 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:36.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:36 smithi086 ceph-mon[27783]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:36.616 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:36 smithi194 ceph-mon[32740]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:36.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:36 smithi183 ceph-mon[32344]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:37.951 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.949+0000 7f3c7884a700 1 -- 172.21.15.194:0/448606272 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c74062e80 con 0x7f3c5c0a3c40 2024-04-25T18:54:37.951 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 5 on host 'smithi194' 2024-04-25T18:54:37.953 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3c5c0a3c40 msgr2=0x7f3c5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:37.953 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3c5c0a3c40 0x7f3c5c0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800b5c0 tx=0x7f3c68003860 comp rx=0 tx=0).stop 2024-04-25T18:54:37.953 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 msgr2=0x7f3c740767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:37.953 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 0x7f3c740767e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400d980 tx=0x7f3c6400dd40 comp rx=0 tx=0).stop 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 shutdown_connections 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3c5c0a3c40 0x7f3c5c0a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c74104670 0x7f3c74075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c74102ad0 0x7f3c740767e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 --2- 172.21.15.194:0/448606272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74103cd0 0x7f3c74074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 >> 172.21.15.194:0/448606272 conn(0x7f3c740fe090 msgr2=0x7f3c74107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 shutdown_connections 2024-04-25T18:54:37.954 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:54:37.952+0000 7f3c7984c700 1 -- 172.21.15.194:0/448606272 wait complete. 2024-04-25T18:54:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: from='osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:54:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:54:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.635 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:38.635 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.635 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.635 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: from='osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:54:38.635 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:54:38.636 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.636 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.663 DEBUG:teuthology.orchestra.run.smithi194:osd.5> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T18:54:38.666 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-04-25T18:54:38.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: from='osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:39.109 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:40.227 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-mon[32740]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:54:40.227 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-mon[32740]: from='osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:54:40.227 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-mon[32740]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:54:40.227 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:40.227 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-mon[32740]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:54:40.227 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-mon[32740]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:40.228 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T18:54:39.958+0000 7f23fb8dd700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T18:54:40.228 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 18:54:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T18:54:39.965+0000 7f23f76d0700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:54:40.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:39 smithi086 ceph-mon[27783]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:54:40.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:39 smithi086 ceph-mon[27783]: from='osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:54:40.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:39 smithi086 ceph-mon[27783]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:54:40.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:39 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:40.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:39 smithi086 ceph-mon[27783]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:54:40.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:39 smithi086 ceph-mon[27783]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:40.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:39 smithi183 ceph-mon[32344]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:54:40.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:39 smithi183 ceph-mon[32344]: from='osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:54:40.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:39 smithi183 ceph-mon[32344]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:54:40.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:39 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:40.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:39 smithi183 ceph-mon[32344]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:54:40.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:39 smithi183 ceph-mon[32344]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:54:41.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:54:41.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:54:41.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] boot 2024-04-25T18:54:41.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T18:54:41.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] boot 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T18:54:41.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:54:41.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:54:41.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: osd.5 [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] boot 2024-04-25T18:54:41.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T18:54:41.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:54:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:54:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:54:41.933 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-25T18:54:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-25T18:54:41.934 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0340079 s, 308 MB/s 2024-04-25T18:54:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-25T18:54:42.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:41 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:54:42.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:41 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:54:42.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:41 smithi194 ceph-mon[32740]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:42.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:41 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:54:42.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:41 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:54:42.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:41 smithi086 ceph-mon[27783]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:42.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:41 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:54:42.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:41 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:54:42.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:41 smithi183 ceph-mon[32344]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:42.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-04-25T18:54:43.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:54:43.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi194 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi194 to 8544M 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:43.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:42 smithi194 ceph-mon[32740]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:54:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:43.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:54:43.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi194 2024-04-25T18:54:43.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi194 to 8544M 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:42 smithi086 ceph-mon[27783]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:54:43.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi194 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi194 to 8544M 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:43.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:54:43.419 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:42 smithi183 ceph-mon[32344]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:54:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:43 smithi086 ceph-mon[27783]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:44.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:43 smithi183 ceph-mon[32344]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:44.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:43 smithi194 ceph-mon[32740]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:45.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:44 smithi086 ceph-mon[27783]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:54:45.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 -- 172.21.15.86:0/731156921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb881026a0 msgr2=0x7fcb88102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:45.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 --2- 172.21.15.86:0/731156921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb881026a0 0x7fcb88102b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009f90 tx=0x7fcb8000b330 comp rx=0 tx=0).stop 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 -- 172.21.15.86:0/731156921 shutdown_connections 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 --2- 172.21.15.86:0/731156921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb881026a0 0x7fcb88102b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 --2- 172.21.15.86:0/731156921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb88101380 0x7fcb881017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 --2- 172.21.15.86:0/731156921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb88101ce0 0x7fcb88102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 -- 172.21.15.86:0/731156921 >> 172.21.15.86:0/731156921 conn(0x7fcb880fcf40 msgr2=0x7fcb880ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 -- 172.21.15.86:0/731156921 shutdown_connections 2024-04-25T18:54:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.310+0000 7fcb8d800700 1 -- 172.21.15.86:0/731156921 wait complete. 2024-04-25T18:54:45.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 Processor -- start 2024-04-25T18:54:45.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 -- start start 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 0x7fcb8819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb88101ce0 0x7fcb8819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 0x7fcb881a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb88110750 con 0x7fcb881026a0 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881105d0 con 0x7fcb88101ce0 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.312+0000 7fcb8d800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881108d0 con 0x7fcb88101380 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 0x7fcb8819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:45.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb87400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 0x7fcb881a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 0x7fcb8819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41012/0 (socket says 172.21.15.86:41012) 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb87400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 0x7fcb881a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56044/0 (socket says 172.21.15.86:56044) 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 -- 172.21.15.86:0/2815952861 learned_addr learned my addr 172.21.15.86:0/2815952861 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86000700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb88101ce0 0x7fcb8819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 -- 172.21.15.86:0/2815952861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb88101ce0 msgr2=0x7fcb8819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb88101ce0 0x7fcb8819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:45.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 -- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 msgr2=0x7fcb881a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 0x7fcb881a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb86a00700 1 -- 172.21.15.86:0/2815952861 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb881a2a70 con 0x7fcb88101380 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.313+0000 7fcb87400700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 0x7fcb881a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.314+0000 7fcb86a00700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 0x7fcb8819ae10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800e9f0 tx=0x7fcb7800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.314+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78004a30 con 0x7fcb88101380 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.314+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb780054d0 con 0x7fcb88101380 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.314+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb881a2d60 con 0x7fcb88101380 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.314+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800cc30 con 0x7fcb88101380 2024-04-25T18:54:45.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.314+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb881a31e0 con 0x7fcb88101380 2024-04-25T18:54:45.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.315+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb88069730 con 0x7fcb88101380 2024-04-25T18:54:45.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.317+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb78004bd0 con 0x7fcb88101380 2024-04-25T18:54:45.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.318+0000 7fcb7f400700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb6c0a3c40 0x7fcb6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:45.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.319+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7fcb780cf630 con 0x7fcb88101380 2024-04-25T18:54:45.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.319+0000 7fcb86000700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb6c0a3c40 0x7fcb6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.319+0000 7fcb86000700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb6c0a3c40 0x7fcb6c0a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009c60 tx=0x7fcb74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.323+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb7809ceb0 con 0x7fcb88101380 2024-04-25T18:54:45.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:44 smithi183 ceph-mon[32344]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:54:45.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:44 smithi194 ceph-mon[32740]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:54:45.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:54:45.562+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcb88062e80 con 0x7fcb6c0a3c40 2024-04-25T18:54:46.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:45 smithi086 ceph-mon[27783]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:54:46.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:45 smithi086 ceph-mon[27783]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:46.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:54:46.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:54:46.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:46.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:45 smithi183 ceph-mon[32344]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:54:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:45 smithi183 ceph-mon[32344]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:54:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:54:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:45 smithi194 ceph-mon[32740]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:54:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:45 smithi194 ceph-mon[32740]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:54:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:54:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:47.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:47 smithi086 ceph-mon[27783]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:47.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:47 smithi183 ceph-mon[32344]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:47 smithi194 ceph-mon[32740]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:48.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:48 smithi086 ceph-mon[27783]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:48.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:48 smithi183 ceph-mon[32344]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:48 smithi194 ceph-mon[32740]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:54:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:50 smithi086 ceph-mon[27783]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T18:54:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:50 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3582217301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b207644-8c34-4045-88b6-7cb304dcccec"}]: dispatch 2024-04-25T18:54:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:50 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3582217301' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b207644-8c34-4045-88b6-7cb304dcccec"}]': finished 2024-04-25T18:54:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:50 smithi086 ceph-mon[27783]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T18:54:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:54:50.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:50 smithi183 ceph-mon[32344]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T18:54:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:50 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3582217301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b207644-8c34-4045-88b6-7cb304dcccec"}]: dispatch 2024-04-25T18:54:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:50 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3582217301' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b207644-8c34-4045-88b6-7cb304dcccec"}]': finished 2024-04-25T18:54:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:50 smithi183 ceph-mon[32344]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T18:54:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:54:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:50 smithi194 ceph-mon[32740]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T18:54:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:50 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3582217301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b207644-8c34-4045-88b6-7cb304dcccec"}]: dispatch 2024-04-25T18:54:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:50 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3582217301' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b207644-8c34-4045-88b6-7cb304dcccec"}]': finished 2024-04-25T18:54:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:50 smithi194 ceph-mon[32740]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T18:54:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:54:51.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:51 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/465925832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:54:51.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:51 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/465925832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:54:51.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:51 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/465925832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:54:52.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:52 smithi086 ceph-mon[27783]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-25T18:54:52.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:52 smithi183 ceph-mon[32344]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-25T18:54:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:52 smithi194 ceph-mon[32740]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-25T18:54:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:54 smithi183 ceph-mon[32344]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T18:54:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:54 smithi194 ceph-mon[32740]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T18:54:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:54 smithi086 ceph-mon[27783]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T18:54:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:56 smithi086 ceph-mon[27783]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:54:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:56 smithi183 ceph-mon[32344]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:54:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:56 smithi194 ceph-mon[32740]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:54:58.573 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:58 smithi086 ceph-mon[27783]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:54:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:58 smithi183 ceph-mon[32344]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:54:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:58 smithi194 ceph-mon[32740]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:54:59.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:54:59.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:54:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:59.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:54:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:54:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:54:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:54:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:00.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:00 smithi086 ceph-mon[27783]: Deploying daemon osd.6 on smithi086 2024-04-25T18:55:00.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:00 smithi086 ceph-mon[27783]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:00 smithi183 ceph-mon[32344]: Deploying daemon osd.6 on smithi086 2024-04-25T18:55:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:00 smithi183 ceph-mon[32344]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:00 smithi194 ceph-mon[32740]: Deploying daemon osd.6 on smithi086 2024-04-25T18:55:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:00 smithi194 ceph-mon[32740]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:02 smithi183 ceph-mon[32344]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:02 smithi194 ceph-mon[32740]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:02.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:02 smithi086 ceph-mon[27783]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:04.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:04 smithi086 ceph-mon[27783]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:04 smithi183 ceph-mon[32344]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:04 smithi194 ceph-mon[32740]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:06.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:06 smithi086 ceph-mon[27783]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:06.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:06 smithi183 ceph-mon[32344]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:06 smithi194 ceph-mon[32740]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:08.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:08 smithi086 ceph-mon[27783]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:08.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:08.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:08.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:08.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:08 smithi183 ceph-mon[32344]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:08 smithi194 ceph-mon[32740]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:10 smithi183 ceph-mon[32344]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:10 smithi194 ceph-mon[32740]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:10.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:10 smithi086 ceph-mon[27783]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:12.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:12.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:12.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:12 smithi183 ceph-mon[32344]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:12.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:12 smithi183 ceph-mon[32344]: from='osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:55:12.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:12 smithi183 ceph-mon[32344]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:55:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:12 smithi194 ceph-mon[32740]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:12 smithi194 ceph-mon[32740]: from='osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:55:12.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:12 smithi194 ceph-mon[32740]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:55:12.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:12.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:12.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:12 smithi086 ceph-mon[27783]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:12.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:12 smithi086 ceph-mon[27783]: from='osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:55:12.510 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:12 smithi086 ceph-mon[27783]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:55:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.615+0000 7fcb7f400700 1 -- 172.21.15.86:0/2815952861 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb88062e80 con 0x7fcb6c0a3c40 2024-04-25T18:55:12.617 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-04-25T18:55:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb6c0a3c40 msgr2=0x7fcb6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb6c0a3c40 0x7fcb6c0a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009c60 tx=0x7fcb74009260 comp rx=0 tx=0).stop 2024-04-25T18:55:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 msgr2=0x7fcb8819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 0x7fcb8819ae10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800e9f0 tx=0x7fcb7800ed00 comp rx=0 tx=0).stop 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 shutdown_connections 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb6c0a3c40 0x7fcb6c0a6100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb881026a0 0x7fcb881a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb88101380 0x7fcb8819ae10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 --2- 172.21.15.86:0/2815952861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb88101ce0 0x7fcb8819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 >> 172.21.15.86:0/2815952861 conn(0x7fcb880fcf40 msgr2=0x7fcb880fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 shutdown_connections 2024-04-25T18:55:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:55:12.619+0000 7fcb8d800700 1 -- 172.21.15.86:0/2815952861 wait complete. 2024-04-25T18:55:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:55:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:55:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:55:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:55:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:13.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:13.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:13.410 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T18:55:13.414 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-04-25T18:55:13.415 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:55:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:55:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:55:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:55:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:55:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:13.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:13.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:55:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:55:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:55:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:55:13.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:13.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:13.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:13.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:13.658 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:55:14.026 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T18:55:13.838+0000 7f9a75800700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T18:55:14.026 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 18:55:13 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T18:55:13.861+0000 7f9a6b600700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:55:14.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:14 smithi086 ceph-mon[27783]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:55:14.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:14 smithi086 ceph-mon[27783]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:55:14.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:14.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:14 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:14.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:14 smithi086 ceph-mon[27783]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:14.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:14 smithi086 ceph-mon[27783]: from='osd.6 ' entity='osd.6' 2024-04-25T18:55:14.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:14 smithi183 ceph-mon[32344]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:55:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:14 smithi183 ceph-mon[32344]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:55:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:14 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:14 smithi183 ceph-mon[32344]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:14.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:14 smithi183 ceph-mon[32344]: from='osd.6 ' entity='osd.6' 2024-04-25T18:55:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:14 smithi194 ceph-mon[32740]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:55:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:14 smithi194 ceph-mon[32740]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:55:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:14 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:14 smithi194 ceph-mon[32740]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:55:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:14 smithi194 ceph-mon[32740]: from='osd.6 ' entity='osd.6' 2024-04-25T18:55:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:15 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:55:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:15 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:55:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:15 smithi086 ceph-mon[27783]: osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] boot 2024-04-25T18:55:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:15 smithi086 ceph-mon[27783]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T18:55:15.414 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:15.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:15 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:55:15.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:15 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:55:15.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:15.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:15 smithi183 ceph-mon[32344]: osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] boot 2024-04-25T18:55:15.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:15 smithi183 ceph-mon[32344]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T18:55:15.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:15 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:55:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:15 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:55:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:15 smithi194 ceph-mon[32740]: osd.6 [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] boot 2024-04-25T18:55:15.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:15 smithi194 ceph-mon[32740]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T18:55:15.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:55:15.748 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:55:15.748 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:55:15.748 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T18:55:15.749 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T18:55:15.749 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126361 s, 830 MB/s 2024-04-25T18:55:15.749 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T18:55:16.390 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:16 smithi183 ceph-mon[32344]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:16.422 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-04-25T18:55:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:16 smithi194 ceph-mon[32740]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:16.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:16 smithi086 ceph-mon[27783]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:16.662 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:55:17.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:17 smithi183 ceph-mon[32344]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:55:17.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:17 smithi194 ceph-mon[32740]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:55:17.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:17 smithi086 ceph-mon[27783]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:55:18.285 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:18 smithi086 ceph-mon[27783]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:18.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:18 smithi183 ceph-mon[32344]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:18.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 -- 172.21.15.183:0/3511426220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c108780 msgr2=0x7f209c10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/3511426220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c108780 0x7f209c10abf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f209400b0a0 tx=0x7f209400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 -- 172.21.15.183:0/3511426220 shutdown_connections 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/3511426220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c108780 0x7f209c10abf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/3511426220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f209c103520 0x7f209c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/3511426220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209c105e50 0x7f209c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 -- 172.21.15.183:0/3511426220 >> 172.21.15.183:0/3511426220 conn(0x7f209c0fcf30 msgr2=0x7f209c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 -- 172.21.15.183:0/3511426220 shutdown_connections 2024-04-25T18:55:18.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.419+0000 7f20a3d3b700 1 -- 172.21.15.183:0/3511426220 wait complete. 2024-04-25T18:55:18.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 Processor -- start 2024-04-25T18:55:18.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 -- start start 2024-04-25T18:55:18.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209c103520 0x7f209c102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 0x7f209c1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f209c108780 0x7f209c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10e570 con 0x7f209c103520 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10e3f0 con 0x7f209c105e50 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a3d3b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10e6f0 con 0x7f209c108780 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 0x7f209c1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 0x7f209c1014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:52830/0 (socket says 172.21.15.183:52830) 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 -- 172.21.15.183:0/1319730466 learned_addr learned my addr 172.21.15.183:0/1319730466 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a22d8700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f209c108780 0x7f209c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 -- 172.21.15.183:0/1319730466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f209c108780 msgr2=0x7f209c101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f209c108780 0x7f209c101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 -- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209c103520 msgr2=0x7f209c102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209c103520 0x7f209c102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.420+0000 7f20a12d6700 1 -- 172.21.15.183:0/1319730466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c1022c0 con 0x7f209c105e50 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.421+0000 7f20a12d6700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 0x7f209c1014d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f208c00b530 tx=0x7f208c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.421+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c015020 con 0x7f209c105e50 2024-04-25T18:55:18.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.421+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c1a2da0 con 0x7f209c105e50 2024-04-25T18:55:18.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.421+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c011ce0 con 0x7f209c105e50 2024-04-25T18:55:18.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.421+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c010d20 con 0x7f209c105e50 2024-04-25T18:55:18.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.421+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c1a32d0 con 0x7f209c105e50 2024-04-25T18:55:18.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.422+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c069730 con 0x7f209c105e50 2024-04-25T18:55:18.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.423+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f208c018020 con 0x7f209c105e50 2024-04-25T18:55:18.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.423+0000 7f2092ffd700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f20880a3c40 0x7f20880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.423+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f208c0ceee0 con 0x7f209c105e50 2024-04-25T18:55:18.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.424+0000 7f20a1ad7700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f20880a3c40 0x7f20880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.425+0000 7f20a1ad7700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f20880a3c40 0x7f20880a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2098005950 tx=0x7f20980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.428+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f208c01f020 con 0x7f209c105e50 2024-04-25T18:55:18.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:18 smithi194 ceph-mon[32740]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:18.639 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:18.638+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f209c062e80 con 0x7f20880a3c40 2024-04-25T18:55:19.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi086 2024-04-25T18:55:19.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:55:19.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi086 2024-04-25T18:55:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:55:19.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi086 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:19.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:55:19.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:55:19.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:20.577 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:20 smithi183 ceph-mon[32344]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:20.578 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:20 smithi183 ceph-mon[32344]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:20 smithi194 ceph-mon[32740]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:20 smithi194 ceph-mon[32740]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:20 smithi086 ceph-mon[27783]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:20 smithi086 ceph-mon[27783]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/3771338402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]: dispatch 2024-04-25T18:55:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]: dispatch 2024-04-25T18:55:22.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]': finished 2024-04-25T18:55:22.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T18:55:22.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:22.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:22 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/151360776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:55:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:22.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/3771338402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]: dispatch 2024-04-25T18:55:22.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]: dispatch 2024-04-25T18:55:22.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]': finished 2024-04-25T18:55:22.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T18:55:22.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:22.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:22 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/151360776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/3771338402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]: dispatch 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]: dispatch 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75d0252d-dd63-4aa7-982b-bd260501a0e1"}]': finished 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:22 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/151360776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:55:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:24 smithi194 ceph-mon[32740]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:24 smithi086 ceph-mon[27783]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:24 smithi183 ceph-mon[32344]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:26.727 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:26 smithi183 ceph-mon[32344]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:26 smithi194 ceph-mon[32740]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:26 smithi086 ceph-mon[27783]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:28 smithi194 ceph-mon[32740]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:28 smithi086 ceph-mon[27783]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:28.830 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:28 smithi183 ceph-mon[32344]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:29.725 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:55:29.725 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:55:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:29.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:55:29.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:30 smithi194 ceph-mon[32740]: Deploying daemon osd.7 on smithi183 2024-04-25T18:55:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:30 smithi194 ceph-mon[32740]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:30.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:30 smithi183 ceph-mon[32344]: Deploying daemon osd.7 on smithi183 2024-04-25T18:55:30.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:30 smithi183 ceph-mon[32344]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:30 smithi086 ceph-mon[27783]: Deploying daemon osd.7 on smithi183 2024-04-25T18:55:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:30 smithi086 ceph-mon[27783]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:32 smithi183 ceph-mon[32344]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:32 smithi194 ceph-mon[32740]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:32 smithi086 ceph-mon[27783]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:34 smithi183 ceph-mon[32344]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:34 smithi194 ceph-mon[32740]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:34 smithi086 ceph-mon[27783]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:35.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:35.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:35.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:35.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:35 smithi183 ceph-mon[32344]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:35 smithi194 ceph-mon[32740]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:35 smithi086 ceph-mon[27783]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:38 smithi086 ceph-mon[27783]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:38.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:38 smithi183 ceph-mon[32344]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:38.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:38.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:38.592 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.674 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-04-25T18:55:38.674 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.672+0000 7f2092ffd700 1 -- 172.21.15.183:0/1319730466 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f209c062e80 con 0x7f20880a3c40 2024-04-25T18:55:38.676 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f20880a3c40 msgr2=0x7f20880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:38.676 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f20880a3c40 0x7f20880a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2098005950 tx=0x7f20980058e0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.676 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 msgr2=0x7f209c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 0x7f209c1014d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f208c00b530 tx=0x7f208c00b8f0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 shutdown_connections 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f20880a3c40 0x7f20880a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f209c108780 0x7f209c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209c103520 0x7f209c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 --2- 172.21.15.183:0/1319730466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c105e50 0x7f209c1014d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 >> 172.21.15.183:0/1319730466 conn(0x7f209c0fcf30 msgr2=0x7f209c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 shutdown_connections 2024-04-25T18:55:38.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:55:38.675+0000 7f20a3d3b700 1 -- 172.21.15.183:0/1319730466 wait complete. 2024-04-25T18:55:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:38 smithi194 ceph-mon[32740]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.441 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T18:55:39.443 INFO:tasks.cephadm:Deploying osd.8 on smithi194 with /dev/vg_nvme/lv_2... 2024-04-25T18:55:39.443 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:55:39.645 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:39 smithi183 ceph-mon[32344]: from='osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:55:39.645 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:39 smithi183 ceph-mon[32344]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:55:39.645 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:39 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.645 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:39 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.645 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:39 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:39.689 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:55:39.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:39 smithi194 ceph-mon[32740]: from='osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:55:39.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:39 smithi194 ceph-mon[32740]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:55:39.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:39 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:39 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:39 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:39 smithi086 ceph-mon[27783]: from='osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:55:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:39 smithi086 ceph-mon[27783]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:55:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:39 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:39 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:39 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: from='osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:55:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-mon[32344]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:55:40.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T18:55:40.329+0000 7f77cdc94700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T18:55:40.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 18:55:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T18:55:40.339+0000 7f77c9a87700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:55:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:55:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: from='osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:55:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:55:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:40.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:55:40.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:55:40.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:40 smithi194 ceph-mon[32740]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:55:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:55:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: from='osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:55:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:55:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:55:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:55:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:40 smithi086 ceph-mon[27783]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:55:41.347 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:41 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:41.348 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:41 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:41.348 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:41 smithi183 ceph-mon[32344]: osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] boot 2024-04-25T18:55:41.348 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:41 smithi183 ceph-mon[32344]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T18:55:41.440 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:41 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:41.440 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:41 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:41.440 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:41 smithi194 ceph-mon[32740]: osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] boot 2024-04-25T18:55:41.440 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:41 smithi194 ceph-mon[32740]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T18:55:41.785 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:55:41.786 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:55:41.786 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-25T18:55:41.786 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-25T18:55:41.786 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130109 s, 806 MB/s 2024-04-25T18:55:41.786 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-25T18:55:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:41 smithi086 ceph-mon[27783]: osd.7 [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] boot 2024-04-25T18:55:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:41 smithi086 ceph-mon[27783]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T18:55:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:42 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:55:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:42 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:55:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:42 smithi194 ceph-mon[32740]: pgmap v193: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:42 smithi194 ceph-mon[32740]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T18:55:42.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:42 smithi194 ceph-mon[32740]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T18:55:42.516 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2024-04-25T18:55:42.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:42 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:55:42.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:42 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:55:42.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:42 smithi183 ceph-mon[32344]: pgmap v193: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:42.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:42.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:42 smithi183 ceph-mon[32344]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T18:55:42.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:42 smithi183 ceph-mon[32344]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T18:55:42.761 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:55:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:42 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:55:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:42 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:55:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:42 smithi086 ceph-mon[27783]: pgmap v193: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:55:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:55:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:42 smithi086 ceph-mon[27783]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T18:55:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:42 smithi086 ceph-mon[27783]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T18:55:43.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi183 2024-04-25T18:55:43.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:43.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:43 smithi194 ceph-mon[32740]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi183 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:44.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:43 smithi086 ceph-mon[27783]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:55:44.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi183 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:44.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:43 smithi183 ceph-mon[32344]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 -- 172.21.15.194:0/3750095240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238103530 msgr2=0x7fd238109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/3750095240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238103530 0x7fd238109de0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd23000b0a0 tx=0x7fd23000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 -- 172.21.15.194:0/3750095240 shutdown_connections 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/3750095240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238103530 0x7fd238109de0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/3750095240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd238101990 0x7fd238101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/3750095240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd238102b90 0x7fd238102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:44.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.606+0000 7fd23e1b8700 1 -- 172.21.15.194:0/3750095240 >> 172.21.15.194:0/3750095240 conn(0x7fd2380fcf30 msgr2=0x7fd2380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:44.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.607+0000 7fd23e1b8700 1 -- 172.21.15.194:0/3750095240 shutdown_connections 2024-04-25T18:55:44.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.607+0000 7fd23e1b8700 1 -- 172.21.15.194:0/3750095240 wait complete. 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.607+0000 7fd23e1b8700 1 Processor -- start 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.607+0000 7fd23e1b8700 1 -- start start 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 0x7fd23806bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd238102b90 0x7fd23819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd238103530 0x7fd23819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd23810d4a0 con 0x7fd238103530 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd23810d320 con 0x7fd238102b90 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd23810d620 con 0x7fd238101990 2024-04-25T18:55:44.611 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 0x7fd23806bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 0x7fd23806bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:32978/0 (socket says 172.21.15.194:32978) 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 -- 172.21.15.194:0/1654855017 learned_addr learned my addr 172.21.15.194:0/1654855017 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd236ffd700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd238102b90 0x7fd23819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 -- 172.21.15.194:0/1654855017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd238102b90 msgr2=0x7fd23819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd238102b90 0x7fd23819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 -- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd238103530 msgr2=0x7fd23819feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd238103530 0x7fd23819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 -- 172.21.15.194:0/1654855017 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2381a0780 con 0x7fd238101990 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd2377fe700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 0x7fd23806bd50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b500 tx=0x7fd22800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228015020 con 0x7fd238101990 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd228011ce0 con 0x7fd238101990 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2381a0a70 con 0x7fd238101990 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228010d20 con 0x7fd238101990 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.608+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2381a1030 con 0x7fd238101990 2024-04-25T18:55:44.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.609+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd238069730 con 0x7fd238101990 2024-04-25T18:55:44.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.611+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd228018020 con 0x7fd238101990 2024-04-25T18:55:44.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.611+0000 7fd234ff9700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd2200a3c40 0x7fd2200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:44.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.611+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7fd2280cf560 con 0x7fd238101990 2024-04-25T18:55:44.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.612+0000 7fd236ffd700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd2200a3c40 0x7fd2200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:44.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.613+0000 7fd236ffd700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd2200a3c40 0x7fd2200a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd23806c2b0 tx=0x7fd22c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:44.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.615+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd22801f020 con 0x7fd238101990 2024-04-25T18:55:44.827 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:55:44.826+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd238062e80 con 0x7fd2200a3c40 2024-04-25T18:55:45.604 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:45 smithi194 ceph-mon[32740]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:55:45.604 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:55:45.604 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:55:45.604 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:45.669 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:45 smithi183 ceph-mon[32344]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:55:45.669 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:55:45.669 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:55:45.669 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:45.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:45 smithi086 ceph-mon[27783]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:55:45.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:55:45.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:55:45.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:46.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:46 smithi086 ceph-mon[27783]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:46.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:46 smithi086 ceph-mon[27783]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:46.520 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:46 smithi086 ceph-mon[27783]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:55:46.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:46 smithi183 ceph-mon[32344]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:46 smithi183 ceph-mon[32344]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:46 smithi183 ceph-mon[32344]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:55:46.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:46 smithi194 ceph-mon[32740]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:46.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:46 smithi194 ceph-mon[32740]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:46.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:46 smithi194 ceph-mon[32740]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:55:47.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:47 smithi183 ceph-mon[32344]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T18:55:47.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:47 smithi183 ceph-mon[32344]: Cluster is now healthy 2024-04-25T18:55:47.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:47 smithi194 ceph-mon[32740]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T18:55:47.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:47 smithi194 ceph-mon[32740]: Cluster is now healthy 2024-04-25T18:55:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:47 smithi086 ceph-mon[27783]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T18:55:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:47 smithi086 ceph-mon[27783]: Cluster is now healthy 2024-04-25T18:55:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/3636332969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]: dispatch 2024-04-25T18:55:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]: dispatch 2024-04-25T18:55:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]': finished 2024-04-25T18:55:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T18:55:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:55:48.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:48 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/2276885096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:55:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/3636332969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]: dispatch 2024-04-25T18:55:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]: dispatch 2024-04-25T18:55:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]': finished 2024-04-25T18:55:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T18:55:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:55:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:48 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/2276885096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:55:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/3636332969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]: dispatch 2024-04-25T18:55:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]: dispatch 2024-04-25T18:55:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcd61ecf-016d-40fd-8a13-e4aa1828d49e"}]': finished 2024-04-25T18:55:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T18:55:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:55:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:48 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/2276885096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:55:50.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:50 smithi183 ceph-mon[32344]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:50 smithi194 ceph-mon[32740]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:50 smithi086 ceph-mon[27783]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:52.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:52 smithi183 ceph-mon[32344]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:52 smithi194 ceph-mon[32740]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:52 smithi086 ceph-mon[27783]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:54.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:54 smithi194 ceph-mon[32740]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:54 smithi086 ceph-mon[27783]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:54 smithi183 ceph-mon[32344]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:55:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:55 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:55:55.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:55 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:55.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:55:55.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:55 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:56 smithi194 ceph-mon[32740]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:56 smithi194 ceph-mon[32740]: Deploying daemon osd.8 on smithi194 2024-04-25T18:55:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:56 smithi086 ceph-mon[27783]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:56 smithi086 ceph-mon[27783]: Deploying daemon osd.8 on smithi194 2024-04-25T18:55:56.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:56 smithi183 ceph-mon[32344]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:56.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:56 smithi183 ceph-mon[32344]: Deploying daemon osd.8 on smithi194 2024-04-25T18:55:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:55:58 smithi194 ceph-mon[32740]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:55:58 smithi086 ceph-mon[27783]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:55:58.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:55:58 smithi183 ceph-mon[32344]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:00.613 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:00 smithi194 ceph-mon[32740]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:00 smithi086 ceph-mon[27783]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:00 smithi183 ceph-mon[32344]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:02 smithi086 ceph-mon[27783]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:02.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:02.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:02 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:02 smithi183 ceph-mon[32344]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:02 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:02 smithi194 ceph-mon[32740]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:02.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:02.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:02.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:02 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:04 smithi194 ceph-mon[32740]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:04 smithi086 ceph-mon[27783]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:04 smithi183 ceph-mon[32344]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:05.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.315+0000 7fd234ff9700 1 -- 172.21.15.194:0/1654855017 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd238062e80 con 0x7fd2200a3c40 2024-04-25T18:56:05.318 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 8 on host 'smithi194' 2024-04-25T18:56:05.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd2200a3c40 msgr2=0x7fd2200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd2200a3c40 0x7fd2200a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd23806c2b0 tx=0x7fd22c00a380 comp rx=0 tx=0).stop 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 msgr2=0x7fd23806bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 0x7fd23806bd50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b500 tx=0x7fd22800b810 comp rx=0 tx=0).stop 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 shutdown_connections 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd2200a3c40 0x7fd2200a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd238103530 0x7fd23819feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd238101990 0x7fd23806bd50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 --2- 172.21.15.194:0/1654855017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd238102b90 0x7fd23819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 >> 172.21.15.194:0/1654855017 conn(0x7fd2380fcf30 msgr2=0x7fd238105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 shutdown_connections 2024-04-25T18:56:05.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:56:05.317+0000 7fd23e1b8700 1 -- 172.21.15.194:0/1654855017 wait complete. 2024-04-25T18:56:05.670 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.670 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.670 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:05.670 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:05.670 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.670 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:56:05.671 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:56:05.671 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.671 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.671 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:05 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:05.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:05 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:05.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:05.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:05 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:06.019 DEBUG:teuthology.orchestra.run.smithi194:osd.8> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T18:56:06.022 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-04-25T18:56:06.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:56:06.463 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-mon[32740]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-mon[32740]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:56:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-mon[32740]: from='osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:56:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-mon[32740]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T18:56:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-mon[32740]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:56:06.732 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T18:56:06.453+0000 7f38309f1700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T18:56:06.732 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 18:56:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T18:56:06.465+0000 7f382c7e4700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:56:06.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:06 smithi086 ceph-mon[27783]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:06.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:06 smithi086 ceph-mon[27783]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:56:06.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:06 smithi086 ceph-mon[27783]: from='osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:56:06.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:06 smithi086 ceph-mon[27783]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T18:56:06.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:06.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:06 smithi086 ceph-mon[27783]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:56:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:06 smithi183 ceph-mon[32344]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:06 smithi183 ceph-mon[32344]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:56:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:06 smithi183 ceph-mon[32344]: from='osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:56:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:06 smithi183 ceph-mon[32344]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T18:56:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:06 smithi183 ceph-mon[32344]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:56:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:07 smithi194 ceph-mon[32740]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:56:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:07 smithi194 ceph-mon[32740]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:56:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:07.809 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:07 smithi086 ceph-mon[27783]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:56:07.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:07 smithi086 ceph-mon[27783]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:56:07.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:07.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:07.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:07 smithi183 ceph-mon[32344]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:56:07.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:07 smithi183 ceph-mon[32344]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:56:07.829 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:07.830 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:08.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:56:08.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:56:08.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:08.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] boot 2024-04-25T18:56:08.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T18:56:08.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:08.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:08 smithi194 ceph-mon[32740]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] boot 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:08 smithi086 ceph-mon[27783]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T18:56:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:56:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:56:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:56:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: osd.8 [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] boot 2024-04-25T18:56:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T18:56:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:56:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:08 smithi183 ceph-mon[32344]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T18:56:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:56:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:56:09.322 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-25T18:56:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-25T18:56:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0329435 s, 318 MB/s 2024-04-25T18:56:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-25T18:56:10.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-04-25T18:56:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi194 2024-04-25T18:56:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi194 to 5696M 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:10 smithi086 ceph-mon[27783]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi194 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi194 to 5696M 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:10.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:10.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:10 smithi183 ceph-mon[32344]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi194 2024-04-25T18:56:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi194 to 5696M 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:10.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:10 smithi194 ceph-mon[32740]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:11.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:11 smithi183 ceph-mon[32344]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:56:11.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:11 smithi194 ceph-mon[32740]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:56:11.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:11 smithi086 ceph-mon[27783]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:56:12.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:12 smithi086 ceph-mon[27783]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:56:12.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:12 smithi086 ceph-mon[27783]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:12.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:12 smithi183 ceph-mon[32344]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:56:12.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:12 smithi183 ceph-mon[32344]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:12 smithi194 ceph-mon[32740]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:56:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:12 smithi194 ceph-mon[32740]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 -- 172.21.15.86:0/2168510490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c105e60 msgr2=0x7f518c108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 --2- 172.21.15.86:0/2168510490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c105e60 0x7f518c108250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5178009990 tx=0x7f5178009ca0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 -- 172.21.15.86:0/2168510490 shutdown_connections 2024-04-25T18:56:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 --2- 172.21.15.86:0/2168510490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f518c108790 0x7f518c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 --2- 172.21.15.86:0/2168510490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f518c103530 0x7f518c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 --2- 172.21.15.86:0/2168510490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c105e60 0x7f518c108250 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 -- 172.21.15.86:0/2168510490 >> 172.21.15.86:0/2168510490 conn(0x7f518c0fcf40 msgr2=0x7f518c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 -- 172.21.15.86:0/2168510490 shutdown_connections 2024-04-25T18:56:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.872+0000 7f5193a00700 1 -- 172.21.15.86:0/2168510490 wait complete. 2024-04-25T18:56:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.873+0000 7f5193a00700 1 Processor -- start 2024-04-25T18:56:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 -- start start 2024-04-25T18:56:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 0x7f518c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f518c105e60 0x7f518c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f518c108790 0x7f518c1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10e5a0 con 0x7f518c103530 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10e420 con 0x7f518c108790 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5193a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10e720 con 0x7f518c105e60 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5191000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 0x7f518c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5191000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 0x7f518c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35604/0 (socket says 172.21.15.86:35604) 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5191000700 1 -- 172.21.15.86:0/2508477455 learned_addr learned my addr 172.21.15.86:0/2508477455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f518be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f518c105e60 0x7f518c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.874+0000 7f5191a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f518c108790 0x7f518c1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5191000700 1 -- 172.21.15.86:0/2508477455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f518c105e60 msgr2=0x7f518c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5191000700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f518c105e60 0x7f518c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5191000700 1 -- 172.21.15.86:0/2508477455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f518c108790 msgr2=0x7f518c1a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5191000700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f518c108790 0x7f518c1a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5191000700 1 -- 172.21.15.86:0/2508477455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a0920 con 0x7f518c103530 2024-04-25T18:56:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5191000700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 0x7f518c198ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9c0 tx=0x7f518000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518000cbd0 con 0x7f518c103530 2024-04-25T18:56:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518000ee60 con 0x7f518c103530 2024-04-25T18:56:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.875+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a0c10 con 0x7f518c103530 2024-04-25T18:56:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.876+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800055c0 con 0x7f518c103530 2024-04-25T18:56:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.876+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a1120 con 0x7f518c103530 2024-04-25T18:56:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.877+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c069730 con 0x7f518c103530 2024-04-25T18:56:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.879+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5180004750 con 0x7f518c103530 2024-04-25T18:56:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.880+0000 7f5189600700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f51740a3cc0 0x7f51740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.880+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f51800cfcc0 con 0x7f518c103530 2024-04-25T18:56:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.880+0000 7f518be00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f51740a3cc0 0x7f51740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.881+0000 7f518be00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f51740a3cc0 0x7f51740a6180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f517800b760 tx=0x7f517800b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:12.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:12.885+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51800d6080 con 0x7f518c103530 2024-04-25T18:56:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:13.122+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f518c062e80 con 0x7f51740a3cc0 2024-04-25T18:56:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:56:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:56:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:56:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:56:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:13.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:56:13.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:56:13.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:14.399 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:14 smithi086 ceph-mon[27783]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:14.400 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:14 smithi086 ceph-mon[27783]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T18:56:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:14 smithi194 ceph-mon[32740]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:14 smithi194 ceph-mon[32740]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T18:56:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:14 smithi183 ceph-mon[32344]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:14 smithi183 ceph-mon[32344]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T18:56:16.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:16 smithi183 ceph-mon[32344]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:56:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:16 smithi194 ceph-mon[32740]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:56:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:16 smithi086 ceph-mon[27783]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:56:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:17 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3234405443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]: dispatch 2024-04-25T18:56:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:17 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]: dispatch 2024-04-25T18:56:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:17 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]': finished 2024-04-25T18:56:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:17 smithi183 ceph-mon[32344]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T18:56:17.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:17 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:17.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:17 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3234405443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]: dispatch 2024-04-25T18:56:17.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:17 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]: dispatch 2024-04-25T18:56:17.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:17 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]': finished 2024-04-25T18:56:17.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:17 smithi086 ceph-mon[27783]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T18:56:17.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:17 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:17 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3234405443' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]: dispatch 2024-04-25T18:56:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:17 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]: dispatch 2024-04-25T18:56:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:17 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8006e686-e97b-4cda-a577-6b8dc8f9d5f7"}]': finished 2024-04-25T18:56:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:17 smithi194 ceph-mon[32740]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T18:56:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:17 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:18.353 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:18 smithi086 ceph-mon[27783]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T18:56:18.354 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:18 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1101107125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:56:18.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:18 smithi183 ceph-mon[32344]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T18:56:18.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:18 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1101107125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:56:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:18 smithi194 ceph-mon[32740]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T18:56:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:18 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1101107125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:56:20.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:20 smithi183 ceph-mon[32344]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T18:56:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:20 smithi194 ceph-mon[32740]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T18:56:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:20 smithi086 ceph-mon[27783]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T18:56:22.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:22 smithi086 ceph-mon[27783]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:56:22.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:22 smithi183 ceph-mon[32344]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:56:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:22 smithi194 ceph-mon[32740]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:56:24.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:24 smithi183 ceph-mon[32344]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:24.703 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:24 smithi086 ceph-mon[27783]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:24 smithi194 ceph-mon[32740]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:26.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:26 smithi183 ceph-mon[32344]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:26 smithi194 ceph-mon[32740]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:26 smithi086 ceph-mon[27783]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:27.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:27 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:56:27.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:27 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:27.626 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:27 smithi086 ceph-mon[27783]: Deploying daemon osd.9 on smithi086 2024-04-25T18:56:27.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:27 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:56:27.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:27 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:27.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:27 smithi183 ceph-mon[32344]: Deploying daemon osd.9 on smithi086 2024-04-25T18:56:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:27 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:56:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:27 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:27.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:27 smithi194 ceph-mon[32740]: Deploying daemon osd.9 on smithi086 2024-04-25T18:56:28.456 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:28 smithi086 ceph-mon[27783]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:28.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:28 smithi183 ceph-mon[32344]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:28 smithi194 ceph-mon[32740]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:30.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:30 smithi183 ceph-mon[32344]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:30 smithi194 ceph-mon[32740]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:30 smithi086 ceph-mon[27783]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:32.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:32 smithi086 ceph-mon[27783]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:32 smithi183 ceph-mon[32344]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:32 smithi194 ceph-mon[32740]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:34.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:34 smithi086 ceph-mon[27783]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:34 smithi183 ceph-mon[32344]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:34 smithi194 ceph-mon[32740]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:36.050 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:36.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:36.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:36.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:35 smithi086 ceph-mon[27783]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:36.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:36.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:36.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:36.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:35 smithi183 ceph-mon[32344]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:36.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:36.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:36.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:36.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:35 smithi194 ceph-mon[32740]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:38 smithi086 ceph-mon[27783]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:38 smithi183 ceph-mon[32344]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:38 smithi194 ceph-mon[32740]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:39.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.627+0000 7f5189600700 1 -- 172.21.15.86:0/2508477455 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f518c062e80 con 0x7f51740a3cc0 2024-04-25T18:56:39.630 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f51740a3cc0 msgr2=0x7f51740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f51740a3cc0 0x7f51740a6180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f517800b760 tx=0x7f517800b2a0 comp rx=0 tx=0).stop 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 msgr2=0x7f518c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 0x7f518c198ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9c0 tx=0x7f518000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 shutdown_connections 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f51740a3cc0 0x7f51740a6180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f518c108790 0x7f518c1a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f518c103530 0x7f518c198ca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 --2- 172.21.15.86:0/2508477455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f518c105e60 0x7f518c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 >> 172.21.15.86:0/2508477455 conn(0x7f518c0fcf40 msgr2=0x7f518c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 shutdown_connections 2024-04-25T18:56:39.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:56:39.631+0000 7f5193a00700 1 -- 172.21.15.86:0/2508477455 wait complete. 2024-04-25T18:56:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:39 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:39 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:39 smithi086 ceph-mon[27783]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:56:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:39 smithi086 ceph-mon[27783]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:39.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:39 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:39.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:39 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:39.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:39 smithi183 ceph-mon[32344]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:56:39.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:39 smithi183 ceph-mon[32344]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:39 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:39 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:39 smithi194 ceph-mon[32740]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:56:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:39 smithi194 ceph-mon[32740]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:40.475 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T18:56:40.520 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-04-25T18:56:40.521 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:56:40.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:40 smithi086 ceph-mon[27783]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:56:40.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:40 smithi086 ceph-mon[27783]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T18:56:40.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:40 smithi086 ceph-mon[27783]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:56:40.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:40.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:40.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:40.767 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:56:40.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:40 smithi183 ceph-mon[32344]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:56:40.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:40 smithi183 ceph-mon[32344]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T18:56:40.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:40 smithi183 ceph-mon[32344]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:56:40.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:40.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:40.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:40 smithi194 ceph-mon[32740]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:56:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:40 smithi194 ceph-mon[32740]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T18:56:40.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:40 smithi194 ceph-mon[32740]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-25T18:56:40.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:40.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:40.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:41 smithi086 ceph-mon[27783]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:56:41.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:41 smithi086 ceph-mon[27783]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:56:41.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:41.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:41 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:41.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:41 smithi086 ceph-mon[27783]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:41.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:41 smithi183 ceph-mon[32344]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:56:41.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:41 smithi183 ceph-mon[32344]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:56:41.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:41 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:41.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:41 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:41.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:41 smithi183 ceph-mon[32344]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:41.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:41 smithi194 ceph-mon[32740]: from='osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-25T18:56:41.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:41 smithi194 ceph-mon[32740]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:56:41.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:41 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:41.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:41 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:41.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:41 smithi194 ceph-mon[32740]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:56:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:42 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:56:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:42 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:56:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:42 smithi183 ceph-mon[32344]: osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] boot 2024-04-25T18:56:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:42 smithi183 ceph-mon[32344]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T18:56:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:42 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:42.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:42 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:56:42.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:42 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:56:42.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:42 smithi086 ceph-mon[27783]: osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] boot 2024-04-25T18:56:42.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:42 smithi086 ceph-mon[27783]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T18:56:42.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:42 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:42.893 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:56:42.894 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:56:42.894 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-25T18:56:42.894 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-25T18:56:42.894 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124017 s, 846 MB/s 2024-04-25T18:56:42.894 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-25T18:56:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:42 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:56:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:42 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:56:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:42 smithi194 ceph-mon[32740]: osd.9 [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] boot 2024-04-25T18:56:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:42 smithi194 ceph-mon[32740]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T18:56:42.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:42 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:56:43.576 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-04-25T18:56:43.811 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:56:43.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:43 smithi086 ceph-mon[27783]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T18:56:43.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:43 smithi086 ceph-mon[27783]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:43.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:43 smithi183 ceph-mon[32344]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T18:56:43.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:43 smithi183 ceph-mon[32344]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:43.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:43 smithi194 ceph-mon[32740]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T18:56:43.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:43 smithi194 ceph-mon[32740]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 -- 172.21.15.183:0/1052767923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 msgr2=0x7f78ac109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/1052767923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac109df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78a80099c0 tx=0x7f78a8009cd0 comp rx=0 tx=0).stop 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 -- 172.21.15.183:0/1052767923 shutdown_connections 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/1052767923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac109df0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/1052767923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac1019a0 0x7f78ac101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/1052767923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78ac102ba0 0x7f78ac103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 -- 172.21.15.183:0/1052767923 >> 172.21.15.183:0/1052767923 conn(0x7f78ac0fcf60 msgr2=0x7f78ac0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 -- 172.21.15.183:0/1052767923 shutdown_connections 2024-04-25T18:56:45.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 -- 172.21.15.183:0/1052767923 wait complete. 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.477+0000 7f78b3ac8700 1 Processor -- start 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 -- start start 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78ac1019a0 0x7f78ac198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac102ba0 0x7f78ac199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac10d4d0 con 0x7f78ac102ba0 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac10d350 con 0x7f78ac103540 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac10d650 con 0x7f78ac1019a0 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:45.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:49402/0 (socket says 172.21.15.183:49402) 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 -- 172.21.15.183:0/2932976642 learned_addr learned my addr 172.21.15.183:0/2932976642 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b1864700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78ac1019a0 0x7f78ac198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 -- 172.21.15.183:0/2932976642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78ac1019a0 msgr2=0x7f78ac198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78ac1019a0 0x7f78ac198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 -- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac102ba0 msgr2=0x7f78ac199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac102ba0 0x7f78ac199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 -- 172.21.15.183:0/2932976642 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac1a0860 con 0x7f78ac103540 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b2065700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac1a0190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78a80059d0 tx=0x7f78a80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8027070 con 0x7f78ac103540 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac1a0af0 con 0x7f78ac103540 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a8004160 con 0x7f78ac103540 2024-04-25T18:56:45.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a800f460 con 0x7f78ac103540 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.478+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac1a0fa0 con 0x7f78ac103540 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.479+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac069730 con 0x7f78ac103540 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.480+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78a802a070 con 0x7f78ac103540 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.481+0000 7f78a2ffd700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f78980a3c40 0x7f78980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.481+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f78a80d0e30 con 0x7f78ac103540 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.481+0000 7f78b1864700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f78980a3c40 0x7f78980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.482+0000 7f78b1864700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f78980a3c40 0x7f78980a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f78ac19a0f0 tx=0x7f78a40094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:45.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.485+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78a8031080 con 0x7f78ac103540 2024-04-25T18:56:45.698 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:56:45.697+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f78ac0ff0f0 con 0x7f78980a3c40 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi086 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.337 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:56:46.338 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi086 2024-04-25T18:56:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:46.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:56:46.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:56:46.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi086 2024-04-25T18:56:46.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:46.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:46.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:46.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:46.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:56:46.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:56:46.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:56:46.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:47.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:47 smithi183 ceph-mon[32344]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:47 smithi194 ceph-mon[32740]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:47.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:47 smithi086 ceph-mon[27783]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:48.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:48 smithi183 ceph-mon[32344]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:48 smithi194 ceph-mon[32740]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:48.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:48 smithi086 ceph-mon[27783]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:49.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:49 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/1368680521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]: dispatch 2024-04-25T18:56:49.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:49 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]: dispatch 2024-04-25T18:56:49.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:49 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]': finished 2024-04-25T18:56:49.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:49 smithi183 ceph-mon[32344]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T18:56:49.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:56:49.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:49 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/1368680521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]: dispatch 2024-04-25T18:56:49.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:49 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]: dispatch 2024-04-25T18:56:49.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:49 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]': finished 2024-04-25T18:56:49.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:49 smithi194 ceph-mon[32740]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T18:56:49.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:56:49.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:49 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/1368680521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]: dispatch 2024-04-25T18:56:49.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:49 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]: dispatch 2024-04-25T18:56:49.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:49 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f7367bb-2998-47a7-ae18-fd713be7d07e"}]': finished 2024-04-25T18:56:49.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:49 smithi086 ceph-mon[27783]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T18:56:49.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:56:50.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:50 smithi183 ceph-mon[32344]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:50.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:50 smithi183 ceph-mon[32344]: from='client.? 172.21.15.183:0/213598179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:56:50.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:50 smithi194 ceph-mon[32740]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:50.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:50 smithi194 ceph-mon[32740]: from='client.? 172.21.15.183:0/213598179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:56:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:50 smithi086 ceph-mon[27783]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:50.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:50 smithi086 ceph-mon[27783]: from='client.? 172.21.15.183:0/213598179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:56:52.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:52 smithi183 ceph-mon[32344]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:52 smithi194 ceph-mon[32740]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:52 smithi086 ceph-mon[27783]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:54 smithi183 ceph-mon[32344]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:54 smithi194 ceph-mon[32740]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:54 smithi086 ceph-mon[27783]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:56.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:56 smithi086 ceph-mon[27783]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:56 smithi183 ceph-mon[32344]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:56 smithi194 ceph-mon[32740]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:57.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:57 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:56:57.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:57 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:57.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:57 smithi183 ceph-mon[32344]: Deploying daemon osd.10 on smithi183 2024-04-25T18:56:57.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:57 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:56:57.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:57 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:57.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:57 smithi194 ceph-mon[32740]: Deploying daemon osd.10 on smithi183 2024-04-25T18:56:57.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:57 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:56:57.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:57 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:57.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:57 smithi086 ceph-mon[27783]: Deploying daemon osd.10 on smithi183 2024-04-25T18:56:58.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:56:58 smithi183 ceph-mon[32344]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:56:58 smithi194 ceph-mon[32740]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:56:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:56:58 smithi086 ceph-mon[27783]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:00.624 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:00 smithi183 ceph-mon[32344]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:00 smithi194 ceph-mon[32740]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:00 smithi086 ceph-mon[27783]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:02.670 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:02 smithi183 ceph-mon[32344]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:02 smithi194 ceph-mon[32740]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:02 smithi086 ceph-mon[27783]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:03.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:03.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:03.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:03 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:03.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:03 smithi086 ceph-mon[27783]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:03.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:03.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:03.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:03 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:03.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:03 smithi183 ceph-mon[32344]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:03 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:03 smithi194 ceph-mon[32740]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:06.489 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-04-25T18:57:06.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.486+0000 7f78a2ffd700 1 -- 172.21.15.183:0/2932976642 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f78ac0ff0f0 con 0x7f78980a3c40 2024-04-25T18:57:06.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f78980a3c40 msgr2=0x7f78980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f78980a3c40 0x7f78980a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f78ac19a0f0 tx=0x7f78a40094f0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 msgr2=0x7f78ac1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac1a0190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78a80059d0 tx=0x7f78a80047b0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 shutdown_connections 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f78980a3c40 0x7f78980a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78ac103540 0x7f78ac1a0190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78ac1019a0 0x7f78ac198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 --2- 172.21.15.183:0/2932976642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac102ba0 0x7f78ac199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 >> 172.21.15.183:0/2932976642 conn(0x7f78ac0fcf60 msgr2=0x7f78ac0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:06.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 shutdown_connections 2024-04-25T18:57:06.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-25T18:57:06.489+0000 7f78b3ac8700 1 -- 172.21.15.183:0/2932976642 wait complete. 2024-04-25T18:57:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:06 smithi183 ceph-mon[32344]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:06.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:06.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:06 smithi194 ceph-mon[32740]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:06.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:06.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:06 smithi086 ceph-mon[27783]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:06.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:06.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:06.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.207 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T18:57:07.209 INFO:tasks.cephadm:Deploying osd.11 on smithi194 with /dev/vg_nvme/lv_1... 2024-04-25T18:57:07.210 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:57:07.449 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:07 smithi183 ceph-mon[32344]: from='osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:57:07.449 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:07 smithi183 ceph-mon[32344]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:57:07.450 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.450 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.450 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:07.458 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:57:07.488 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:07 smithi194 ceph-mon[32740]: from='osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:57:07.488 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:07 smithi194 ceph-mon[32740]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:57:07.488 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.489 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.489 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:07 smithi086 ceph-mon[27783]: from='osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:57:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:07 smithi086 ceph-mon[27783]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:57:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:07.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-mon[32344]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-mon[32344]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:57:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-mon[32344]: from='osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:57:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-mon[32344]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T18:57:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-mon[32344]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:57:08.673 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:08 smithi194 ceph-mon[32740]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:08.673 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:08 smithi194 ceph-mon[32740]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:57:08.673 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:08 smithi194 ceph-mon[32740]: from='osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:57:08.673 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:08 smithi194 ceph-mon[32740]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T18:57:08.674 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:08.674 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:08 smithi194 ceph-mon[32740]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:57:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:08 smithi086 ceph-mon[27783]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:08 smithi086 ceph-mon[27783]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:57:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:08 smithi086 ceph-mon[27783]: from='osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:57:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:08 smithi086 ceph-mon[27783]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T18:57:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:08 smithi086 ceph-mon[27783]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-25T18:57:09.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T18:57:08.865+0000 7f624c083700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T18:57:09.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 18:57:08 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T18:57:08.877+0000 7f624366d700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:57:09.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:09 smithi183 ceph-mon[32344]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:57:09.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:09 smithi183 ceph-mon[32344]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:57:09.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:09.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:09 smithi183 ceph-mon[32344]: from='osd.10 ' entity='osd.10' 2024-04-25T18:57:09.581 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:57:09.581 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:57:09.581 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-25T18:57:09.581 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-25T18:57:09.581 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129578 s, 809 MB/s 2024-04-25T18:57:09.581 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-25T18:57:09.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:09 smithi194 ceph-mon[32740]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:57:09.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:09 smithi194 ceph-mon[32740]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:57:09.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:09.648 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:09 smithi194 ceph-mon[32740]: from='osd.10 ' entity='osd.10' 2024-04-25T18:57:09.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:09 smithi086 ceph-mon[27783]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-25T18:57:09.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:09 smithi086 ceph-mon[27783]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:57:09.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:09.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:09 smithi086 ceph-mon[27783]: from='osd.10 ' entity='osd.10' 2024-04-25T18:57:10.302 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_1 2024-04-25T18:57:10.546 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] boot 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T18:57:10.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] boot 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T18:57:10.608 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: osd.10 [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] boot 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T18:57:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:57:11.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:11 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:11 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:11.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:11 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 -- 172.21.15.194:0/3692081549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 msgr2=0x7f043810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 --2- 172.21.15.194:0/3692081549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f043810ac00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0434009990 tx=0x7f0434009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 -- 172.21.15.194:0/3692081549 shutdown_connections 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 --2- 172.21.15.194:0/3692081549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f043810ac00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 --2- 172.21.15.194:0/3692081549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438103530 0x7f0438105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 --2- 172.21.15.194:0/3692081549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438105e60 0x7f0438108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 -- 172.21.15.194:0/3692081549 >> 172.21.15.194:0/3692081549 conn(0x7f04380fcf40 msgr2=0x7f04380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:12.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 -- 172.21.15.194:0/3692081549 shutdown_connections 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.327+0000 7f04405c9700 1 -- 172.21.15.194:0/3692081549 wait complete. 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 Processor -- start 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 -- start start 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438103530 0x7f0438198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438105e60 0x7f04381992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f04381a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f043810e650 con 0x7f0438105e60 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f043810e4d0 con 0x7f0438103530 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f04405c9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f043810e7d0 con 0x7f0438108790 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f043eb66700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f04381a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:12.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f043eb66700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f04381a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:56322/0 (socket says 172.21.15.194:56322) 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f043eb66700 1 -- 172.21.15.194:0/2867359890 learned_addr learned my addr 172.21.15.194:0/2867359890 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.328+0000 7f043e365700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438103530 0x7f0438198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043eb66700 1 -- 172.21.15.194:0/2867359890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438103530 msgr2=0x7f0438198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043eb66700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438103530 0x7f0438198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043eb66700 1 -- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438105e60 msgr2=0x7f04381992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043eb66700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438105e60 0x7f04381992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043eb66700 1 -- 172.21.15.194:0/2867359890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04381a09c0 con 0x7f0438108790 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043e365700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438103530 0x7f0438198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f043eb66700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f04381a0320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0434009f50 tx=0x7f0434009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434026070 con 0x7f0438108790 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0434004810 con 0x7f0438108790 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04381a0c50 con 0x7f0438108790 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434005340 con 0x7f0438108790 2024-04-25T18:57:12.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.329+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a1120 con 0x7f0438108790 2024-04-25T18:57:12.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.330+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043802df80 con 0x7f0438108790 2024-04-25T18:57:12.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.331+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0434029070 con 0x7f0438108790 2024-04-25T18:57:12.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.331+0000 7f042f7fe700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f04240a3c40 0x7f04240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:12.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.332+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f04340d0a40 con 0x7f0438108790 2024-04-25T18:57:12.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.332+0000 7f043e365700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f04240a3c40 0x7f04240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:12.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.333+0000 7f043e365700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f04240a3c40 0x7f04240a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0430007eb0 tx=0x7f04300073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:12.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.336+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f043405dff0 con 0x7f0438108790 2024-04-25T18:57:12.544 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:12.541+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0438062e80 con 0x7f04240a3c40 2024-04-25T18:57:12.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:12 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi183 2024-04-25T18:57:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:12 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-25T18:57:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:12 smithi183 ceph-mon[32344]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:12 smithi183 ceph-mon[32344]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T18:57:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:12 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi183 2024-04-25T18:57:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:12 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-25T18:57:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:12 smithi194 ceph-mon[32740]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:12 smithi194 ceph-mon[32740]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T18:57:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:12 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi183 2024-04-25T18:57:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:12 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-25T18:57:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:12 smithi086 ceph-mon[27783]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:12 smithi086 ceph-mon[27783]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T18:57:13.435 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:57:13.435 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:57:13.435 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:13.435 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:13 smithi194 ceph-mon[32740]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:57:13.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:57:13.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:57:13.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:13.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:13 smithi183 ceph-mon[32344]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:57:13.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:57:13.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:57:13.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:13.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:13 smithi086 ceph-mon[27783]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:57:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:14 smithi194 ceph-mon[32740]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:14 smithi194 ceph-mon[32740]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:14.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:14 smithi194 ceph-mon[32740]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:57:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:14 smithi183 ceph-mon[32344]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:14 smithi183 ceph-mon[32344]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:14 smithi183 ceph-mon[32344]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:57:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:14 smithi086 ceph-mon[27783]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:14 smithi086 ceph-mon[27783]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:14 smithi086 ceph-mon[27783]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:57:15.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:15 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/2483930243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]: dispatch 2024-04-25T18:57:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:15 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]: dispatch 2024-04-25T18:57:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:15 smithi183 ceph-mon[32344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]': finished 2024-04-25T18:57:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:15 smithi183 ceph-mon[32344]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T18:57:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:15.709 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:15 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/2483930243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]: dispatch 2024-04-25T18:57:15.709 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:15 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]: dispatch 2024-04-25T18:57:15.709 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:15 smithi194 ceph-mon[32740]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]': finished 2024-04-25T18:57:15.710 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:15 smithi194 ceph-mon[32740]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T18:57:15.710 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:15.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:15 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/2483930243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]: dispatch 2024-04-25T18:57:15.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:15 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]: dispatch 2024-04-25T18:57:15.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:15 smithi086 ceph-mon[27783]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc1df1f5-bf92-4d45-8c5d-e80fef77cccf"}]': finished 2024-04-25T18:57:15.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:15 smithi086 ceph-mon[27783]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T18:57:15.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:16 smithi194 ceph-mon[32740]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:16 smithi194 ceph-mon[32740]: from='client.? 172.21.15.194:0/1125523087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:57:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:16 smithi086 ceph-mon[27783]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:16 smithi086 ceph-mon[27783]: from='client.? 172.21.15.194:0/1125523087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:57:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:16 smithi183 ceph-mon[32344]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:16 smithi183 ceph-mon[32344]: from='client.? 172.21.15.194:0/1125523087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:57:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:18 smithi194 ceph-mon[32740]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T18:57:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:18 smithi086 ceph-mon[27783]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T18:57:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:18 smithi183 ceph-mon[32344]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T18:57:20.674 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:20 smithi194 ceph-mon[32740]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T18:57:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:20 smithi086 ceph-mon[27783]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T18:57:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:20 smithi183 ceph-mon[32344]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T18:57:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:22 smithi194 ceph-mon[32740]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:57:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:22 smithi086 ceph-mon[27783]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:57:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:22 smithi183 ceph-mon[32344]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:57:23.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:23 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:57:23.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:23 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:23.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:23 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:57:23.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:23 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:23.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:23 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:57:23.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:23 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:24.711 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:24 smithi194 ceph-mon[32740]: Deploying daemon osd.11 on smithi194 2024-04-25T18:57:24.711 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:24 smithi194 ceph-mon[32740]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:57:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:24 smithi086 ceph-mon[27783]: Deploying daemon osd.11 on smithi194 2024-04-25T18:57:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:24 smithi086 ceph-mon[27783]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:57:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:24 smithi183 ceph-mon[32344]: Deploying daemon osd.11 on smithi194 2024-04-25T18:57:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:24 smithi183 ceph-mon[32344]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:57:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:26 smithi194 ceph-mon[32740]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:57:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:26 smithi086 ceph-mon[27783]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:57:26.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:26 smithi183 ceph-mon[32344]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:57:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:28 smithi194 ceph-mon[32740]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T18:57:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:28 smithi086 ceph-mon[27783]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T18:57:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:28 smithi183 ceph-mon[32344]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T18:57:29.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:29.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:29 smithi183 ceph-mon[32344]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:29.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:29.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:29.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:29.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:29 smithi194 ceph-mon[32740]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:30.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:30.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:30.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:30.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:29 smithi086 ceph-mon[27783]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 11 on host 'smithi194' 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.335+0000 7f042f7fe700 1 -- 172.21.15.194:0/2867359890 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0438062e80 con 0x7f04240a3c40 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f04240a3c40 msgr2=0x7f04240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f04240a3c40 0x7f04240a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0430007eb0 tx=0x7f04300073c0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 msgr2=0x7f04381a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f04381a0320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0434009f50 tx=0x7f0434009f80 comp rx=0 tx=0).stop 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 shutdown_connections 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f04240a3c40 0x7f04240a6100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0438108790 0x7f04381a0320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438103530 0x7f0438198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 --2- 172.21.15.194:0/2867359890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438105e60 0x7f04381992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 >> 172.21.15.194:0/2867359890 conn(0x7f04380fcf40 msgr2=0x7f0438103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 shutdown_connections 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:32.337+0000 7f04405c9700 1 -- 172.21.15.194:0/2867359890 wait complete. 2024-04-25T18:57:32.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:32 smithi194 ceph-mon[32740]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:32.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:32.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:32 smithi183 ceph-mon[32344]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:32.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:32.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:32 smithi086 ceph-mon[27783]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:33.075 DEBUG:teuthology.orchestra.run.smithi194:osd.11> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T18:57:33.077 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T18:57:33.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T18:57:33.532 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:33.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:33 smithi086 ceph-mon[27783]: from='osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:57:33.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:33 smithi086 ceph-mon[27783]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:57:33.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:33.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:33.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:33 smithi183 ceph-mon[32344]: from='osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:57:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:33 smithi183 ceph-mon[32344]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:57:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:33 smithi194 ceph-mon[32740]: from='osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:57:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:33 smithi194 ceph-mon[32740]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:57:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:57:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: from='osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:57:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T18:57:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:57:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:34 smithi183 ceph-mon[32344]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:57:34.719 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:34.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:57:34.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: from='osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:57:34.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T18:57:34.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:34.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:57:34.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:34 smithi086 ceph-mon[27783]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: from='osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-25T18:57:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-mon[32740]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-25T18:57:35.067 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T18:57:34.756+0000 7f8be9bc6700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T18:57:35.067 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 18:57:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T18:57:34.771+0000 7f8be11b0700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:57:35.529 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:35 smithi086 ceph-mon[27783]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T18:57:35.529 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:35.529 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:35 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:35.529 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:35 smithi086 ceph-mon[27783]: from='osd.11 ' entity='osd.11' 2024-04-25T18:57:35.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:35 smithi183 ceph-mon[32344]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T18:57:35.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:35.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:35 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:35.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:35 smithi183 ceph-mon[32344]: from='osd.11 ' entity='osd.11' 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 -- 172.21.15.86:0/1773785480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c101320 msgr2=0x7f6c1c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 --2- 172.21.15.86:0/1773785480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c101320 0x7f6c1c101740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009990 tx=0x7f6c0c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 -- 172.21.15.86:0/1773785480 shutdown_connections 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 --2- 172.21.15.86:0/1773785480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c102640 0x7f6c1c102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 --2- 172.21.15.86:0/1773785480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c101320 0x7f6c1c101740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 --2- 172.21.15.86:0/1773785480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101c80 0x7f6c1c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 -- 172.21.15.86:0/1773785480 >> 172.21.15.86:0/1773785480 conn(0x7f6c1c0fcf40 msgr2=0x7f6c1c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 -- 172.21.15.86:0/1773785480 shutdown_connections 2024-04-25T18:57:35.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.696+0000 7f6c20c00700 1 -- 172.21.15.86:0/1773785480 wait complete. 2024-04-25T18:57:35.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 Processor -- start 2024-04-25T18:57:35.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 -- start start 2024-04-25T18:57:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101320 0x7f6c1c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 0x7f6c1c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c102640 0x7f6c1c1a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1106f0 con 0x7f6c1c102640 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c110570 con 0x7f6c1c101320 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.698+0000 7f6c20c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c110870 con 0x7f6c1c101c80 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 0x7f6c1c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101320 0x7f6c1c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 0x7f6c1c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:57842/0 (socket says 172.21.15.86:57842) 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19400700 1 -- 172.21.15.86:0/3407282883 learned_addr learned my addr 172.21.15.86:0/3407282883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c1a800700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c102640 0x7f6c1c1a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19e00700 1 -- 172.21.15.86:0/3407282883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 msgr2=0x7f6c1c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19e00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 0x7f6c1c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19e00700 1 -- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c102640 msgr2=0x7f6c1c1a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19e00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c102640 0x7f6c1c1a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19e00700 1 -- 172.21.15.86:0/3407282883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c1a2b00 con 0x7f6c1c101320 2024-04-25T18:57:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.699+0000 7f6c19400700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 0x7f6c1c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.700+0000 7f6c19e00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101320 0x7f6c1c19aea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000c00 tx=0x7f6c0c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.700+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c026070 con 0x7f6c1c101320 2024-04-25T18:57:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.700+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0c021410 con 0x7f6c1c101320 2024-04-25T18:57:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.700+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c1a2d90 con 0x7f6c1c101320 2024-04-25T18:57:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.700+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c017650 con 0x7f6c1c101320 2024-04-25T18:57:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.700+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1a3190 con 0x7f6c1c101320 2024-04-25T18:57:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.701+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c02df80 con 0x7f6c1c101320 2024-04-25T18:57:35.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.703+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c0c029070 con 0x7f6c1c101320 2024-04-25T18:57:35.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.704+0000 7f6c12a00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c000a3c40 0x7f6c000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:35.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.705+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f6c0c0d0eb0 con 0x7f6c1c101320 2024-04-25T18:57:35.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.705+0000 7f6c19400700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c000a3c40 0x7f6c000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:35.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.705+0000 7f6c19400700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c000a3c40 0x7f6c000a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c08011550 tx=0x7f6c08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:35.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.707+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c0c030080 con 0x7f6c1c101320 2024-04-25T18:57:35.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:35 smithi194 ceph-mon[32740]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T18:57:35.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:35.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:35 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:35.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:35 smithi194 ceph-mon[32740]: from='osd.11 ' entity='osd.11' 2024-04-25T18:57:35.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.941+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6c1c0636f0 con 0x7f6c1c101320 2024-04-25T18:57:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.942+0000 7f6c12a00700 1 -- 172.21.15.86:0/3407282883 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f6c0c013070 con 0x7f6c1c101320 2024-04-25T18:57:35.943 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:57:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.944+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c000a3c40 msgr2=0x7f6c000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.944+0000 7f6c20c00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c000a3c40 0x7f6c000a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c08011550 tx=0x7f6c08009260 comp rx=0 tx=0).stop 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.944+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101320 msgr2=0x7f6c1c19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.944+0000 7f6c20c00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101320 0x7f6c1c19aea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000c00 tx=0x7f6c0c009580 comp rx=0 tx=0).stop 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 shutdown_connections 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c000a3c40 0x7f6c000a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c1c102640 0x7f6c1c1a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c1c101320 0x7f6c1c19aea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 --2- 172.21.15.86:0/3407282883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c1c101c80 0x7f6c1c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 >> 172.21.15.86:0/3407282883 conn(0x7f6c1c0fcf40 msgr2=0x7f6c1c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 shutdown_connections 2024-04-25T18:57:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:35.945+0000 7f6c20c00700 1 -- 172.21.15.86:0/3407282883 wait complete. 2024-04-25T18:57:36.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: purged_snaps scrub starts 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: purged_snaps scrub ok 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] boot 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:36 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3407282883' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: purged_snaps scrub starts 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: purged_snaps scrub ok 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] boot 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T18:57:36.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:36.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:36 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3407282883' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:57:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: purged_snaps scrub starts 2024-04-25T18:57:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: purged_snaps scrub ok 2024-04-25T18:57:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:57:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: osd.11 [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] boot 2024-04-25T18:57:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T18:57:36.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:57:36.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:36 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3407282883' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:57:36.751 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714071455,"num_in_osds":12,"osd_in_since":1714071435,"num_remapped_pgs":0} 2024-04-25T18:57:36.751 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:57:37.193 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:38.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: Detected new or changed devices on smithi194 2024-04-25T18:57:38.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi194 to 4272M 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:38.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: Detected new or changed devices on smithi194 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi194 to 4272M 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:38.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:38.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T18:57:38.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:38.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:38.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: Detected new or changed devices on smithi194 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi194 to 4272M 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:38.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:38.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T18:57:38.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:38.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:38.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:39.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 -- 172.21.15.86:0/3547234924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 msgr2=0x7f7ba41118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 --2- 172.21.15.86:0/3547234924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba41118e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b0a0 tx=0x7f7b9c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 -- 172.21.15.86:0/3547234924 shutdown_connections 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 --2- 172.21.15.86:0/3547234924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba41118e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 --2- 172.21.15.86:0/3547234924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 --2- 172.21.15.86:0/3547234924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 0x7f7ba410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 -- 172.21.15.86:0/3547234924 >> 172.21.15.86:0/3547234924 conn(0x7f7ba40788f0 msgr2=0x7f7ba4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 -- 172.21.15.86:0/3547234924 shutdown_connections 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.364+0000 7f7baac00700 1 -- 172.21.15.86:0/3547234924 wait complete. 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.365+0000 7f7baac00700 1 Processor -- start 2024-04-25T18:57:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.365+0000 7f7baac00700 1 -- start start 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.365+0000 7f7baac00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7baac00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 0x7f7ba419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7baac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7baac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4114e40 con 0x7f7ba410f4f0 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7baac00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4114cc0 con 0x7f7ba406b070 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7baac00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4114fc0 con 0x7f7ba406a750 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba3e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba3e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:57868/0 (socket says 172.21.15.86:57868) 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba3e00700 1 -- 172.21.15.86:0/3351811908 learned_addr learned my addr 172.21.15.86:0/3351811908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:57:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba3400700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 0x7f7ba419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba8c00700 1 -- 172.21.15.86:0/3351811908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 msgr2=0x7f7ba419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba8c00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba8c00700 1 -- 172.21.15.86:0/3351811908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 msgr2=0x7f7ba419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba8c00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 0x7f7ba419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba8c00700 1 -- 172.21.15.86:0/3351811908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba41a0370 con 0x7f7ba410f4f0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba3e00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.366+0000 7f7ba3400700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 0x7f7ba419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.367+0000 7f7ba8c00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba419fc30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b070 tx=0x7f7b9c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.367+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c021070 con 0x7f7ba410f4f0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.367+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b9c007e30 con 0x7f7ba410f4f0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.367+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba41a0600 con 0x7f7ba410f4f0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.367+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0045f0 con 0x7f7ba410f4f0 2024-04-25T18:57:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.367+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba41a7ee0 con 0x7f7ba410f4f0 2024-04-25T18:57:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.369+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba4101f60 con 0x7f7ba410f4f0 2024-04-25T18:57:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.371+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b9c007980 con 0x7f7ba410f4f0 2024-04-25T18:57:39.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.372+0000 7f7ba0c00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7b880a3c40 0x7f7b880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.372+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7b9c0d0d30 con 0x7f7ba410f4f0 2024-04-25T18:57:39.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.372+0000 7f7ba3e00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7b880a3c40 0x7f7b880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.373+0000 7f7ba3e00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7b880a3c40 0x7f7b880a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b94011550 tx=0x7f7b94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.375+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b9c09dba0 con 0x7f7ba410f4f0 2024-04-25T18:57:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.601+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ba402df80 con 0x7f7ba410f4f0 2024-04-25T18:57:39.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.603+0000 7f7ba0c00700 1 -- 172.21.15.86:0/3351811908 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18465 (secure 0 0 0) 0x7f7b9c09d4a0 con 0x7f7ba410f4f0 2024-04-25T18:57:39.605 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:57:39.605 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T18:57:37.373337+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:57:39.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7b880a3c40 msgr2=0x7f7b880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7b880a3c40 0x7f7b880a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b94011550 tx=0x7f7b94009260 comp rx=0 tx=0).stop 2024-04-25T18:57:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 msgr2=0x7f7ba419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba419fc30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b070 tx=0x7f7b9c004c80 comp rx=0 tx=0).stop 2024-04-25T18:57:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 shutdown_connections 2024-04-25T18:57:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7b880a3c40 0x7f7b880a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ba410f4f0 0x7f7ba419fc30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ba406a750 0x7f7ba419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 --2- 172.21.15.86:0/3351811908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406b070 0x7f7ba419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.606+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 >> 172.21.15.86:0/3351811908 conn(0x7f7ba40788f0 msgr2=0x7f7ba41109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.607+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 shutdown_connections 2024-04-25T18:57:39.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:39.607+0000 7f7baac00700 1 -- 172.21.15.86:0/3351811908 wait complete. 2024-04-25T18:57:40.400 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T18:53:19.337576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T18:57:40.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T18:57:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:40 smithi183 ceph-mon[32344]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:40 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3351811908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:57:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:40 smithi194 ceph-mon[32740]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:40 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3351811908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:57:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:40 smithi086 ceph-mon[27783]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:40 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3351811908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:57:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:42.632 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:42 smithi086 ceph-mon[27783]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:42 smithi183 ceph-mon[32344]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:42 smithi194 ceph-mon[32740]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:42.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 -- 172.21.15.86:0/818416253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794103450 msgr2=0x7f779410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 --2- 172.21.15.86:0/818416253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794103450 0x7f779410f8b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f778c00b0a0 tx=0x7f778c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 -- 172.21.15.86:0/818416253 shutdown_connections 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 --2- 172.21.15.86:0/818416253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794103450 0x7f779410f8b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 --2- 172.21.15.86:0/818416253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7794108a50 0x7f7794108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 --2- 172.21.15.86:0/818416253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794102a90 0x7f7794102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 -- 172.21.15.86:0/818416253 >> 172.21.15.86:0/818416253 conn(0x7f7794079ba0 msgr2=0x7f7794079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 -- 172.21.15.86:0/818416253 shutdown_connections 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.914+0000 7f779a400700 1 -- 172.21.15.86:0/818416253 wait complete. 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.915+0000 7f779a400700 1 Processor -- start 2024-04-25T18:57:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.915+0000 7f779a400700 1 -- start start 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f779a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 0x7f779419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f779a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794103450 0x7f779419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f779a400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7794108a50 0x7f779419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f779a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7794112be0 con 0x7f7794102a90 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f779a400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7794112a60 con 0x7f7794103450 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f779a400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7794112d60 con 0x7f7794108a50 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f7793400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 0x7f779419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f7793400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 0x7f779419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47544/0 (socket says 172.21.15.86:47544) 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f7793400700 1 -- 172.21.15.86:0/2108106553 learned_addr learned my addr 172.21.15.86:0/2108106553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:57:42.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f7792a00700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794103450 0x7f779419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.916+0000 7f7793e00700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7794108a50 0x7f779419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7792a00700 1 -- 172.21.15.86:0/2108106553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7794108a50 msgr2=0x7f779419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7792a00700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7794108a50 0x7f779419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7792a00700 1 -- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 msgr2=0x7f779419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7792a00700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 0x7f779419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7792a00700 1 -- 172.21.15.86:0/2108106553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77941a0290 con 0x7f7794103450 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7793400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 0x7f779419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f7792a00700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794103450 0x7f779419b7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f778000e9f0 tx=0x7f778000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778000c9c0 con 0x7f7794103450 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f778000cb60 con 0x7f7794103450 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77941a0580 con 0x7f7794103450 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780018680 con 0x7f7794103450 2024-04-25T18:57:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.917+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77941a7e00 con 0x7f7794103450 2024-04-25T18:57:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.918+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7794069730 con 0x7f7794103450 2024-04-25T18:57:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.921+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7780010430 con 0x7f7794103450 2024-04-25T18:57:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.922+0000 7f778be00700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77780a3c40 0x7f77780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.922+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f77800d0470 con 0x7f7794103450 2024-04-25T18:57:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.922+0000 7f7793400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77780a3c40 0x7f77780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.923+0000 7f7793400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77780a3c40 0x7f77780a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77840103d0 tx=0x7f7784007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:42.924+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f778009d250 con 0x7f7794103450 2024-04-25T18:57:43.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.147+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f779402df80 con 0x7f7794103450 2024-04-25T18:57:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.148+0000 7f778be00700 1 -- 172.21.15.86:0/2108106553 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f778009cb50 con 0x7f7794103450 2024-04-25T18:57:43.150 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77780a3c40 msgr2=0x7f77780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77780a3c40 0x7f77780a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77840103d0 tx=0x7f7784007480 comp rx=0 tx=0).stop 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794103450 msgr2=0x7f779419b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794103450 0x7f779419b7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f778000e9f0 tx=0x7f778000ed00 comp rx=0 tx=0).stop 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 shutdown_connections 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77780a3c40 0x7f77780a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7794108a50 0x7f779419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794102a90 0x7f779419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 --2- 172.21.15.86:0/2108106553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7794103450 0x7f779419b7c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.151+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 >> 172.21.15.86:0/2108106553 conn(0x7f7794079ba0 msgr2=0x7f779410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.152+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 shutdown_connections 2024-04-25T18:57:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:43.152+0000 7f779a400700 1 -- 172.21.15.86:0/2108106553 wait complete. 2024-04-25T18:57:43.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:43 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2108106553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:57:43.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:43 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2108106553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:57:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:43 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2108106553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-04-25T18:57:43.945 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-04-25T18:57:43.946 INFO:tasks.cephadm:Adding mds.c on smithi194 2024-04-25T18:57:43.946 INFO:tasks.cephadm:Adding mds.f on smithi194 2024-04-25T18:57:43.946 INFO:tasks.cephadm:Adding mds.i on smithi194 2024-04-25T18:57:43.946 INFO:tasks.cephadm:Adding mds.l on smithi194 2024-04-25T18:57:43.946 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l' 2024-04-25T18:57:44.192 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:57:44.600 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:44 smithi194 ceph-mon[32740]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:44.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:44 smithi183 ceph-mon[32344]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:44 smithi086 ceph-mon[27783]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:45.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 -- 172.21.15.194:0/4041615851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf401040d0 msgr2=0x7fdf401044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:45.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 --2- 172.21.15.194:0/4041615851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf401040d0 0x7fdf401044f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009990 tx=0x7fdf3c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:45.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 -- 172.21.15.194:0/4041615851 shutdown_connections 2024-04-25T18:57:45.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 --2- 172.21.15.194:0/4041615851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 0x7fdf4010b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:45.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 --2- 172.21.15.194:0/4041615851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf401040d0 0x7fdf401044f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:45.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 --2- 172.21.15.194:0/4041615851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf40100b50 0x7fdf40100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.920+0000 7fdf47397700 1 -- 172.21.15.194:0/4041615851 >> 172.21.15.194:0/4041615851 conn(0x7fdf400fc6f0 msgr2=0x7fdf400feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.921+0000 7fdf47397700 1 -- 172.21.15.194:0/4041615851 shutdown_connections 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.921+0000 7fdf47397700 1 -- 172.21.15.194:0/4041615851 wait complete. 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.921+0000 7fdf47397700 1 Processor -- start 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.921+0000 7fdf47397700 1 -- start start 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.921+0000 7fdf47397700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 0x7fdf40198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 0x7fdf401991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf401040d0 0x7fdf401a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4010e620 con 0x7fdf401040d0 2024-04-25T18:57:45.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4010e4a0 con 0x7fdf40101510 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4010e7a0 con 0x7fdf40100b50 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 0x7fdf40198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 0x7fdf40198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50920/0 (socket says 172.21.15.194:50920) 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 -- 172.21.15.194:0/2659799875 learned_addr learned my addr 172.21.15.194:0/2659799875 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf44932700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 0x7fdf401991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 -- 172.21.15.194:0/2659799875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 msgr2=0x7fdf401991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 0x7fdf401991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 -- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf401040d0 msgr2=0x7fdf401a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf401040d0 0x7fdf401a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 -- 172.21.15.194:0/2659799875 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf401a08f0 con 0x7fdf40100b50 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf44932700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 0x7fdf401991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf45133700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 0x7fdf40198c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c000c00 tx=0x7fdf3c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:45.926 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c027070 con 0x7fdf40100b50 2024-04-25T18:57:45.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3c0041c0 con 0x7fdf40100b50 2024-04-25T18:57:45.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf401a0b80 con 0x7fdf40100b50 2024-04-25T18:57:45.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00f460 con 0x7fdf40100b50 2024-04-25T18:57:45.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.922+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf401a0fd0 con 0x7fdf40100b50 2024-04-25T18:57:45.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.923+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf40069730 con 0x7fdf40100b50 2024-04-25T18:57:45.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.924+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf3c02a070 con 0x7fdf40100b50 2024-04-25T18:57:45.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.925+0000 7fdf367fc700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdf2c0a3c40 0x7fdf2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:45.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.925+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf3c0d0f70 con 0x7fdf40100b50 2024-04-25T18:57:45.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.925+0000 7fdf44932700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdf2c0a3c40 0x7fdf2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:45.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.926+0000 7fdf44932700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdf2c0a3c40 0x7fdf2c0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf4019a180 tx=0x7fdf30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:45.930 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:45.929+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf3c05eff0 con 0x7fdf40100b50 2024-04-25T18:57:46.139 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.138+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}) -- 0x7fdf40063140 con 0x7fdf2c0a3c40 2024-04-25T18:57:46.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.155+0000 7fdf367fc700 1 -- 172.21.15.194:0/2659799875 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdf40063140 con 0x7fdf2c0a3c40 2024-04-25T18:57:46.157 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mds.all update... 2024-04-25T18:57:46.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdf2c0a3c40 msgr2=0x7fdf2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdf2c0a3c40 0x7fdf2c0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf4019a180 tx=0x7fdf30008040 comp rx=0 tx=0).stop 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 msgr2=0x7fdf40198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 0x7fdf40198c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c000c00 tx=0x7fdf3c009580 comp rx=0 tx=0).stop 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 shutdown_connections 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdf2c0a3c40 0x7fdf2c0a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf401040d0 0x7fdf401a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf40100b50 0x7fdf40198c70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 --2- 172.21.15.194:0/2659799875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf40101510 0x7fdf401991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 >> 172.21.15.194:0/2659799875 conn(0x7fdf400fc6f0 msgr2=0x7fdf401071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 shutdown_connections 2024-04-25T18:57:46.160 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-25T18:57:46.158+0000 7fdf47397700 1 -- 172.21.15.194:0/2659799875 wait complete. 2024-04-25T18:57:46.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:46 smithi086 ceph-mon[27783]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:46.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:46.639 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:46 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:46.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:46 smithi183 ceph-mon[32344]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:46 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:46 smithi194 ceph-mon[32740]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:46 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:46.874 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T18:57:46.880 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T18:57:46.925 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T18:57:46.971 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T18:57:47.016 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T18:57:47.019 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T18:57:47.063 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T18:57:47.106 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T18:57:47.149 DEBUG:teuthology.orchestra.run.smithi194:mds.c> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T18:57:47.152 DEBUG:teuthology.orchestra.run.smithi194:mds.f> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T18:57:47.195 DEBUG:teuthology.orchestra.run.smithi194:mds.i> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T18:57:47.238 DEBUG:teuthology.orchestra.run.smithi194:mds.l> sudo journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T18:57:47.281 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T18:57:47.281 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:57:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:47 smithi194 ceph-mon[32740]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:47.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:47 smithi194 ceph-mon[32740]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-04-25T18:57:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:47 smithi183 ceph-mon[32344]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:47.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:47 smithi183 ceph-mon[32344]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-04-25T18:57:47.702 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:47 smithi086 ceph-mon[27783]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:47 smithi086 ceph-mon[27783]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:48 smithi086 ceph-mon[27783]: Deploying daemon mds.l on smithi194 2024-04-25T18:57:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:48 smithi183 ceph-mon[32344]: Deploying daemon mds.l on smithi194 2024-04-25T18:57:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:48 smithi194 ceph-mon[32740]: Deploying daemon mds.l on smithi194 2024-04-25T18:57:49.731 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:49 smithi194 systemd[1]: Starting Ceph mds.l for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 -- 172.21.15.86:0/1637090904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 msgr2=0x7f000c1136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 --2- 172.21.15.86:0/1637090904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1136d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efff8009990 tx=0x7efff8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 -- 172.21.15.86:0/1637090904 shutdown_connections 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 --2- 172.21.15.86:0/1637090904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 0x7f000c116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 --2- 172.21.15.86:0/1637090904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 --2- 172.21.15.86:0/1637090904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1136d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 -- 172.21.15.86:0/1637090904 >> 172.21.15.86:0/1637090904 conn(0x7f000c079ba0 msgr2=0x7f000c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:49.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 -- 172.21.15.86:0/1637090904 shutdown_connections 2024-04-25T18:57:49.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.861+0000 7f0014600700 1 -- 172.21.15.86:0/1637090904 wait complete. 2024-04-25T18:57:49.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.862+0000 7f0014600700 1 Processor -- start 2024-04-25T18:57:49.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.862+0000 7f0014600700 1 -- start start 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.862+0000 7f0014600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.862+0000 7f0014600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0014600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 0x7f000c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0014600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f000c119250 con 0x7f000c075590 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0014600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1190d0 con 0x7f000c113c10 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0014600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1193d0 con 0x7f000c074c70 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c1a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53098/0 (socket says 172.21.15.86:53098) 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 -- 172.21.15.86:0/3056781892 learned_addr learned my addr 172.21.15.86:0/3056781892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011200700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:49.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0012600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 0x7f000c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 -- 172.21.15.86:0/3056781892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 msgr2=0x7f000c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 0x7f000c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 -- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 msgr2=0x7f000c1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011c00700 1 -- 172.21.15.86:0/3056781892 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c19fd30 con 0x7f000c074c70 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0012600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 0x7f000c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.863+0000 7f0011200700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.864+0000 7f0011c00700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c1a5410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f000000e9a0 tx=0x7f000000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.864+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000cbd0 con 0x7f000c074c70 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.864+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000000cd70 con 0x7f000c074c70 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.864+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000018680 con 0x7f000c074c70 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.864+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c1a0020 con 0x7f000c074c70 2024-04-25T18:57:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.864+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c1ac2e0 con 0x7f000c074c70 2024-04-25T18:57:49.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.865+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c069730 con 0x7f000c074c70 2024-04-25T18:57:49.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.867+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0000010970 con 0x7f000c074c70 2024-04-25T18:57:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.868+0000 7f000a000700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efff40a3c40 0x7efff40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.869+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f00000d0320 con 0x7f000c074c70 2024-04-25T18:57:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.869+0000 7f0011200700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efff40a3c40 0x7efff40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.869+0000 7f0011200700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efff40a3c40 0x7efff40a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efff8006010 tx=0x7efff801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:49.871+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f000009d100 con 0x7f000c074c70 2024-04-25T18:57:50.102 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:49 smithi194 podman[62989]: 2024-04-25 18:57:49.72396013 +0000 UTC m=+0.018078708 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:57:50.102 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:49 smithi194 podman[62989]: 2024-04-25 18:57:49.840156471 +0000 UTC m=+0.134275093 container create bcbf93062965d05ab74ec785373f3975bcab9d7ca61fc8eb3e104ce28d5bc41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-25T18:57:50.102 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:50 smithi194 podman[62989]: 2024-04-25 18:57:50.101135177 +0000 UTC m=+0.395253752 container init bcbf93062965d05ab74ec785373f3975bcab9d7ca61fc8eb3e104ce28d5bc41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T18:57:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.144+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f000c1139d0 con 0x7f000c074c70 2024-04-25T18:57:50.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.145+0000 7f000a000700 1 -- 172.21.15.86:0/3056781892 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7f000c1139d0 con 0x7f000c074c70 2024-04-25T18:57:50.147 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:57:50.147 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T18:57:50.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.149+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efff40a3c40 msgr2=0x7efff40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:50.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.149+0000 7f0014600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efff40a3c40 0x7efff40a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efff8006010 tx=0x7efff801a040 comp rx=0 tx=0).stop 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 msgr2=0x7f000c1a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c1a5410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f000000e9a0 tx=0x7f000000ed60 comp rx=0 tx=0).stop 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 shutdown_connections 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efff40a3c40 0x7efff40a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f000c113c10 0x7f000c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f000c074c70 0x7f000c1a5410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 --2- 172.21.15.86:0/3056781892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c075590 0x7f000c1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 >> 172.21.15.86:0/3056781892 conn(0x7f000c079ba0 msgr2=0x7f000c110b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 shutdown_connections 2024-04-25T18:57:50.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:50.150+0000 7f0014600700 1 -- 172.21.15.86:0/3056781892 wait complete. 2024-04-25T18:57:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-04-25T18:57:50.394 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:50 smithi194 podman[62989]: 2024-04-25 18:57:50.104476991 +0000 UTC m=+0.398595567 container start bcbf93062965d05ab74ec785373f3975bcab9d7ca61fc8eb3e104ce28d5bc41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:57:50.395 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mds[63008]: -- 172.21.15.194:0/2946254789 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562369c1a40 con 0x556236995000 2024-04-25T18:57:50.395 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l[63004]: starting mds.l at 2024-04-25T18:57:50.395 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:50 smithi194 bash[62989]: bcbf93062965d05ab74ec785373f3975bcab9d7ca61fc8eb3e104ce28d5bc41f 2024-04-25T18:57:50.395 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 18:57:50 smithi194 systemd[1]: Started Ceph mds.l for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:57:50.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:50.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6832/45783241,v1:172.21.15.194:6833/45783241] up:boot 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6832/45783241,v1:172.21.15.194:6833/45783241] up:boot 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: fsmap 1 up:standby 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3056781892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6832/45783241,v1:172.21.15.194:6833/45783241] up:boot 2024-04-25T18:57:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: fsmap 1 up:standby 2024-04-25T18:57:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:57:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3056781892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:50.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: fsmap 1 up:standby 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3056781892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:50.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:51.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:51 smithi183 ceph-mon[32344]: Deploying daemon mds.k on smithi183 2024-04-25T18:57:51.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:51 smithi194 ceph-mon[32740]: Deploying daemon mds.k on smithi183 2024-04-25T18:57:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:51 smithi086 ceph-mon[27783]: Deploying daemon mds.k on smithi183 2024-04-25T18:57:52.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:52 smithi183 ceph-mon[32344]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:52.667 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:52 smithi183 systemd[1]: Starting Ceph mds.k for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:57:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:52 smithi194 ceph-mon[32740]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:52 smithi086 ceph-mon[27783]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:52.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:57:53.084 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:52 smithi183 podman[62031]: 2024-04-25 18:57:52.688043354 +0000 UTC m=+0.017850200 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:57:53.084 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:52 smithi183 podman[62031]: 2024-04-25 18:57:52.813263684 +0000 UTC m=+0.143070510 container create 99f24bcc8c826775f8d7be606e1d1ed31654b995ca53ddd9ecaa205f446d2762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:57:53.084 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:53 smithi183 podman[62031]: 2024-04-25 18:57:53.082824696 +0000 UTC m=+0.412631522 container init 99f24bcc8c826775f8d7be606e1d1ed31654b995ca53ddd9ecaa205f446d2762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T18:57:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:53.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:53 smithi183 podman[62031]: 2024-04-25 18:57:53.085928576 +0000 UTC m=+0.415735411 container start 99f24bcc8c826775f8d7be606e1d1ed31654b995ca53ddd9ecaa205f446d2762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T18:57:53.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:53 smithi183 ceph-mds[62050]: -- 172.21.15.183:0/1987776820 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff42533a40 con 0x55ff42506c00 2024-04-25T18:57:53.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k[62046]: starting mds.k at 2024-04-25T18:57:53.417 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:53 smithi183 bash[62031]: 99f24bcc8c826775f8d7be606e1d1ed31654b995ca53ddd9ecaa205f446d2762 2024-04-25T18:57:53.417 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 18:57:53 smithi183 systemd[1]: Started Ceph mds.k for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:57:54.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: Deploying daemon mds.j on smithi086 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6832/2135214296,v1:172.21.15.183:6833/2135214296] up:boot 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: fsmap 2 up:standby 2024-04-25T18:57:54.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:54 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:54.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:54.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: Deploying daemon mds.j on smithi086 2024-04-25T18:57:54.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6832/2135214296,v1:172.21.15.183:6833/2135214296] up:boot 2024-04-25T18:57:54.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: fsmap 2 up:standby 2024-04-25T18:57:54.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:54 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:57:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: Deploying daemon mds.j on smithi086 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6832/2135214296,v1:172.21.15.183:6833/2135214296] up:boot 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: fsmap 2 up:standby 2024-04-25T18:57:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:54 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 -- 172.21.15.86:0/1048757798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c10e3b0 msgr2=0x7f854c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 --2- 172.21.15.86:0/1048757798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c10e3b0 0x7f854c115f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f854400b0d0 tx=0x7f854400b3e0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 -- 172.21.15.86:0/1048757798 shutdown_connections 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 --2- 172.21.15.86:0/1048757798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c10e3b0 0x7f854c115f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 --2- 172.21.15.86:0/1048757798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 --2- 172.21.15.86:0/1048757798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f854c109c80 0x7f854c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 -- 172.21.15.86:0/1048757798 >> 172.21.15.86:0/1048757798 conn(0x7f854c076210 msgr2=0x7f854c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 -- 172.21.15.86:0/1048757798 shutdown_connections 2024-04-25T18:57:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.106+0000 7f8553a00700 1 -- 172.21.15.86:0/1048757798 wait complete. 2024-04-25T18:57:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.107+0000 7f8553a00700 1 Processor -- start 2024-04-25T18:57:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 -- start start 2024-04-25T18:57:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c109c80 0x7f854c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f854c10e3b0 0x7f854c1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1192c0 con 0x7f854c109c80 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f854c119140 con 0x7f854c10e3b0 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f8553a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f854c119440 con 0x7f854c1092d0 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.108+0000 7f854be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c109c80 0x7f854c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53114/0 (socket says 172.21.15.86:53114) 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 -- 172.21.15.86:0/2280403602 learned_addr learned my addr 172.21.15.86:0/2280403602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f854c10e3b0 0x7f854c1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 -- 172.21.15.86:0/2280403602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f854c10e3b0 msgr2=0x7f854c1a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f854c10e3b0 0x7f854c1a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 -- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c109c80 msgr2=0x7f854c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c109c80 0x7f854c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 -- 172.21.15.86:0/2280403602 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c1a4760 con 0x7f854c1092d0 2024-04-25T18:57:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f854be00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c109c80 0x7f854c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.109+0000 7f8551000700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c19f750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f854000e9c0 tx=0x7f854000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.110+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854000cbd0 con 0x7f854c1092d0 2024-04-25T18:57:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.110+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c1a4a50 con 0x7f854c1092d0 2024-04-25T18:57:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.110+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854000ee60 con 0x7f854c1092d0 2024-04-25T18:57:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.110+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85400055c0 con 0x7f854c1092d0 2024-04-25T18:57:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.110+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1ac2d0 con 0x7f854c1092d0 2024-04-25T18:57:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.111+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c069730 con 0x7f854c1092d0 2024-04-25T18:57:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.113+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8540004750 con 0x7f854c1092d0 2024-04-25T18:57:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.114+0000 7f8549600700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f85340a3c40 0x7f85340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.114+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f85400d02b0 con 0x7f854c1092d0 2024-04-25T18:57:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.114+0000 7f854be00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f85340a3c40 0x7f85340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.115+0000 7f854be00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f85340a3c40 0x7f85340a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f854c108830 tx=0x7f8538009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:56.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.119+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f854009d090 con 0x7f854c1092d0 2024-04-25T18:57:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.417+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f854c1a0ab0 con 0x7f854c1092d0 2024-04-25T18:57:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.418+0000 7f8549600700 1 -- 172.21.15.86:0/2280403602 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f854009c990 con 0x7f854c1092d0 2024-04-25T18:57:56.419 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:57:56.420 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f85340a3c40 msgr2=0x7f85340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f85340a3c40 0x7f85340a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f854c108830 tx=0x7f8538009260 comp rx=0 tx=0).stop 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 msgr2=0x7f854c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c19f750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f854000e9c0 tx=0x7f854000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 shutdown_connections 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f85340a3c40 0x7f85340a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f854c10e3b0 0x7f854c1a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854c1092d0 0x7f854c19f750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 --2- 172.21.15.86:0/2280403602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f854c109c80 0x7f854c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:56.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 >> 172.21.15.86:0/2280403602 conn(0x7f854c076210 msgr2=0x7f854c076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:56.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.421+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 shutdown_connections 2024-04-25T18:57:56.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:57:56.422+0000 7f8553a00700 1 -- 172.21.15.86:0/2280403602 wait complete. 2024-04-25T18:57:56.424 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-25T18:57:56.552 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:56 smithi086 ceph-mon[27783]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:56 smithi183 ceph-mon[32344]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:56 smithi194 ceph-mon[32740]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:57.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:57 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2280403602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:57.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:57 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2280403602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:57.804 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:57 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2280403602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:58.071 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:57 smithi086 podman[66254]: 2024-04-25 18:57:57.823465281 +0000 UTC m=+0.149044316 container create d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T18:57:58.380 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:58 smithi086 podman[66254]: 2024-04-25 18:57:58.094020705 +0000 UTC m=+0.419599766 container init d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T18:57:58.380 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:58 smithi086 podman[66254]: 2024-04-25 18:57:58.103519696 +0000 UTC m=+0.429098764 container start d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:57:58.381 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:58 smithi086 ceph-mds[66273]: -- 172.21.15.86:0/3358340421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5653105bfa40 con 0x565310592800 2024-04-25T18:57:58.381 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:58 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j[66269]: starting mds.j at 2024-04-25T18:57:58.381 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:58 smithi086 bash[66254]: d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 2024-04-25T18:57:58.381 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 18:57:58 smithi086 systemd[1]: Started Ceph mds.j for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:57:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:58 smithi183 ceph-mon[32344]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:58 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:58 smithi194 ceph-mon[32740]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:58 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:58 smithi086 ceph-mon[27783]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:57:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:58 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:59.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6834/3780909923,v1:172.21.15.86:6835/3780909923] up:boot 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: fsmap 3 up:standby 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:59.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:57:59 smithi183 ceph-mon[32344]: Deploying daemon mds.d on smithi086 2024-04-25T18:57:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6834/3780909923,v1:172.21.15.86:6835/3780909923] up:boot 2024-04-25T18:57:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: fsmap 3 up:standby 2024-04-25T18:57:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:57:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:59.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:59.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:59.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:59.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:57:59 smithi194 ceph-mon[32740]: Deploying daemon mds.d on smithi086 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6834/3780909923,v1:172.21.15.86:6835/3780909923] up:boot 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: fsmap 3 up:standby 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:59.745 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:57:59 smithi086 ceph-mon[27783]: Deploying daemon mds.d on smithi086 2024-04-25T18:58:00.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:00 smithi086 ceph-mon[27783]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:00.622 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:00 smithi183 ceph-mon[32344]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:00 smithi194 ceph-mon[32740]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:01.571 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:01 smithi086 systemd[1]: Starting Ceph mds.d for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:02.321 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 podman[66648]: 2024-04-25 18:58:02.045056477 +0000 UTC m=+0.376975456 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:02.321 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 podman[66648]: 2024-04-25 18:58:02.169810191 +0000 UTC m=+0.501729210 container create 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T18:58:02.643 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:02 smithi086 ceph-mon[27783]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:02 smithi183 ceph-mon[32344]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:02 smithi194 ceph-mon[32740]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:02.896 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 podman[66648]: 2024-04-25 18:58:02.686035871 +0000 UTC m=+1.017954850 container init 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T18:58:02.896 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 podman[66648]: 2024-04-25 18:58:02.695868617 +0000 UTC m=+1.027787560 container start 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T18:58:02.896 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 ceph-mds[66690]: -- 172.21.15.86:0/2517905096 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627aa357a40 con 0x5627aa32ac00 2024-04-25T18:58:02.896 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d[66682]: starting mds.d at 2024-04-25T18:58:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 -- 172.21.15.86:0/3361945265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8113970 msgr2=0x7efdc8115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 --2- 172.21.15.86:0/3361945265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8113970 0x7efdc8115d60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009f90 tx=0x7efdc000b330 comp rx=0 tx=0).stop 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 -- 172.21.15.86:0/3361945265 shutdown_connections 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 --2- 172.21.15.86:0/3361945265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8113970 0x7efdc8115d60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 --2- 172.21.15.86:0/3361945265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8074de0 0x7efdc80751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 --2- 172.21.15.86:0/3361945265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc8113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 -- 172.21.15.86:0/3361945265 >> 172.21.15.86:0/3361945265 conn(0x7efdc80fe450 msgr2=0x7efdc8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 -- 172.21.15.86:0/3361945265 shutdown_connections 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.960+0000 7efdc7200700 1 -- 172.21.15.86:0/3361945265 wait complete. 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.961+0000 7efdc7200700 1 Processor -- start 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 -- start start 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8074de0 0x7efdc819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8113970 0x7efdc81a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8119160 con 0x7efdc8113970 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8118fe0 con 0x7efdc8075700 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.962+0000 7efdc7200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81192e0 con 0x7efdc8074de0 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdc5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8074de0 0x7efdc819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48760/0 (socket says 172.21.15.86:48760) 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdc5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8074de0 0x7efdc819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35652/0 (socket says 172.21.15.86:35652) 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 -- 172.21.15.86:0/1302101249 learned_addr learned my addr 172.21.15.86:0/1302101249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdc6800700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8113970 0x7efdc81a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 -- 172.21.15.86:0/1302101249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8074de0 msgr2=0x7efdc819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8074de0 0x7efdc819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 -- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8113970 msgr2=0x7efdc81a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8113970 0x7efdc81a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.963+0000 7efdbfe00700 1 -- 172.21.15.86:0/1302101249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc81a4690 con 0x7efdc8075700 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.964+0000 7efdbfe00700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc819fc10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e9a0 tx=0x7efdb000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.964+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000cbd0 con 0x7efdc8075700 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.964+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb000cd70 con 0x7efdc8075700 2024-04-25T18:58:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.964+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc81a4980 con 0x7efdc8075700 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.964+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0018680 con 0x7efdc8075700 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.964+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc81ac200 con 0x7efdc8075700 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.966+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc8069730 con 0x7efdc8075700 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.967+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efdb0018820 con 0x7efdc8075700 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.968+0000 7efdbea00700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efdac0a3cc0 0x7efdac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.969+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efdb00d01d0 con 0x7efdc8075700 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.969+0000 7efdc5e00700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efdac0a3cc0 0x7efdac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.969+0000 7efdc5e00700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efdac0a3cc0 0x7efdac0a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009750 tx=0x7efdb8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:02.973+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdb009cfb0 con 0x7efdc8075700 2024-04-25T18:58:03.253 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 bash[66648]: 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 2024-04-25T18:58:03.253 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 18:58:02 smithi086 systemd[1]: Started Ceph mds.d for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.250+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdc802df80 con 0x7efdc8075700 2024-04-25T18:58:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.251+0000 7efdbea00700 1 -- 172.21.15.86:0/1302101249 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7efdb009c8b0 con 0x7efdc8075700 2024-04-25T18:58:03.254 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:03.254 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T18:58:03.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.254+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efdac0a3cc0 msgr2=0x7efdac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efdac0a3cc0 0x7efdac0a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009750 tx=0x7efdb8009310 comp rx=0 tx=0).stop 2024-04-25T18:58:03.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 msgr2=0x7efdc819fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc819fc10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e9a0 tx=0x7efdb000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 shutdown_connections 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efdac0a3cc0 0x7efdac0a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc8113970 0x7efdc81a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc8074de0 0x7efdc819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 --2- 172.21.15.86:0/1302101249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc8075700 0x7efdc819fc10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 >> 172.21.15.86:0/1302101249 conn(0x7efdc80fe450 msgr2=0x7efdc80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 shutdown_connections 2024-04-25T18:58:03.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:03.255+0000 7efdc7200700 1 -- 172.21.15.86:0/1302101249 wait complete. 2024-04-25T18:58:03.259 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-04-25T18:58:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:04.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: Deploying daemon mds.e on smithi183 2024-04-25T18:58:04.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1302101249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:04.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:04.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6836/3883478130,v1:172.21.15.86:6837/3883478130] up:boot 2024-04-25T18:58:04.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: fsmap 4 up:standby 2024-04-25T18:58:04.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:04 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: Deploying daemon mds.e on smithi183 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1302101249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:04.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6836/3883478130,v1:172.21.15.86:6837/3883478130] up:boot 2024-04-25T18:58:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: fsmap 4 up:standby 2024-04-25T18:58:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:04 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:58:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: Deploying daemon mds.e on smithi183 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1302101249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6836/3883478130,v1:172.21.15.86:6837/3883478130] up:boot 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: fsmap 4 up:standby 2024-04-25T18:58:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:04 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:58:05.417 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 systemd[1]: Starting Ceph mds.e for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:05.797 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 podman[62350]: 2024-04-25 18:58:05.424492023 +0000 UTC m=+0.017783803 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:05.797 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 podman[62350]: 2024-04-25 18:58:05.55192623 +0000 UTC m=+0.145218010 container create aa9aefaa080eea85797836029351e47a7e6d1654cbe94ab2b319a5e5dc4af66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T18:58:05.797 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 podman[62350]: 2024-04-25 18:58:05.796779386 +0000 UTC m=+0.390071166 container init aa9aefaa080eea85797836029351e47a7e6d1654cbe94ab2b319a5e5dc4af66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T18:58:06.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:06.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 podman[62350]: 2024-04-25 18:58:05.799653974 +0000 UTC m=+0.392945763 container start aa9aefaa080eea85797836029351e47a7e6d1654cbe94ab2b319a5e5dc4af66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T18:58:06.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 ceph-mds[62370]: -- 172.21.15.183:0/1180641626 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b24ecda40 con 0x563b24ea0c00 2024-04-25T18:58:06.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e[62366]: starting mds.e at 2024-04-25T18:58:06.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 bash[62350]: aa9aefaa080eea85797836029351e47a7e6d1654cbe94ab2b319a5e5dc4af66e 2024-04-25T18:58:06.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 18:58:05 smithi183 systemd[1]: Started Ceph mds.e for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:06.500 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:06.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:06.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:06 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:06.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:06 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:06.693 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:06 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:07.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:07 smithi183 ceph-mon[32344]: Deploying daemon mds.f on smithi194 2024-04-25T18:58:07.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:07 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6834/4174239962,v1:172.21.15.183:6835/4174239962] up:boot 2024-04-25T18:58:07.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:07 smithi183 ceph-mon[32344]: fsmap 5 up:standby 2024-04-25T18:58:07.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:58:07.704 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:07 smithi194 ceph-mon[32740]: Deploying daemon mds.f on smithi194 2024-04-25T18:58:07.704 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:07 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6834/4174239962,v1:172.21.15.183:6835/4174239962] up:boot 2024-04-25T18:58:07.704 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:07 smithi194 ceph-mon[32740]: fsmap 5 up:standby 2024-04-25T18:58:07.704 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:58:07.720 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:07 smithi086 ceph-mon[27783]: Deploying daemon mds.f on smithi194 2024-04-25T18:58:07.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:07 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6834/4174239962,v1:172.21.15.183:6835/4174239962] up:boot 2024-04-25T18:58:07.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:07 smithi086 ceph-mon[27783]: fsmap 5 up:standby 2024-04-25T18:58:07.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:58:08.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:08 smithi194 ceph-mon[32740]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:08.498 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:08 smithi086 ceph-mon[27783]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 -- 172.21.15.86:0/4011900590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 msgr2=0x7ff63010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 --2- 172.21.15.86:0/4011900590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63010f900 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff628009f90 tx=0x7ff62800b330 comp rx=0 tx=0).stop 2024-04-25T18:58:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 -- 172.21.15.86:0/4011900590 shutdown_connections 2024-04-25T18:58:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 --2- 172.21.15.86:0/4011900590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63010f900 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 --2- 172.21.15.86:0/4011900590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff630108a30 0x7ff630108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 --2- 172.21.15.86:0/4011900590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff63006a750 0x7ff63006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 -- 172.21.15.86:0/4011900590 >> 172.21.15.86:0/4011900590 conn(0x7ff6300788f0 msgr2=0x7ff630078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.660+0000 7ff635e00700 1 -- 172.21.15.86:0/4011900590 shutdown_connections 2024-04-25T18:58:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.661+0000 7ff635e00700 1 -- 172.21.15.86:0/4011900590 wait complete. 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 Processor -- start 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 -- start start 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63006a750 0x7ff63019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff630108a30 0x7ff63019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff630112c60 con 0x7ff63006a750 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff630112ae0 con 0x7ff630108a30 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.662+0000 7ff635e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff630112de0 con 0x7ff63006b0f0 2024-04-25T18:58:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff626000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63006a750 0x7ff63019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff626000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35670/0 (socket says 172.21.15.86:35670) 2024-04-25T18:58:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff626000700 1 -- 172.21.15.86:0/1786080940 learned_addr learned my addr 172.21.15.86:0/1786080940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff630108a30 0x7ff63019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 -- 172.21.15.86:0/1786080940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 msgr2=0x7ff63019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 -- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63006a750 msgr2=0x7ff63019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63006a750 0x7ff63019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 -- 172.21.15.86:0/1786080940 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301a0340 con 0x7ff630108a30 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff626000700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62ea00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63006a750 0x7ff63019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.663+0000 7ff62f400700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff630108a30 0x7ff63019fc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff62800b6c0 tx=0x7ff62800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.664+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628021070 con 0x7ff630108a30 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.664+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff628007ac0 con 0x7ff630108a30 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.664+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6301a05d0 con 0x7ff630108a30 2024-04-25T18:58:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.664+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628005040 con 0x7ff630108a30 2024-04-25T18:58:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.664+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301a7eb0 con 0x7ff630108a30 2024-04-25T18:58:08.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:08 smithi183 ceph-mon[32344]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.665+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff630069730 con 0x7ff630108a30 2024-04-25T18:58:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.667+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff628024070 con 0x7ff630108a30 2024-04-25T18:58:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.668+0000 7ff634a00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6140a3cc0 0x7ff6140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.668+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff62800f020 con 0x7ff630108a30 2024-04-25T18:58:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.669+0000 7ff62ea00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6140a3cc0 0x7ff6140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.669+0000 7ff62ea00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6140a3cc0 0x7ff6140a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff620005fd0 tx=0x7ff620009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:08.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.671+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff62802b020 con 0x7ff630108a30 2024-04-25T18:58:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.946+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff63019c690 con 0x7ff630108a30 2024-04-25T18:58:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.947+0000 7ff634a00700 1 -- 172.21.15.86:0/1786080940 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7ff62809e510 con 0x7ff630108a30 2024-04-25T18:58:08.950 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:08.950 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T18:58:08.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6140a3cc0 msgr2=0x7ff6140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6140a3cc0 0x7ff6140a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff620005fd0 tx=0x7ff620009310 comp rx=0 tx=0).stop 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff630108a30 msgr2=0x7ff63019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff630108a30 0x7ff63019fc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff62800b6c0 tx=0x7ff62800cc30 comp rx=0 tx=0).stop 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 shutdown_connections 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff6140a3cc0 0x7ff6140a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff630108a30 0x7ff63019fc00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.950+0000 7ff635e00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff63006a750 0x7ff63019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.951+0000 7ff635e00700 1 --2- 172.21.15.86:0/1786080940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff63006b0f0 0x7ff63019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.951+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 >> 172.21.15.86:0/1786080940 conn(0x7ff6300788f0 msgr2=0x7ff63010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.951+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 shutdown_connections 2024-04-25T18:58:08.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:08.951+0000 7ff635e00700 1 -- 172.21.15.86:0/1786080940 wait complete. 2024-04-25T18:58:08.955 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-04-25T18:58:09.231 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 18:58:08 smithi194 systemd[1]: Started Ceph mds.f for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:09.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1786080940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:09.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:09.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:09.650 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:09.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1786080940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:09.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:09.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:09.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:09.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1786080940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:09.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:09.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:09.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:09.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:10.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:10 smithi183 ceph-mon[32344]: Deploying daemon mds.c on smithi194 2024-04-25T18:58:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:10 smithi183 ceph-mon[32344]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:10 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6834/425079996,v1:172.21.15.194:6835/425079996] up:boot 2024-04-25T18:58:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:10 smithi183 ceph-mon[32344]: fsmap 6 up:standby 2024-04-25T18:58:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:58:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:10 smithi194 ceph-mon[32740]: Deploying daemon mds.c on smithi194 2024-04-25T18:58:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:10 smithi194 ceph-mon[32740]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:10 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6834/425079996,v1:172.21.15.194:6835/425079996] up:boot 2024-04-25T18:58:10.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:10 smithi194 ceph-mon[32740]: fsmap 6 up:standby 2024-04-25T18:58:10.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:58:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:10 smithi086 ceph-mon[27783]: Deploying daemon mds.c on smithi194 2024-04-25T18:58:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:10 smithi086 ceph-mon[27783]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:10 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6834/425079996,v1:172.21.15.194:6835/425079996] up:boot 2024-04-25T18:58:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:10 smithi086 ceph-mon[27783]: fsmap 6 up:standby 2024-04-25T18:58:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:58:11.482 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 systemd[1]: Starting Ceph mds.c for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:11.738 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:11.868 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 podman[63610]: 2024-04-25 18:58:11.429050055 +0000 UTC m=+0.018093613 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:11.868 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 podman[63610]: 2024-04-25 18:58:11.58972607 +0000 UTC m=+0.178769690 container create 73eed52809e28ffffec198b4f69700dcf6fb59f99c004aa4055bd036e6d6e408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T18:58:11.868 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 podman[63610]: 2024-04-25 18:58:11.867237394 +0000 UTC m=+0.456280942 container init 73eed52809e28ffffec198b4f69700dcf6fb59f99c004aa4055bd036e6d6e408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T18:58:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:12.231 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 podman[63610]: 2024-04-25 18:58:11.870050959 +0000 UTC m=+0.459094516 container start 73eed52809e28ffffec198b4f69700dcf6fb59f99c004aa4055bd036e6d6e408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-04-25T18:58:12.231 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 ceph-mds[63629]: -- 172.21.15.194:0/4241837890 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645b0405a40 con 0x5645b03d9000 2024-04-25T18:58:12.231 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c[63625]: starting mds.c at 2024-04-25T18:58:12.232 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 bash[63610]: 73eed52809e28ffffec198b4f69700dcf6fb59f99c004aa4055bd036e6d6e408 2024-04-25T18:58:12.232 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 18:58:11 smithi194 systemd[1]: Started Ceph mds.c for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:12.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:12 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:12.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:12.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.684 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:12.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:12.685 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:12 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:12 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:13.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:13 smithi194 ceph-mon[32740]: Deploying daemon mds.h on smithi183 2024-04-25T18:58:13.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] up:boot 2024-04-25T18:58:13.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:13 smithi194 ceph-mon[32740]: fsmap 7 up:standby 2024-04-25T18:58:13.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:13 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:58:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:13 smithi183 ceph-mon[32344]: Deploying daemon mds.h on smithi183 2024-04-25T18:58:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] up:boot 2024-04-25T18:58:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:13 smithi183 ceph-mon[32344]: fsmap 7 up:standby 2024-04-25T18:58:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:13 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:58:13.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:13 smithi086 ceph-mon[27783]: Deploying daemon mds.h on smithi183 2024-04-25T18:58:13.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] up:boot 2024-04-25T18:58:13.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:13 smithi086 ceph-mon[27783]: fsmap 7 up:standby 2024-04-25T18:58:13.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:13 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:58:14.417 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 systemd[1]: Starting Ceph mds.h for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 -- 172.21.15.86:0/944602971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 msgr2=0x7f4c48107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/944602971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 0x7f4c48107d20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4c380099c0 tx=0x7f4c38009cd0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 -- 172.21.15.86:0/944602971 shutdown_connections 2024-04-25T18:58:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/944602971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/944602971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 0x7f4c48107d20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/944602971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c48102af0 0x7f4c48102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 -- 172.21.15.86:0/944602971 >> 172.21.15.86:0/944602971 conn(0x7f4c48079ba0 msgr2=0x7f4c48079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 -- 172.21.15.86:0/944602971 shutdown_connections 2024-04-25T18:58:14.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.453+0000 7f4c4cc00700 1 -- 172.21.15.86:0/944602971 wait complete. 2024-04-25T18:58:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.454+0000 7f4c4cc00700 1 Processor -- start 2024-04-25T18:58:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 -- start start 2024-04-25T18:58:14.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c48102af0 0x7f4c4819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 0x7f4c4819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48112ca0 con 0x7f4c48107940 2024-04-25T18:58:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48112b20 con 0x7f4c481034b0 2024-04-25T18:58:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c4cc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48112e20 con 0x7f4c48102af0 2024-04-25T18:58:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c45400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c46800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 0x7f4c4819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c45400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:52096/0 (socket says 172.21.15.86:52096) 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c45400700 1 -- 172.21.15.86:0/2060732561 learned_addr learned my addr 172.21.15.86:0/2060732561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.455+0000 7f4c45e00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c48102af0 0x7f4c4819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c45400700 1 -- 172.21.15.86:0/2060732561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c48102af0 msgr2=0x7f4c4819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c45400700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c48102af0 0x7f4c4819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c45400700 1 -- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 msgr2=0x7f4c4819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c45400700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 0x7f4c4819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c45400700 1 -- 172.21.15.86:0/2060732561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c481a0390 con 0x7f4c481034b0 2024-04-25T18:58:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c45400700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4819b8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400e9c0 tx=0x7f4c3400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400cbd0 con 0x7f4c481034b0 2024-04-25T18:58:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c481a0680 con 0x7f4c481034b0 2024-04-25T18:58:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3400ee60 con 0x7f4c481034b0 2024-04-25T18:58:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.456+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c340055c0 con 0x7f4c481034b0 2024-04-25T18:58:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.457+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c481a7f00 con 0x7f4c481034b0 2024-04-25T18:58:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.458+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c48069730 con 0x7f4c481034b0 2024-04-25T18:58:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.459+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4c34004750 con 0x7f4c481034b0 2024-04-25T18:58:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.460+0000 7f4c3ea00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c2c0a3c40 0x7f4c2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.461+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4c340d02b0 con 0x7f4c481034b0 2024-04-25T18:58:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.461+0000 7f4c45e00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c2c0a3c40 0x7f4c2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.461+0000 7f4c45e00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c2c0a3c40 0x7f4c2c0a6100 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c38003760 tx=0x7f4c38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.464+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c3409d090 con 0x7f4c481034b0 2024-04-25T18:58:14.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:14 smithi183 ceph-mon[32344]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:14.685 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 podman[62675]: 2024-04-25 18:58:14.409787677 +0000 UTC m=+0.017980241 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:14.685 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 podman[62675]: 2024-04-25 18:58:14.550021802 +0000 UTC m=+0.158214358 container create 877b444a2ecd639068269df37da812ce3d50821c8a6f14d579ad458c53e81824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:58:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:14 smithi194 ceph-mon[32740]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:14.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.735+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c481a80d0 con 0x7f4c481034b0 2024-04-25T18:58:14.736 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:14 smithi086 ceph-mon[27783]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:14.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.735+0000 7f4c3ea00700 1 -- 172.21.15.86:0/2060732561 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f4c3409c990 con 0x7f4c481034b0 2024-04-25T18:58:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:14.737 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T18:58:14.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c2c0a3c40 msgr2=0x7f4c2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c2c0a3c40 0x7f4c2c0a6100 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c38003760 tx=0x7f4c38005fb0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 msgr2=0x7f4c4819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4819b8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400e9c0 tx=0x7f4c3400ecd0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 shutdown_connections 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c2c0a3c40 0x7f4c2c0a6100 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c48107940 0x7f4c4819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c48102af0 0x7f4c4819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 --2- 172.21.15.86:0/2060732561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c481034b0 0x7f4c4819b8c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 >> 172.21.15.86:0/2060732561 conn(0x7f4c48079ba0 msgr2=0x7f4c48100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.739+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 shutdown_connections 2024-04-25T18:58:14.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:14.740+0000 7f4c4cc00700 1 -- 172.21.15.86:0/2060732561 wait complete. 2024-04-25T18:58:14.742 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 8 2024-04-25T18:58:15.167 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 podman[62675]: 2024-04-25 18:58:14.819710112 +0000 UTC m=+0.427902667 container init 877b444a2ecd639068269df37da812ce3d50821c8a6f14d579ad458c53e81824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:58:15.167 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 podman[62675]: 2024-04-25 18:58:14.822428153 +0000 UTC m=+0.430620710 container start 877b444a2ecd639068269df37da812ce3d50821c8a6f14d579ad458c53e81824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T18:58:15.167 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 ceph-mds[62694]: -- 172.21.15.183:0/1221516910 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab4d8bda40 con 0x55ab4d890800 2024-04-25T18:58:15.167 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h[62690]: starting mds.h at 2024-04-25T18:58:15.167 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 bash[62675]: 877b444a2ecd639068269df37da812ce3d50821c8a6f14d579ad458c53e81824 2024-04-25T18:58:15.167 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 18:58:14 smithi183 systemd[1]: Started Ceph mds.h for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:15.730 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2060732561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:15.730 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.730 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.730 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.730 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:15.730 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:15.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:15 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:15.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2060732561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:15.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:15.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:15.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:15 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:15.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2060732561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:15 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:16 smithi194 ceph-mon[32740]: Deploying daemon mds.i on smithi194 2024-04-25T18:58:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:16 smithi194 ceph-mon[32740]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:16 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] up:boot 2024-04-25T18:58:16.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:16 smithi194 ceph-mon[32740]: fsmap 8 up:standby 2024-04-25T18:58:16.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:16 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:58:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:16 smithi086 ceph-mon[27783]: Deploying daemon mds.i on smithi194 2024-04-25T18:58:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:16 smithi086 ceph-mon[27783]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:16 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] up:boot 2024-04-25T18:58:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:16 smithi086 ceph-mon[27783]: fsmap 8 up:standby 2024-04-25T18:58:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:16 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:58:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:16 smithi183 ceph-mon[32344]: Deploying daemon mds.i on smithi194 2024-04-25T18:58:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:16 smithi183 ceph-mon[32344]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:16 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] up:boot 2024-04-25T18:58:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:16 smithi183 ceph-mon[32344]: fsmap 8 up:standby 2024-04-25T18:58:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:16 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:58:17.481 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 systemd[1]: Starting Ceph mds.i for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:17.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:17.936 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 podman[63917]: 2024-04-25 18:58:17.532744614 +0000 UTC m=+0.017926457 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:17.937 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 podman[63917]: 2024-04-25 18:58:17.650153582 +0000 UTC m=+0.135335418 container create 68818f808e5f733eac275a9ea466b65d85f20db291b63a63422c29cc8aea15bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T18:58:17.937 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 podman[63917]: 2024-04-25 18:58:17.936247983 +0000 UTC m=+0.421429824 container init 68818f808e5f733eac275a9ea466b65d85f20db291b63a63422c29cc8aea15bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T18:58:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:18.231 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 podman[63917]: 2024-04-25 18:58:17.938947403 +0000 UTC m=+0.424129240 container start 68818f808e5f733eac275a9ea466b65d85f20db291b63a63422c29cc8aea15bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:58:18.231 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 ceph-mds[63936]: -- 172.21.15.194:0/2141811713 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a96202fa40 con 0x55a962002800 2024-04-25T18:58:18.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i[63932]: starting mds.i at 2024-04-25T18:58:18.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:18 smithi194 bash[63917]: 68818f808e5f733eac275a9ea466b65d85f20db291b63a63422c29cc8aea15bc 2024-04-25T18:58:18.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 18:58:18 smithi194 systemd[1]: Started Ceph mds.i for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:18.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:18.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:18 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:18.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:18.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:18 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:18 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:19 smithi194 ceph-mon[32740]: Deploying daemon mds.g on smithi086 2024-04-25T18:58:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:19 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] up:boot 2024-04-25T18:58:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:19 smithi194 ceph-mon[32740]: fsmap 9 up:standby 2024-04-25T18:58:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:19 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:58:19.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:19 smithi086 ceph-mon[27783]: Deploying daemon mds.g on smithi086 2024-04-25T18:58:19.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:19 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] up:boot 2024-04-25T18:58:19.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:19 smithi086 ceph-mon[27783]: fsmap 9 up:standby 2024-04-25T18:58:19.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:19 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:58:19.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:19 smithi183 ceph-mon[32344]: Deploying daemon mds.g on smithi086 2024-04-25T18:58:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:19 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] up:boot 2024-04-25T18:58:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:19 smithi183 ceph-mon[32344]: fsmap 9 up:standby 2024-04-25T18:58:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:19 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:58:20.632 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:20 smithi194 ceph-mon[32740]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:20 smithi086 ceph-mon[27783]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:20 smithi183 ceph-mon[32344]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 -- 172.21.15.86:0/3973648690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0103450 msgr2=0x7fd7c010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3973648690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0103450 0x7fd7c010f8b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800b0a0 tx=0x7fd7b800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 -- 172.21.15.86:0/3973648690 shutdown_connections 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3973648690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0103450 0x7fd7c010f8b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3973648690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c0108a50 0x7fd7c0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3973648690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 -- 172.21.15.86:0/3973648690 >> 172.21.15.86:0/3973648690 conn(0x7fd7c0079ba0 msgr2=0x7fd7c0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:21.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 -- 172.21.15.86:0/3973648690 shutdown_connections 2024-04-25T18:58:21.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.350+0000 7fd7c7400700 1 -- 172.21.15.86:0/3973648690 wait complete. 2024-04-25T18:58:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.351+0000 7fd7c7400700 1 Processor -- start 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.351+0000 7fd7c7400700 1 -- start start 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c7400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c0103450 0x7fd7c0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c7400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0108a50 0x7fd7c00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c7400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0112c70 con 0x7fd7c0103450 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c7400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0112af0 con 0x7fd7c0108a50 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c7400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0112df0 con 0x7fd7c0102a90 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:57446/0 (socket says 172.21.15.86:57446) 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 -- 172.21.15.86:0/3023838341 learned_addr learned my addr 172.21.15.86:0/3023838341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c5400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0108a50 0x7fd7c00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7bfe00700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c0103450 0x7fd7c0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 -- 172.21.15.86:0/3023838341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0108a50 msgr2=0x7fd7c00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0108a50 0x7fd7c00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 -- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c0103450 msgr2=0x7fd7c0075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c0103450 0x7fd7c0075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c4a00700 1 -- 172.21.15.86:0/3023838341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c01a7650 con 0x7fd7c0102a90 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.352+0000 7fd7c5400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0108a50 0x7fd7c00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.353+0000 7fd7c4a00700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0074d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e9a0 tx=0x7fd7b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.353+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400cbd0 con 0x7fd7c0102a90 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.353+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b400cd70 con 0x7fd7c0102a90 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.353+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01a7940 con 0x7fd7c0102a90 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.353+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4018680 con 0x7fd7c0102a90 2024-04-25T18:58:21.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.353+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01a7ed0 con 0x7fd7c0102a90 2024-04-25T18:58:21.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.354+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c0069730 con 0x7fd7c0102a90 2024-04-25T18:58:21.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.356+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd7b4010430 con 0x7fd7c0102a90 2024-04-25T18:58:21.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.358+0000 7fd7bd600700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd7a80a3cc0 0x7fd7a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.358+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd7b40d05f0 con 0x7fd7c0102a90 2024-04-25T18:58:21.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.358+0000 7fd7bfe00700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd7a80a3cc0 0x7fd7a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.359+0000 7fd7bfe00700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd7a80a3cc0 0x7fd7a80a6180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7c007a210 tx=0x7fd7ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:21.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.360+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7b409d3d0 con 0x7fd7c0102a90 2024-04-25T18:58:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.693+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7c002df80 con 0x7fd7c0102a90 2024-04-25T18:58:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.694+0000 7fd7bd600700 1 -- 172.21.15.86:0/3023838341 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fd7b409ccd0 con 0x7fd7c0102a90 2024-04-25T18:58:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:21.696 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T18:58:21.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd7a80a3cc0 msgr2=0x7fd7a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd7a80a3cc0 0x7fd7a80a6180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7c007a210 tx=0x7fd7ac009260 comp rx=0 tx=0).stop 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 msgr2=0x7fd7c0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0074d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e9a0 tx=0x7fd7b400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 shutdown_connections 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd7a80a3cc0 0x7fd7a80a6180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7c0108a50 0x7fd7c00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7c0102a90 0x7fd7c0074d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 --2- 172.21.15.86:0/3023838341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7c0103450 0x7fd7c0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 >> 172.21.15.86:0/3023838341 conn(0x7fd7c0079ba0 msgr2=0x7fd7c01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 shutdown_connections 2024-04-25T18:58:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:21.697+0000 7fd7c7400700 1 -- 172.21.15.86:0/3023838341 wait complete. 2024-04-25T18:58:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-04-25T18:58:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:22 smithi194 ceph-mon[32740]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:22 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3023838341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:22.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:22 smithi086 ceph-mon[27783]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:22.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:22 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3023838341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:22 smithi183 ceph-mon[32344]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:22 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3023838341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:23.071 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:22 smithi086 podman[67498]: 2024-04-25 18:58:22.8684362 +0000 UTC m=+0.182375123 container create 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:58:23.574 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:23 smithi086 podman[67498]: 2024-04-25 18:58:23.122429967 +0000 UTC m=+0.436368897 container init 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T18:58:23.574 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:23 smithi086 podman[67498]: 2024-04-25 18:58:23.133576982 +0000 UTC m=+0.447515921 container start 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T18:58:23.574 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:23 smithi086 ceph-mds[67517]: -- 172.21.15.86:0/3315697020 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c569b9a40 con 0x564c5698c800 2024-04-25T18:58:23.574 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g[67513]: starting mds.g at 2024-04-25T18:58:23.574 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:23 smithi086 bash[67498]: 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 2024-04-25T18:58:23.574 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 18:58:23 smithi086 systemd[1]: Started Ceph mds.g for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:24.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:24.624 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: Deploying daemon mds.a on smithi086 2024-04-25T18:58:24.625 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] up:boot 2024-04-25T18:58:24.625 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: fsmap 10 up:standby 2024-04-25T18:58:24.625 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:24 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:58:24.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: Deploying daemon mds.a on smithi086 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] up:boot 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: fsmap 10 up:standby 2024-04-25T18:58:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:24 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:58:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: Deploying daemon mds.a on smithi086 2024-04-25T18:58:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] up:boot 2024-04-25T18:58:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: fsmap 10 up:standby 2024-04-25T18:58:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:24 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:58:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:26.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:26 smithi086 ceph-mon[27783]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:26.571 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:26 smithi086 systemd[1]: Starting Ceph mds.a for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:26.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:26 smithi183 ceph-mon[32344]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:26 smithi194 ceph-mon[32740]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:27.521 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:27 smithi086 podman[67900]: 2024-04-25 18:58:27.178107409 +0000 UTC m=+0.036741194 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:27.820 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:27 smithi086 podman[67900]: 2024-04-25 18:58:27.667731612 +0000 UTC m=+0.526365420 container create 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 -- 172.21.15.86:0/1114941374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c102ac0 msgr2=0x7f809c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 --2- 172.21.15.86:0/1114941374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c102ac0 0x7f809c102f40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8088009990 tx=0x7f8088009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 -- 172.21.15.86:0/1114941374 shutdown_connections 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 --2- 172.21.15.86:0/1114941374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c103480 0x7f809c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 --2- 172.21.15.86:0/1114941374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809c108ab0 0x7f809c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 --2- 172.21.15.86:0/1114941374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c102ac0 0x7f809c102f40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 -- 172.21.15.86:0/1114941374 >> 172.21.15.86:0/1114941374 conn(0x7f809c079bf0 msgr2=0x7f809c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 -- 172.21.15.86:0/1114941374 shutdown_connections 2024-04-25T18:58:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.935+0000 7f80a3a00700 1 -- 172.21.15.86:0/1114941374 wait complete. 2024-04-25T18:58:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.936+0000 7f80a3a00700 1 Processor -- start 2024-04-25T18:58:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.936+0000 7f80a3a00700 1 -- start start 2024-04-25T18:58:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a3a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c102ac0 0x7f809c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a3a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809c103480 0x7f809c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a3a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 0x7f809c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a3a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f809c112cf0 con 0x7f809c103480 2024-04-25T18:58:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a3a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f809c112b70 con 0x7f809c108ab0 2024-04-25T18:58:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a3a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f809c112e70 con 0x7f809c102ac0 2024-04-25T18:58:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a1a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 0x7f809c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f809be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809c103480 0x7f809c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a1a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 0x7f809c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58304/0 (socket says 172.21.15.86:58304) 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.937+0000 7f80a1a00700 1 -- 172.21.15.86:0/2951153034 learned_addr learned my addr 172.21.15.86:0/2951153034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1000700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c102ac0 0x7f809c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1a00700 1 -- 172.21.15.86:0/2951153034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c102ac0 msgr2=0x7f809c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c102ac0 0x7f809c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1a00700 1 -- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809c103480 msgr2=0x7f809c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809c103480 0x7f809c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1a00700 1 -- 172.21.15.86:0/2951153034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c1a0490 con 0x7f809c108ab0 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1000700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c102ac0 0x7f809c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f80a1a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 0x7f809c19fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f809400ef30 tx=0x7f809400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.938+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940092e0 con 0x7f809c108ab0 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.939+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8094004bb0 con 0x7f809c108ab0 2024-04-25T18:58:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.939+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c1a0720 con 0x7f809c108ab0 2024-04-25T18:58:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.939+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094004280 con 0x7f809c108ab0 2024-04-25T18:58:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.939+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c1a8000 con 0x7f809c108ab0 2024-04-25T18:58:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.940+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809c069730 con 0x7f809c108ab0 2024-04-25T18:58:27.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.942+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f809401b030 con 0x7f809c108ab0 2024-04-25T18:58:27.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.943+0000 7f8099600700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80840a3cc0 0x7f80840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:27.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.943+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f80940cfac0 con 0x7f809c108ab0 2024-04-25T18:58:27.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.943+0000 7f80a1000700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80840a3cc0 0x7f80840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:27.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.944+0000 7f80a1000700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80840a3cc0 0x7f80840a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f80900099d0 tx=0x7f8090008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:27.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:27.947+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8094022020 con 0x7f809c108ab0 2024-04-25T18:58:28.158 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:28 smithi086 podman[67900]: 2024-04-25 18:58:28.147946291 +0000 UTC m=+1.006580105 container init 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T18:58:28.159 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:28 smithi086 podman[67900]: 2024-04-25 18:58:28.157282568 +0000 UTC m=+1.015916383 container start 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T18:58:28.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.231+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f809c1a81d0 con 0x7f809c108ab0 2024-04-25T18:58:28.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.232+0000 7f8099600700 1 -- 172.21.15.86:0/2951153034 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f8094010040 con 0x7f809c108ab0 2024-04-25T18:58:28.234 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:28.234 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24509,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-25T18:58:28.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80840a3cc0 msgr2=0x7f80840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80840a3cc0 0x7f80840a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f80900099d0 tx=0x7f8090008040 comp rx=0 tx=0).stop 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 msgr2=0x7f809c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 0x7f809c19fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f809400ef30 tx=0x7f809400c3a0 comp rx=0 tx=0).stop 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 shutdown_connections 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80840a3cc0 0x7f80840a6180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f809c108ab0 0x7f809c19fd50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c102ac0 0x7f809c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 --2- 172.21.15.86:0/2951153034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809c103480 0x7f809c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 >> 172.21.15.86:0/2951153034 conn(0x7f809c079bf0 msgr2=0x7f809c10dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 shutdown_connections 2024-04-25T18:58:28.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:28.235+0000 7f80a3a00700 1 -- 172.21.15.86:0/2951153034 wait complete. 2024-04-25T18:58:28.238 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 11 2024-04-25T18:58:28.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:28 smithi086 ceph-mon[27783]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:28.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:28 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2951153034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:28.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:28 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:28.571 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:28 smithi086 ceph-mds[67959]: -- 172.21.15.86:0/966409954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ec00fba40 con 0x558ec00ce800 2024-04-25T18:58:28.571 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a[67955]: starting mds.a at 2024-04-25T18:58:28.571 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:28 smithi086 bash[67900]: 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b 2024-04-25T18:58:28.571 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 18:58:28 smithi086 systemd[1]: Started Ceph mds.a for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:28.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:28 smithi183 ceph-mon[32344]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:28.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:28 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2951153034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:28.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:28 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:28 smithi194 ceph-mon[32740]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:28 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2951153034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:28 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6840/2332559651,v1:172.21.15.86:6841/2332559651] up:boot 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: fsmap 11 up:standby 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:29.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:29 smithi183 ceph-mon[32344]: Deploying daemon mds.b on smithi183 2024-04-25T18:58:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6840/2332559651,v1:172.21.15.86:6841/2332559651] up:boot 2024-04-25T18:58:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: fsmap 11 up:standby 2024-04-25T18:58:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:58:29.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:29.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:29.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:29.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:29 smithi194 ceph-mon[32740]: Deploying daemon mds.b on smithi183 2024-04-25T18:58:29.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6840/2332559651,v1:172.21.15.86:6841/2332559651] up:boot 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: fsmap 11 up:standby 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:29 smithi086 ceph-mon[27783]: Deploying daemon mds.b on smithi183 2024-04-25T18:58:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:30 smithi183 ceph-mon[32344]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:30.667 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:30 smithi183 systemd[1]: Starting Ceph mds.b for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T18:58:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:30 smithi194 ceph-mon[32740]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:30 smithi086 ceph-mon[27783]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:31.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:31.041 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:30 smithi183 podman[62987]: 2024-04-25 18:58:30.674378889 +0000 UTC m=+0.018643512 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:58:31.041 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:30 smithi183 podman[62987]: 2024-04-25 18:58:30.796412803 +0000 UTC m=+0.140677395 container create 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:58:31.041 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:31 smithi183 podman[62987]: 2024-04-25 18:58:31.040729537 +0000 UTC m=+0.384994129 container init 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T18:58:31.416 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:31 smithi183 podman[62987]: 2024-04-25 18:58:31.043406597 +0000 UTC m=+0.387671198 container start 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:58:31.417 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:31 smithi183 ceph-mds[63005]: -- 172.21.15.183:0/2989804610 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f31d9e3a40 con 0x55f31d9b7000 2024-04-25T18:58:31.417 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: starting mds.b at 2024-04-25T18:58:31.417 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:31 smithi183 bash[62987]: 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be 2024-04-25T18:58:31.417 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:31 smithi183 systemd[1]: Started Ceph mds.b for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T18:58:31.501 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] up:boot 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: fsmap 12 up:standby 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:58:32.470 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:32 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] up:boot 2024-04-25T18:58:32.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: fsmap 12 up:standby 2024-04-25T18:58:32.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:58:32.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:32 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] up:boot 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: fsmap 12 up:standby 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:58:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:32 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:33.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:33 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:33 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:33 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 -- 172.21.15.86:0/2961477152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006a750 msgr2=0x7efee006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 --2- 172.21.15.86:0/2961477152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006a750 0x7efee006abb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efecc009990 tx=0x7efecc009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 -- 172.21.15.86:0/2961477152 shutdown_connections 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 --2- 172.21.15.86:0/2961477152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee006b0f0 0x7efee010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 --2- 172.21.15.86:0/2961477152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee0108a30 0x7efee0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 --2- 172.21.15.86:0/2961477152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006a750 0x7efee006abb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 -- 172.21.15.86:0/2961477152 >> 172.21.15.86:0/2961477152 conn(0x7efee00788f0 msgr2=0x7efee0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 -- 172.21.15.86:0/2961477152 shutdown_connections 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.790+0000 7efee5e00700 1 -- 172.21.15.86:0/2961477152 wait complete. 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.791+0000 7efee5e00700 1 Processor -- start 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.791+0000 7efee5e00700 1 -- start start 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efee5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee006a750 0x7efee0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efee5e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 0x7efee01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efee5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee0108a30 0x7efee019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efee5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112c60 con 0x7efee006a750 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efee5e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112ae0 con 0x7efee006b0f0 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efee5e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112de0 con 0x7efee0108a30 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 0x7efee01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efedea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee006a750 0x7efee0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 0x7efee01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:33578/0 (socket says 172.21.15.86:33578) 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 -- 172.21.15.86:0/2311889537 learned_addr learned my addr 172.21.15.86:0/2311889537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efedf400700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee0108a30 0x7efee019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 -- 172.21.15.86:0/2311889537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee0108a30 msgr2=0x7efee019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee0108a30 0x7efee019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 -- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee006a750 msgr2=0x7efee0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee006a750 0x7efee0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efede000700 1 -- 172.21.15.86:0/2311889537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee019e180 con 0x7efee006b0f0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.792+0000 7efedea00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee006a750 0x7efee0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.793+0000 7efede000700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 0x7efee01996b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efecc005910 tx=0x7efecc00b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.793+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc027070 con 0x7efee006b0f0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.793+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee019e410 con 0x7efee006b0f0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.793+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efecc022470 con 0x7efee006b0f0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.793+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc017650 con 0x7efee006b0f0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.793+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a5cf0 con 0x7efee006b0f0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.794+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0069730 con 0x7efee006b0f0 2024-04-25T18:58:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.796+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efecc02a070 con 0x7efee006b0f0 2024-04-25T18:58:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.798+0000 7efee4a00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efec40a3cc0 0x7efec40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.798+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efecc0d0c70 con 0x7efee006b0f0 2024-04-25T18:58:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.798+0000 7efedea00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efec40a3cc0 0x7efec40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.799+0000 7efedea00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efec40a3cc0 0x7efec40a6180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efed0009fd0 tx=0x7efed0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:33.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:33.806+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efecc013070 con 0x7efee006b0f0 2024-04-25T18:58:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.086+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee019a4d0 con 0x7efee006b0f0 2024-04-25T18:58:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.087+0000 7efee4a00700 1 -- 172.21.15.86:0/2311889537 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7efecc09dae0 con 0x7efee006b0f0 2024-04-25T18:58:34.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:34.089 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24509,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T18:58:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efec40a3cc0 msgr2=0x7efec40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efec40a3cc0 0x7efec40a6180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efed0009fd0 tx=0x7efed0009260 comp rx=0 tx=0).stop 2024-04-25T18:58:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 msgr2=0x7efee01996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 0x7efee01996b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efecc005910 tx=0x7efecc00b670 comp rx=0 tx=0).stop 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 shutdown_connections 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efec40a3cc0 0x7efec40a6180 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee0108a30 0x7efee019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee006a750 0x7efee0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 --2- 172.21.15.86:0/2311889537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efee006b0f0 0x7efee01996b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 >> 172.21.15.86:0/2311889537 conn(0x7efee00788f0 msgr2=0x7efee00ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 shutdown_connections 2024-04-25T18:58:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:34.090+0000 7efee5e00700 1 -- 172.21.15.86:0/2311889537 wait complete. 2024-04-25T18:58:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-04-25T18:58:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:34 smithi183 ceph-mon[32344]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:34.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:34 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2311889537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:34.703 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:34 smithi086 ceph-mon[27783]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:34.703 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:34 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2311889537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:34 smithi194 ceph-mon[32740]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:34 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2311889537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:35.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-25T18:58:35.710 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:36.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:36 smithi086 ceph-mon[27783]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:36.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:36 smithi183 ceph-mon[32344]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:36 smithi194 ceph-mon[32740]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:37.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 -- 172.21.15.86:0/1681357110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 msgr2=0x7f44ec10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 --2- 172.21.15.86:0/1681357110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 0x7f44ec10ef00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009990 tx=0x7f44d8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 -- 172.21.15.86:0/1681357110 shutdown_connections 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 --2- 172.21.15.86:0/1681357110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 0x7f44ec111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 --2- 172.21.15.86:0/1681357110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 --2- 172.21.15.86:0/1681357110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 0x7f44ec10ef00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 -- 172.21.15.86:0/1681357110 >> 172.21.15.86:0/1681357110 conn(0x7f44ec079ba0 msgr2=0x7f44ec079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 -- 172.21.15.86:0/1681357110 shutdown_connections 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.824+0000 7f44f2000700 1 -- 172.21.15.86:0/1681357110 wait complete. 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.825+0000 7f44f2000700 1 Processor -- start 2024-04-25T18:58:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.825+0000 7f44f2000700 1 -- start start 2024-04-25T18:58:37.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44f2000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44f2000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 0x7f44ec19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44f2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 0x7f44ec19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44f2000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec114e50 con 0x7f44ec10f440 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44f2000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec114cd0 con 0x7f44ec074c70 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44f2000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec114fd0 con 0x7f44ec075590 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:33608/0 (socket says 172.21.15.86:33608) 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 -- 172.21.15.86:0/1153676897 learned_addr learned my addr 172.21.15.86:0/1153676897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eaa00700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 0x7f44ec19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44ebe00700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 0x7f44ec19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 -- 172.21.15.86:0/1153676897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 msgr2=0x7f44ec19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 0x7f44ec19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 -- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 msgr2=0x7f44ec19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 0x7f44ec19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44eb400700 1 -- 172.21.15.86:0/1153676897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec1a0300 con 0x7f44ec074c70 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.826+0000 7f44ebe00700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 0x7f44ec19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.827+0000 7f44eb400700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec19b340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e9a0 tx=0x7f44dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.827+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00cbd0 con 0x7f44ec074c70 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.827+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44dc00cd70 con 0x7f44ec074c70 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.827+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec1a05f0 con 0x7f44ec074c70 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.827+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc018680 con 0x7f44ec074c70 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.827+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec1a7e70 con 0x7f44ec074c70 2024-04-25T18:58:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.828+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec069730 con 0x7f44ec074c70 2024-04-25T18:58:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.830+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44dc010430 con 0x7f44ec074c70 2024-04-25T18:58:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.831+0000 7f44e3e00700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f44d00a3cc0 0x7f44d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.832+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f44dc0d01a0 con 0x7f44ec074c70 2024-04-25T18:58:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.832+0000 7f44eaa00700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f44d00a3cc0 0x7f44d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.832+0000 7f44eaa00700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f44d00a3cc0 0x7f44d00a6180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f44ec102690 tx=0x7f44d800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:37.835+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44dc09cf80 con 0x7f44ec074c70 2024-04-25T18:58:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.062+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f44ec02df80 con 0x7f44ec074c70 2024-04-25T18:58:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.063+0000 7f44e3e00700 1 -- 172.21.15.86:0/1153676897 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f44dc09c880 con 0x7f44ec074c70 2024-04-25T18:58:38.064 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:38.064 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T18:58:38.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T18:58:38.065 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T18:58:38.065 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T18:58:38.065 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T18:58:38.065 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T18:58:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.067+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f44d00a3cc0 msgr2=0x7f44d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.067+0000 7f44f2000700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f44d00a3cc0 0x7f44d00a6180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f44ec102690 tx=0x7f44d800b260 comp rx=0 tx=0).stop 2024-04-25T18:58:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 msgr2=0x7f44ec19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec19b340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e9a0 tx=0x7f44dc00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:58:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 shutdown_connections 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f44d00a3cc0 0x7f44d00a6180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44ec10f440 0x7f44ec19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44ec074c70 0x7f44ec19b340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 --2- 172.21.15.86:0/1153676897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ec075590 0x7f44ec19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 >> 172.21.15.86:0/1153676897 conn(0x7f44ec079ba0 msgr2=0x7f44ec0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 shutdown_connections 2024-04-25T18:58:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:38.068+0000 7f44f2000700 1 -- 172.21.15.86:0/1153676897 wait complete. 2024-04-25T18:58:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:38.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:38.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:38 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1153676897' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:38.442 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:38 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1153676897' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:58:38.445 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.445 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.445 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:38.445 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:38.445 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:38.445 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:38.446 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:38.446 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:38 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1153676897' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:58:38.952 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T18:58:38.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T18:58:39.982 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:40.450 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:40 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.451 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:40 smithi086 ceph-mon[27783]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:40 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:40 smithi194 ceph-mon[32740]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:40 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:40.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:40 smithi183 ceph-mon[32344]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 -- 172.21.15.86:0/327439890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798108a50 msgr2=0x7f2798108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 --2- 172.21.15.86:0/327439890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798108a50 0x7f2798108e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2788009990 tx=0x7f2788009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 -- 172.21.15.86:0/327439890 shutdown_connections 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 --2- 172.21.15.86:0/327439890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798103450 0x7f279810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 --2- 172.21.15.86:0/327439890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798108a50 0x7f2798108e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 --2- 172.21.15.86:0/327439890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798102a90 0x7f2798102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 -- 172.21.15.86:0/327439890 >> 172.21.15.86:0/327439890 conn(0x7f2798079ba0 msgr2=0x7f2798079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 -- 172.21.15.86:0/327439890 shutdown_connections 2024-04-25T18:58:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.260+0000 7f279e000700 1 -- 172.21.15.86:0/327439890 wait complete. 2024-04-25T18:58:42.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.261+0000 7f279e000700 1 Processor -- start 2024-04-25T18:58:42.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.261+0000 7f279e000700 1 -- start start 2024-04-25T18:58:42.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f279e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 0x7f279819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f279e000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798103450 0x7f279819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f279e000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798108a50 0x7f279819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f279e000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2798112c60 con 0x7f2798102a90 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f279e000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2798112ae0 con 0x7f2798108a50 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f279e000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2798112de0 con 0x7f2798103450 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f2797400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 0x7f279819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f2797400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 0x7f279819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35524/0 (socket says 172.21.15.86:35524) 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f2797400700 1 -- 172.21.15.86:0/57773801 learned_addr learned my addr 172.21.15.86:0/57773801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f2796a00700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798103450 0x7f279819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.262+0000 7f2797e00700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798108a50 0x7f279819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797400700 1 -- 172.21.15.86:0/57773801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798103450 msgr2=0x7f279819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797400700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798103450 0x7f279819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797400700 1 -- 172.21.15.86:0/57773801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798108a50 msgr2=0x7f279819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797400700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798108a50 0x7f279819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797400700 1 -- 172.21.15.86:0/57773801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981a0390 con 0x7f2798102a90 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797e00700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798108a50 0x7f279819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f2797400700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 0x7f279819b3d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f27880047b0 tx=0x7f2788009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788027070 con 0x7f2798102a90 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2788022470 con 0x7f2798102a90 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981a0620 con 0x7f2798102a90 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.263+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788017650 con 0x7f2798102a90 2024-04-25T18:58:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.264+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981a7f00 con 0x7f2798102a90 2024-04-25T18:58:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.265+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2798069730 con 0x7f2798102a90 2024-04-25T18:58:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.268+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f278802a070 con 0x7f2798102a90 2024-04-25T18:58:42.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.269+0000 7f278fe00700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f277c0a3cc0 0x7f277c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.269+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27880d1380 con 0x7f2798102a90 2024-04-25T18:58:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.270+0000 7f2796a00700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f277c0a3cc0 0x7f277c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.270+0000 7f2796a00700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f277c0a3cc0 0x7f277c0a6180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f279810fcd0 tx=0x7f278400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:42.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.274+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2788031070 con 0x7f2798102a90 2024-04-25T18:58:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:42.534+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f27981a80d0 con 0x7f2798102a90 2024-04-25T18:58:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:42 smithi183 ceph-mon[32344]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:42 smithi194 ceph-mon[32740]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:42 smithi086 ceph-mon[27783]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:43.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.488+0000 7f278fe00700 1 -- 172.21.15.86:0/57773801 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f2788013070 con 0x7f2798102a90 2024-04-25T18:58:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.495+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f277c0a3cc0 msgr2=0x7f277c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.495+0000 7f279e000700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f277c0a3cc0 0x7f277c0a6180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f279810fcd0 tx=0x7f278400b040 comp rx=0 tx=0).stop 2024-04-25T18:58:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 msgr2=0x7f279819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 0x7f279819b3d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f27880047b0 tx=0x7f2788009550 comp rx=0 tx=0).stop 2024-04-25T18:58:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 shutdown_connections 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f277c0a3cc0 0x7f277c0a6180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2798108a50 0x7f279819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2798102a90 0x7f279819b3d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 --2- 172.21.15.86:0/57773801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798103450 0x7f279819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.496+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 >> 172.21.15.86:0/57773801 conn(0x7f2798079ba0 msgr2=0x7f27981015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.498+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 shutdown_connections 2024-04-25T18:58:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:43.498+0000 7f279e000700 1 -- 172.21.15.86:0/57773801 wait complete. 2024-04-25T18:58:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T18:58:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:43.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:43.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:43 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/57773801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:58:43.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:43.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:43.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:43 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/57773801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:58:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:58:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:43 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/57773801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:58:44.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T18:58:44.819 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:44 smithi086 ceph-mon[27783]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:44 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/57773801' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:58:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:44 smithi086 ceph-mon[27783]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T18:58:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:44 smithi183 ceph-mon[32344]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:44 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/57773801' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:58:44.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:44 smithi183 ceph-mon[32344]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T18:58:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:44 smithi194 ceph-mon[32740]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:44 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/57773801' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:58:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:44 smithi194 ceph-mon[32740]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T18:58:45.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:45 smithi086 ceph-mon[27783]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:58:45.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:45 smithi183 ceph-mon[32344]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:58:45.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:45 smithi194 ceph-mon[32740]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:58:46.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:46 smithi086 ceph-mon[27783]: pgmap v318: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:46.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:46 smithi086 ceph-mon[27783]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:58:46.683 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:46 smithi086 ceph-mon[27783]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:58:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:46 smithi183 ceph-mon[32344]: pgmap v318: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:46 smithi183 ceph-mon[32344]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:58:46.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:46 smithi183 ceph-mon[32344]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:58:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:46 smithi194 ceph-mon[32740]: pgmap v318: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:46 smithi194 ceph-mon[32740]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:58:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:46 smithi194 ceph-mon[32740]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 -- 172.21.15.86:0/2409531971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c101da0 msgr2=0x7f2a5c111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 --2- 172.21.15.86:0/2409531971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c101da0 0x7f2a5c111b90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400b0d0 tx=0x7f2a5400b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 -- 172.21.15.86:0/2409531971 shutdown_connections 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 --2- 172.21.15.86:0/2409531971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c101da0 0x7f2a5c111b90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 --2- 172.21.15.86:0/2409531971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c1038b0 0x7f2a5c103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 --2- 172.21.15.86:0/2409531971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 0x7f2a5c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 -- 172.21.15.86:0/2409531971 >> 172.21.15.86:0/2409531971 conn(0x7f2a5c076c60 msgr2=0x7f2a5c077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 -- 172.21.15.86:0/2409531971 shutdown_connections 2024-04-25T18:58:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.226+0000 7f2a60e00700 1 -- 172.21.15.86:0/2409531971 wait complete. 2024-04-25T18:58:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.227+0000 7f2a60e00700 1 Processor -- start 2024-04-25T18:58:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.227+0000 7f2a60e00700 1 -- start start 2024-04-25T18:58:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.227+0000 7f2a60e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 0x7f2a5c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.227+0000 7f2a60e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 0x7f2a5c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a60e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c1038b0 0x7f2a5c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a60e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c114f60 con 0x7f2a5c101400 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a60e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c114de0 con 0x7f2a5c101da0 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a60e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1150e0 con 0x7f2a5c1038b0 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a59e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 0x7f2a5c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 0x7f2a5c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 0x7f2a5c19b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48782/0 (socket says 172.21.15.86:48782) 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 -- 172.21.15.86:0/3764660231 learned_addr learned my addr 172.21.15.86:0/3764660231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a5a800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c1038b0 0x7f2a5c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a59e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 0x7f2a5c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35554/0 (socket says 172.21.15.86:35554) 2024-04-25T18:58:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 -- 172.21.15.86:0/3764660231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c1038b0 msgr2=0x7f2a5c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c1038b0 0x7f2a5c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 -- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 msgr2=0x7f2a5c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 0x7f2a5c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.228+0000 7f2a51400700 1 -- 172.21.15.86:0/3764660231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c1a0300 con 0x7f2a5c101da0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.229+0000 7f2a51400700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 0x7f2a5c19b830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400e9a0 tx=0x7f2a4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.229+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400ca10 con 0x7f2a5c101da0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.229+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c1a05f0 con 0x7f2a5c101da0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.229+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a4400cbb0 con 0x7f2a5c101da0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.229+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44018680 con 0x7f2a5c101da0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.229+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1a7e70 con 0x7f2a5c101da0 2024-04-25T18:58:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.231+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c069730 con 0x7f2a5c101da0 2024-04-25T18:58:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.233+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a44010430 con 0x7f2a5c101da0 2024-04-25T18:58:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.234+0000 7f2a53400700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2a400a3c40 0x7f2a400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.234+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f2a440d02d0 con 0x7f2a5c101da0 2024-04-25T18:58:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.234+0000 7f2a59e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2a400a3c40 0x7f2a400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.235+0000 7f2a59e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2a400a3c40 0x7f2a400a6100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009680 tx=0x7f2a4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.238+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a4409cf40 con 0x7f2a5c101da0 2024-04-25T18:58:47.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.492+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2a5c02df80 con 0x7f2a5c101da0 2024-04-25T18:58:47.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.539+0000 7f2a53400700 1 -- 172.21.15.86:0/3764660231 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f2a4409c840 con 0x7f2a5c101da0 2024-04-25T18:58:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.544+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2a400a3c40 msgr2=0x7f2a400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2a400a3c40 0x7f2a400a6100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009680 tx=0x7f2a4c009260 comp rx=0 tx=0).stop 2024-04-25T18:58:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 msgr2=0x7f2a5c19b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 0x7f2a5c19b830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400e9a0 tx=0x7f2a4400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 shutdown_connections 2024-04-25T18:58:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2a400a3c40 0x7f2a400a6100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a5c1038b0 0x7f2a5c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5c101400 0x7f2a5c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 --2- 172.21.15.86:0/3764660231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5c101da0 0x7f2a5c19b830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.545+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 >> 172.21.15.86:0/3764660231 conn(0x7f2a5c076c60 msgr2=0x7f2a5c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:47.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.546+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 shutdown_connections 2024-04-25T18:58:47.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:47.547+0000 7f2a60e00700 1 -- 172.21.15.86:0/3764660231 wait complete. 2024-04-25T18:58:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T18:58:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:47 smithi086 ceph-mon[27783]: pgmap v320: 33 pgs: 15 unknown, 18 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:47 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3764660231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:58:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:47 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:58:47.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:47 smithi183 ceph-mon[32344]: pgmap v320: 33 pgs: 15 unknown, 18 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:47.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:47 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3764660231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:58:47.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:47 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:58:47.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:47 smithi194 ceph-mon[32740]: pgmap v320: 33 pgs: 15 unknown, 18 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:47.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:47 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3764660231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:58:47.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:47 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:58:48.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T18:58:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:48 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:58:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:48 smithi086 ceph-mon[27783]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:58:48.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:48 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:58:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:48 smithi183 ceph-mon[32344]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:58:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:48 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:58:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:48 smithi194 ceph-mon[32740]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:58:49.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:49 smithi086 ceph-mon[27783]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:58:49.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:49 smithi086 ceph-mon[27783]: pgmap v323: 97 pgs: 22 creating+peering, 33 active+clean, 42 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:49.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:49 smithi183 ceph-mon[32344]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:58:49.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:49 smithi183 ceph-mon[32344]: pgmap v323: 97 pgs: 22 creating+peering, 33 active+clean, 42 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:49.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:49 smithi194 ceph-mon[32740]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:58:49.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:49 smithi194 ceph-mon[32740]: pgmap v323: 97 pgs: 22 creating+peering, 33 active+clean, 42 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:50 smithi086 ceph-mon[27783]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:58:50.882 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:50 smithi194 ceph-mon[32740]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:58:50.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:50 smithi183 ceph-mon[32344]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:58:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 -- 172.21.15.86:0/2652874479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020109fa0 msgr2=0x7f3020111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 --2- 172.21.15.86:0/2652874479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020109fa0 0x7f3020111b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f301800b0d0 tx=0x7f301800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 -- 172.21.15.86:0/2652874479 shutdown_connections 2024-04-25T18:58:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 --2- 172.21.15.86:0/2652874479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020109fa0 0x7f3020111b70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 --2- 172.21.15.86:0/2652874479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3020104ec0 0x7f30201052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 --2- 172.21.15.86:0/2652874479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3020105870 0x7f30201098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 -- 172.21.15.86:0/2652874479 >> 172.21.15.86:0/2652874479 conn(0x7f3020100750 msgr2=0x7f3020102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 -- 172.21.15.86:0/2652874479 shutdown_connections 2024-04-25T18:58:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.159+0000 7f3026e00700 1 -- 172.21.15.86:0/2652874479 wait complete. 2024-04-25T18:58:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.160+0000 7f3026e00700 1 Processor -- start 2024-04-25T18:58:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 -- start start 2024-04-25T18:58:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3020104ec0 0x7f302019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 0x7f302019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3020109fa0 0x7f302019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3020114eb0 con 0x7f3020109fa0 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3020114d30 con 0x7f3020104ec0 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3026e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3020115030 con 0x7f3020105870 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f301f400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 0x7f302019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f3024e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3020109fa0 0x7f302019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f301f400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 0x7f302019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50624/0 (socket says 172.21.15.86:50624) 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f301f400700 1 -- 172.21.15.86:0/1866810552 learned_addr learned my addr 172.21.15.86:0/1866810552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.161+0000 7f301fe00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3020104ec0 0x7f302019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301f400700 1 -- 172.21.15.86:0/1866810552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3020104ec0 msgr2=0x7f302019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301f400700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3020104ec0 0x7f302019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301f400700 1 -- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3020109fa0 msgr2=0x7f302019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301f400700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3020109fa0 0x7f302019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301f400700 1 -- 172.21.15.86:0/1866810552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30201a0390 con 0x7f3020105870 2024-04-25T18:58:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301f400700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 0x7f302019b8c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9f0 tx=0x7f300c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c004a30 con 0x7f3020105870 2024-04-25T18:58:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300c00cc70 con 0x7f3020105870 2024-04-25T18:58:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30201a0680 con 0x7f3020105870 2024-04-25T18:58:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.162+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c005530 con 0x7f3020105870 2024-04-25T18:58:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.163+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30201a7f00 con 0x7f3020105870 2024-04-25T18:58:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.164+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3020069730 con 0x7f3020105870 2024-04-25T18:58:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.166+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f300c004bd0 con 0x7f3020105870 2024-04-25T18:58:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.167+0000 7f301cc00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f30040a3c40 0x7f30040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.167+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7f300c0d0440 con 0x7f3020105870 2024-04-25T18:58:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.167+0000 7f301fe00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f30040a3c40 0x7f30040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.168+0000 7f301fe00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f30040a3c40 0x7f30040a6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3010009c60 tx=0x7f3010009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.171+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f300c00ce20 con 0x7f3020105870 2024-04-25T18:58:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.443+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f302002df80 con 0x7f3020105870 2024-04-25T18:58:51.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.576+0000 7f301cc00700 1 -- 172.21.15.86:0/1866810552 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f300c09cf30 con 0x7f3020105870 2024-04-25T18:58:51.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f30040a3c40 msgr2=0x7f30040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:51.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f30040a3c40 0x7f30040a6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3010009c60 tx=0x7f3010009260 comp rx=0 tx=0).stop 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 msgr2=0x7f302019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 0x7f302019b8c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9f0 tx=0x7f300c00ed00 comp rx=0 tx=0).stop 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 shutdown_connections 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f30040a3c40 0x7f30040a6100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3020109fa0 0x7f302019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3020104ec0 0x7f302019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 --2- 172.21.15.86:0/1866810552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3020105870 0x7f302019b8c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 >> 172.21.15.86:0/1866810552 conn(0x7f3020100750 msgr2=0x7f30201012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:51.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.580+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 shutdown_connections 2024-04-25T18:58:51.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:51.581+0000 7f3026e00700 1 -- 172.21.15.86:0/1866810552 wait complete. 2024-04-25T18:58:51.586 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T18:58:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:51 smithi086 ceph-mon[27783]: pgmap v325: 97 pgs: 22 creating+peering, 50 active+clean, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:51 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1866810552' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:58:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:51 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:58:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:51 smithi086 ceph-mon[27783]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:58:51.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:51 smithi183 ceph-mon[32344]: pgmap v325: 97 pgs: 22 creating+peering, 50 active+clean, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:51.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:51 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1866810552' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:58:51.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:51 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:58:51.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:51 smithi183 ceph-mon[32344]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:58:51.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:51 smithi194 ceph-mon[32740]: pgmap v325: 97 pgs: 22 creating+peering, 50 active+clean, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:51.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:51 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1866810552' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:58:51.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:51 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:58:51.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:51 smithi194 ceph-mon[32740]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:58:52.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:58:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:52 smithi086 ceph-mon[27783]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:58:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:52 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:58:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:52 smithi086 ceph-mon[27783]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:58:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:52 smithi086 ceph-mon[27783]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:58:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:52 smithi183 ceph-mon[32344]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:58:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:52 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:58:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:52 smithi183 ceph-mon[32344]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:58:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:52 smithi183 ceph-mon[32344]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:58:52.960 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:52 smithi194 ceph-mon[32740]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:58:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:52 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:58:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:52 smithi194 ceph-mon[32740]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:58:52.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:52 smithi194 ceph-mon[32740]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:58:53.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:53 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] up:active 2024-04-25T18:58:53.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:53 smithi086 ceph-mon[27783]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:58:53.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:53 smithi086 ceph-mon[27783]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:58:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:53 smithi086 ceph-mon[27783]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:58:53.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:53 smithi086 ceph-mon[27783]: pgmap v327: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-04-25T18:58:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:53 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] up:active 2024-04-25T18:58:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:53 smithi183 ceph-mon[32344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:58:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:53 smithi183 ceph-mon[32344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:58:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:53 smithi183 ceph-mon[32344]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:58:53.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:53 smithi183 ceph-mon[32344]: pgmap v327: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-04-25T18:58:53.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:53 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] up:active 2024-04-25T18:58:53.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:53 smithi194 ceph-mon[32740]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:58:53.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:53 smithi194 ceph-mon[32740]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:58:53.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:53 smithi194 ceph-mon[32740]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:58:53.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:53 smithi194 ceph-mon[32740]: pgmap v327: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-04-25T18:58:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:54 smithi086 ceph-mon[27783]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:58:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:54 smithi086 ceph-mon[27783]: Cluster is now healthy 2024-04-25T18:58:54.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:54 smithi183 ceph-mon[32344]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:58:54.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:54 smithi183 ceph-mon[32344]: Cluster is now healthy 2024-04-25T18:58:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:54 smithi194 ceph-mon[32740]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:58:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:54 smithi194 ceph-mon[32740]: Cluster is now healthy 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 -- 172.21.15.86:0/2415615007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5690108a30 msgr2=0x7f5690108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 --2- 172.21.15.86:0/2415615007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5690108a30 0x7f5690108e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5680009990 tx=0x7f5680009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 -- 172.21.15.86:0/2415615007 shutdown_connections 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 --2- 172.21.15.86:0/2415615007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f569006b0f0 0x7f569010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 --2- 172.21.15.86:0/2415615007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5690108a30 0x7f5690108e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 --2- 172.21.15.86:0/2415615007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006a750 0x7f569006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 -- 172.21.15.86:0/2415615007 >> 172.21.15.86:0/2415615007 conn(0x7f56900788f0 msgr2=0x7f5690078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 -- 172.21.15.86:0/2415615007 shutdown_connections 2024-04-25T18:58:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.131+0000 7f5696a00700 1 -- 172.21.15.86:0/2415615007 wait complete. 2024-04-25T18:58:55.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.132+0000 7f5696a00700 1 Processor -- start 2024-04-25T18:58:55.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 -- start start 2024-04-25T18:58:55.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f569006a750 0x7f569019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:55.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006b0f0 0x7f569019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:55.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 0x7f569019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:55.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5690112c20 con 0x7f5690108a30 2024-04-25T18:58:55.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5690112aa0 con 0x7f569006b0f0 2024-04-25T18:58:55.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.133+0000 7f5696a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5690112da0 con 0x7f569006a750 2024-04-25T18:58:55.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f5694a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 0x7f569019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:55.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f5694a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 0x7f569019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54164/0 (socket says 172.21.15.86:54164) 2024-04-25T18:58:55.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006b0f0 0x7f569019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f5694a00700 1 -- 172.21.15.86:0/1013669175 learned_addr learned my addr 172.21.15.86:0/1013669175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568fe00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f569006a750 0x7f569019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 -- 172.21.15.86:0/1013669175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f569006a750 msgr2=0x7f569019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f569006a750 0x7f569019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 -- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 msgr2=0x7f569019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 0x7f569019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 -- 172.21.15.86:0/1013669175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56901a0380 con 0x7f569006b0f0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f5694a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 0x7f569019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.134+0000 7f568f400700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006b0f0 0x7f569019b8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e9f0 tx=0x7f567c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.135+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c004220 con 0x7f569006b0f0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.135+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f567c0043c0 con 0x7f569006b0f0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.135+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c005020 con 0x7f569006b0f0 2024-04-25T18:58:55.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.135+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56901a0670 con 0x7f569006b0f0 2024-04-25T18:58:55.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.135+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a7ef0 con 0x7f569006b0f0 2024-04-25T18:58:55.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.136+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5690069730 con 0x7f569006b0f0 2024-04-25T18:58:55.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.138+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f567c004750 con 0x7f569006b0f0 2024-04-25T18:58:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.139+0000 7f568cc00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f56740a3cc0 0x7f56740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.140+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f567c0d0250 con 0x7f569006b0f0 2024-04-25T18:58:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.140+0000 7f568fe00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f56740a3cc0 0x7f56740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.140+0000 7f568fe00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f56740a3cc0 0x7f56740a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5680006010 tx=0x7f568000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.144+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f567c09cce0 con 0x7f569006b0f0 2024-04-25T18:58:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.372+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f569002df80 con 0x7f569006b0f0 2024-04-25T18:58:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.374+0000 7f568cc00700 1 -- 172.21.15.86:0/1013669175 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22228 (secure 0 0 0) 0x7f567c09c5e0 con 0x7f569006b0f0 2024-04-25T18:58:55.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:55.378 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T18:58:51.446712+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:58:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f56740a3cc0 msgr2=0x7f56740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f56740a3cc0 0x7f56740a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5680006010 tx=0x7f568000b260 comp rx=0 tx=0).stop 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006b0f0 msgr2=0x7f569019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006b0f0 0x7f569019b8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e9f0 tx=0x7f567c00ed00 comp rx=0 tx=0).stop 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 shutdown_connections 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f56740a3cc0 0x7f56740a6180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5690108a30 0x7f569019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f569006a750 0x7f569019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 --2- 172.21.15.86:0/1013669175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f569006b0f0 0x7f569019b8b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 >> 172.21.15.86:0/1013669175 conn(0x7f56900788f0 msgr2=0x7f569010de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 shutdown_connections 2024-04-25T18:58:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:55.378+0000 7f5696a00700 1 -- 172.21.15.86:0/1013669175 wait complete. 2024-04-25T18:58:55.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:55 smithi086 ceph-mon[27783]: pgmap v328: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T18:58:55.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:55 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1013669175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:55.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:55 smithi183 ceph-mon[32344]: pgmap v328: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T18:58:55.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:55 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1013669175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:55.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:55 smithi194 ceph-mon[32740]: pgmap v328: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T18:58:55.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:55 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1013669175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:56.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:58:56.646 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:56.666 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:58:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: 2024-04-25T18:58:56.409+0000 7fa17bfad700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T18:58:58.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:58 smithi086 ceph-mon[27783]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:58:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:58 smithi183 ceph-mon[32344]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:58:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:58 smithi194 ceph-mon[32740]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:58:58.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 -- 172.21.15.86:0/1027430519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 msgr2=0x7f67001052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 --2- 172.21.15.86:0/1027430519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f67001052a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66f00099c0 tx=0x7f66f0009cd0 comp rx=0 tx=0).stop 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 -- 172.21.15.86:0/1027430519 shutdown_connections 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 --2- 172.21.15.86:0/1027430519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700109fa0 0x7f6700111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 --2- 172.21.15.86:0/1027430519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f67001052a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 --2- 172.21.15.86:0/1027430519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6700105870 0x7f67001098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 -- 172.21.15.86:0/1027430519 >> 172.21.15.86:0/1027430519 conn(0x7f6700100750 msgr2=0x7f6700102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 -- 172.21.15.86:0/1027430519 shutdown_connections 2024-04-25T18:58:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.865+0000 7f6706e00700 1 -- 172.21.15.86:0/1027430519 wait complete. 2024-04-25T18:58:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 Processor -- start 2024-04-25T18:58:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 -- start start 2024-04-25T18:58:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f670019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 0x7f670019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6700109fa0 0x7f670019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6700114e50 con 0x7f6700109fa0 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6700114cd0 con 0x7f6700105870 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6706e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6700114fd0 con 0x7f6700104ec0 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 0x7f670019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 0x7f670019b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:42776/0 (socket says 172.21.15.86:42776) 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 -- 172.21.15.86:0/3262319770 learned_addr learned my addr 172.21.15.86:0/3262319770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ffe00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f670019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f6704e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6700109fa0 0x7f670019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 -- 172.21.15.86:0/3262319770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 msgr2=0x7f670019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f670019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 -- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6700109fa0 msgr2=0x7f670019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6700109fa0 0x7f670019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.867+0000 7f66ff400700 1 -- 172.21.15.86:0/3262319770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67001a0330 con 0x7f6700105870 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f66ffe00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f670019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f66ff400700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 0x7f670019b860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e970 tx=0x7f66ec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec004a30 con 0x7f6700105870 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec00cc90 con 0x7f6700105870 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec005610 con 0x7f6700105870 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67001a0620 con 0x7f6700105870 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.868+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67001a7ea0 con 0x7f6700105870 2024-04-25T18:58:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.869+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6700069730 con 0x7f6700105870 2024-04-25T18:58:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.872+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f66ec00c7e0 con 0x7f6700105870 2024-04-25T18:58:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.873+0000 7f66fcc00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f66e40a3c40 0x7f66e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.873+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f66ec0d04c0 con 0x7f6700105870 2024-04-25T18:58:58.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.873+0000 7f66ffe00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f66e40a3c40 0x7f66e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.874+0000 7f66ffe00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f66e40a3c40 0x7f66e40a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f66f0003760 tx=0x7f66f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:58.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:58.876+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66ec09cf50 con 0x7f6700105870 2024-04-25T18:58:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.102+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f670002df80 con 0x7f6700105870 2024-04-25T18:58:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.104+0000 7f66fcc00700 1 -- 172.21.15.86:0/3262319770 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22228 (secure 0 0 0) 0x7f66ec09c850 con 0x7f6700105870 2024-04-25T18:58:59.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:58:59.107 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T18:58:51.446712+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:58:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f66e40a3c40 msgr2=0x7f66e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f66e40a3c40 0x7f66e40a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f66f0003760 tx=0x7f66f0005fb0 comp rx=0 tx=0).stop 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 msgr2=0x7f670019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 0x7f670019b860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e970 tx=0x7f66ec00ec80 comp rx=0 tx=0).stop 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 shutdown_connections 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f66e40a3c40 0x7f66e40a6100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6700109fa0 0x7f670019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104ec0 0x7f670019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 --2- 172.21.15.86:0/3262319770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700105870 0x7f670019b860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 >> 172.21.15.86:0/3262319770 conn(0x7f6700100750 msgr2=0x7f6700109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 shutdown_connections 2024-04-25T18:58:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:58:59.108+0000 7f6706e00700 1 -- 172.21.15.86:0/3262319770 wait complete. 2024-04-25T18:58:59.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:58:59 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3262319770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:59.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:58:59 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3262319770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:59.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:58:59 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3262319770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:59.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:00.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:00 smithi086 ceph-mon[27783]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:59:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:00 smithi183 ceph-mon[32344]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:59:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:00 smithi194 ceph-mon[32740]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.439+0000 7f9d57600700 1 -- 172.21.15.86:0/3518351766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 msgr2=0x7f9d501096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.439+0000 7f9d57600700 1 --2- 172.21.15.86:0/3518351766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d501096b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9d440099c0 tx=0x7f9d44009cd0 comp rx=0 tx=0).stop 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 -- 172.21.15.86:0/3518351766 shutdown_connections 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 --2- 172.21.15.86:0/3518351766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d5010e3b0 0x7f9d50115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 --2- 172.21.15.86:0/3518351766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d501096b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 --2- 172.21.15.86:0/3518351766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d50109c80 0x7f9d5010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 -- 172.21.15.86:0/3518351766 >> 172.21.15.86:0/3518351766 conn(0x7f9d50076210 msgr2=0x7f9d50107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 -- 172.21.15.86:0/3518351766 shutdown_connections 2024-04-25T18:59:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.440+0000 7f9d57600700 1 -- 172.21.15.86:0/3518351766 wait complete. 2024-04-25T18:59:02.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.441+0000 7f9d57600700 1 Processor -- start 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.441+0000 7f9d57600700 1 -- start start 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d57600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d5019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d57600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d50109c80 0x7f9d5019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d57600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d5010e3b0 0x7f9d501a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d57600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50119270 con 0x7f9d501092d0 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d57600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d501190f0 con 0x7f9d5010e3b0 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d57600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9d501193f0 con 0x7f9d50109c80 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d5019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d5019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39740/0 (socket says 172.21.15.86:39740) 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 -- 172.21.15.86:0/880069549 learned_addr learned my addr 172.21.15.86:0/880069549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d4fe00700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d50109c80 0x7f9d5019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d55600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d5010e3b0 0x7f9d501a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 -- 172.21.15.86:0/880069549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d50109c80 msgr2=0x7f9d5019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d50109c80 0x7f9d5019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 -- 172.21.15.86:0/880069549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d5010e3b0 msgr2=0x7f9d501a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d5010e3b0 0x7f9d501a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.442+0000 7f9d54c00700 1 -- 172.21.15.86:0/880069549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d501a4740 con 0x7f9d501092d0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.443+0000 7f9d54c00700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d5019f780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d44006010 tx=0x7f9d440055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.443+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44026070 con 0x7f9d501092d0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.443+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d4400f460 con 0x7f9d501092d0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.443+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44009e60 con 0x7f9d501092d0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.443+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d501a49d0 con 0x7f9d501092d0 2024-04-25T18:59:02.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.443+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d501ac2b0 con 0x7f9d501092d0 2024-04-25T18:59:02.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.444+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d50069730 con 0x7f9d501092d0 2024-04-25T18:59:02.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.446+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d44029070 con 0x7f9d501092d0 2024-04-25T18:59:02.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.448+0000 7f9d4d600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f9d380a3cc0 0x7f9d380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:02.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.448+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f9d440d0e80 con 0x7f9d501092d0 2024-04-25T18:59:02.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.448+0000 7f9d4fe00700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f9d380a3cc0 0x7f9d380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:02.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.449+0000 7f9d4fe00700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f9d380a3cc0 0x7f9d380a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d50108910 tx=0x7f9d3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:02.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.452+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d44030020 con 0x7f9d501092d0 2024-04-25T18:59:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:02 smithi194 ceph-mon[32740]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:59:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:02 smithi086 ceph-mon[27783]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:59:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:02 smithi183 ceph-mon[32344]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:59:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:02.728+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9d501a0a90 con 0x7f9d501092d0 2024-04-25T18:59:03.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.409+0000 7f9d4d600700 1 -- 172.21.15.86:0/880069549 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9d44013070 con 0x7f9d501092d0 2024-04-25T18:59:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f9d380a3cc0 msgr2=0x7f9d380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f9d380a3cc0 0x7f9d380a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d50108910 tx=0x7f9d3c009260 comp rx=0 tx=0).stop 2024-04-25T18:59:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 msgr2=0x7f9d5019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d5019f780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d44006010 tx=0x7f9d440055f0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 shutdown_connections 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f9d380a3cc0 0x7f9d380a6180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d5010e3b0 0x7f9d501a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d501092d0 0x7f9d5019f780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 --2- 172.21.15.86:0/880069549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d50109c80 0x7f9d5019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.413+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 >> 172.21.15.86:0/880069549 conn(0x7f9d50076210 msgr2=0x7f9d50077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.414+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 shutdown_connections 2024-04-25T18:59:03.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:03.414+0000 7f9d57600700 1 -- 172.21.15.86:0/880069549 wait complete. 2024-04-25T18:59:03.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:03 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/880069549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:59:03.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:03 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/880069549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:59:03.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:03 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/880069549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:59:04.253 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T18:59:04.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-04-25T18:59:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:04 smithi086 ceph-mon[27783]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:59:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:04 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/880069549' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:59:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:04 smithi086 ceph-mon[27783]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:59:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:04 smithi183 ceph-mon[32344]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:59:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:04 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/880069549' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:59:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:04 smithi183 ceph-mon[32344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:59:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:04 smithi194 ceph-mon[32740]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:59:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:04 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/880069549' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:59:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:04 smithi194 ceph-mon[32740]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:59:06.692 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:06 smithi086 ceph-mon[27783]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-04-25T18:59:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:06 smithi194 ceph-mon[32740]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-04-25T18:59:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 -- 172.21.15.86:0/94902012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d01034b0 msgr2=0x7fb4d010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:06.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/94902012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d01034b0 0x7fb4d010f920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800b0d0 tx=0x7fb4c800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 -- 172.21.15.86:0/94902012 shutdown_connections 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/94902012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d01034b0 0x7fb4d010f920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/94902012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4d0107940 0x7fb4d0107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/94902012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d0102af0 0x7fb4d0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 -- 172.21.15.86:0/94902012 >> 172.21.15.86:0/94902012 conn(0x7fb4d0079ba0 msgr2=0x7fb4d0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 -- 172.21.15.86:0/94902012 shutdown_connections 2024-04-25T18:59:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.863+0000 7fb4d6a00700 1 -- 172.21.15.86:0/94902012 wait complete. 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 Processor -- start 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 -- start start 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0102af0 0x7fb4d01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 0x7fb4d0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4d0107940 0x7fb4d019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0112c90 con 0x7fb4d0102af0 2024-04-25T18:59:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0112b10 con 0x7fb4d01034b0 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.864+0000 7fb4d6a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0112e10 con 0x7fb4d0107940 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 0x7fb4d0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 0x7fb4d0199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55738/0 (socket says 172.21.15.86:55738) 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 -- 172.21.15.86:0/1189209898 learned_addr learned my addr 172.21.15.86:0/1189209898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4d0107940 0x7fb4d019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 -- 172.21.15.86:0/1189209898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4d0107940 msgr2=0x7fb4d019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4d0107940 0x7fb4d019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 -- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0102af0 msgr2=0x7fb4d01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0102af0 0x7fb4d01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 -- 172.21.15.86:0/1189209898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d019e0f0 con 0x7fb4d01034b0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.865+0000 7fb4cf400700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 0x7fb4d0199620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d980 tx=0x7fb4bc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.866+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc016070 con 0x7fb4d01034b0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.866+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d019e3e0 con 0x7fb4d01034b0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.866+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4bc004830 con 0x7fb4d01034b0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.866+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0054c0 con 0x7fb4d01034b0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.866+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d01a5c60 con 0x7fb4d01034b0 2024-04-25T18:59:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.867+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d0069730 con 0x7fb4d01034b0 2024-04-25T18:59:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.869+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4bc004220 con 0x7fb4d01034b0 2024-04-25T18:59:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.870+0000 7fb4ccc00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb4b40a3c40 0x7fb4b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.870+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb4bc0d0150 con 0x7fb4d01034b0 2024-04-25T18:59:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.870+0000 7fb4cfe00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb4b40a3c40 0x7fb4b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.871+0000 7fb4cfe00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb4b40a3c40 0x7fb4b40a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005e90 tx=0x7fb4c0005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:06.875+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4bc01f020 con 0x7fb4d01034b0 2024-04-25T18:59:06.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:06 smithi183 ceph-mon[32344]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-04-25T18:59:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.152+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fb4d002df80 con 0x7fb4d01034b0 2024-04-25T18:59:07.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.433+0000 7fb4ccc00700 1 -- 172.21.15.86:0/1189209898 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fb4bc09cbe0 con 0x7fb4d01034b0 2024-04-25T18:59:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.437+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb4b40a3c40 msgr2=0x7fb4b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.437+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb4b40a3c40 0x7fb4b40a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005e90 tx=0x7fb4c0005d00 comp rx=0 tx=0).stop 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.437+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 msgr2=0x7fb4d0199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.437+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 0x7fb4d0199620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d980 tx=0x7fb4bc00dc90 comp rx=0 tx=0).stop 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 shutdown_connections 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb4b40a3c40 0x7fb4b40a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4d0107940 0x7fb4d019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0102af0 0x7fb4d01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 --2- 172.21.15.86:0/1189209898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4d01034b0 0x7fb4d0199620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 >> 172.21.15.86:0/1189209898 conn(0x7fb4d0079ba0 msgr2=0x7fb4d0100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 shutdown_connections 2024-04-25T18:59:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:07.438+0000 7fb4d6a00700 1 -- 172.21.15.86:0/1189209898 wait complete. 2024-04-25T18:59:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:07 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1189209898' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T18:59:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:07 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T18:59:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:07 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1189209898' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T18:59:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:07 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T18:59:07.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:07 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1189209898' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T18:59:07.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:07 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T18:59:08.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-25T18:59:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-04-25T18:59:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:59:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T18:59:08.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:59:08.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:59:08.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-25T18:59:08.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:08 smithi194 ceph-mon[32740]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-25T18:59:08.746 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-04-25T18:59:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:59:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T18:59:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:59:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:59:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-25T18:59:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:08 smithi086 ceph-mon[27783]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-25T18:59:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-04-25T18:59:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:59:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T18:59:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:59:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:59:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-25T18:59:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:08 smithi183 ceph-mon[32344]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-25T18:59:09.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:09 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] up:active 2024-04-25T18:59:09.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:09 smithi086 ceph-mon[27783]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:59:09.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:09 smithi086 ceph-mon[27783]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-25T18:59:09.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:09 smithi086 ceph-mon[27783]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-04-25T18:59:09.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:09 smithi086 ceph-mon[27783]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T18:59:09.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:09 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] up:active 2024-04-25T18:59:09.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:09 smithi194 ceph-mon[32740]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:59:09.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:09 smithi194 ceph-mon[32740]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-25T18:59:09.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:09 smithi194 ceph-mon[32740]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-04-25T18:59:09.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:09 smithi194 ceph-mon[32740]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T18:59:09.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:09 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] up:active 2024-04-25T18:59:09.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:09 smithi183 ceph-mon[32344]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:59:09.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:09 smithi183 ceph-mon[32344]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-25T18:59:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:09 smithi183 ceph-mon[32344]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-04-25T18:59:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:09 smithi183 ceph-mon[32344]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T18:59:10.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:10 smithi086 ceph-mon[27783]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-04-25T18:59:10.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:10 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] up:active 2024-04-25T18:59:10.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:10 smithi086 ceph-mon[27783]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T18:59:10.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:10 smithi086 ceph-mon[27783]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-04-25T18:59:10.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:10 smithi086 ceph-mon[27783]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-04-25T18:59:10.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:10 smithi086 ceph-mon[27783]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-25T18:59:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:10 smithi194 ceph-mon[32740]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-04-25T18:59:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:10 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] up:active 2024-04-25T18:59:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:10 smithi194 ceph-mon[32740]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T18:59:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:10 smithi194 ceph-mon[32740]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-04-25T18:59:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:10 smithi194 ceph-mon[32740]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-04-25T18:59:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:10 smithi194 ceph-mon[32740]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-25T18:59:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:10 smithi183 ceph-mon[32344]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-04-25T18:59:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:10 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] up:active 2024-04-25T18:59:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:10 smithi183 ceph-mon[32344]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T18:59:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:10 smithi183 ceph-mon[32344]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-04-25T18:59:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:10 smithi183 ceph-mon[32344]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-04-25T18:59:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:10 smithi183 ceph-mon[32344]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-25T18:59:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 -- 172.21.15.86:0/1609186859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 msgr2=0x7f3074111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 --2- 172.21.15.86:0/1609186859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f3074111a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b0a0 tx=0x7f306c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 -- 172.21.15.86:0/1609186859 shutdown_connections 2024-04-25T18:59:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 --2- 172.21.15.86:0/1609186859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f3074111a80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 --2- 172.21.15.86:0/1609186859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3074104dd0 0x7f30741051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 --2- 172.21.15.86:0/1609186859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3074105780 0x7f30741097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 -- 172.21.15.86:0/1609186859 >> 172.21.15.86:0/1609186859 conn(0x7f3074100660 msgr2=0x7f3074102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 -- 172.21.15.86:0/1609186859 shutdown_connections 2024-04-25T18:59:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.997+0000 7f307b000700 1 -- 172.21.15.86:0/1609186859 wait complete. 2024-04-25T18:59:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.998+0000 7f307b000700 1 Processor -- start 2024-04-25T18:59:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 -- start start 2024-04-25T18:59:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3074104dd0 0x7f307419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3074105780 0x7f307419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f307419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3074114d90 con 0x7f3074105780 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3074114c10 con 0x7f3074109eb0 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307b000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3074114f10 con 0x7f3074104dd0 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f3079200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3074105780 0x7f307419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307a600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f307419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307a600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f307419fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59676/0 (socket says 172.21.15.86:59676) 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f307a600700 1 -- 172.21.15.86:0/1192922571 learned_addr learned my addr 172.21.15.86:0/1192922571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:10.999+0000 7f3079c00700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3074104dd0 0x7f307419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307a600700 1 -- 172.21.15.86:0/1192922571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3074104dd0 msgr2=0x7f307419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307a600700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3074104dd0 0x7f307419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307a600700 1 -- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3074105780 msgr2=0x7f307419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307a600700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3074105780 0x7f307419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307a600700 1 -- 172.21.15.86:0/1192922571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30741a0210 con 0x7f3074109eb0 2024-04-25T18:59:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307a600700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f307419fad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b070 tx=0x7f306c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c021070 con 0x7f3074109eb0 2024-04-25T18:59:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f306c003c00 con 0x7f3074109eb0 2024-04-25T18:59:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30741a04a0 con 0x7f3074109eb0 2024-04-25T18:59:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.000+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c004750 con 0x7f3074109eb0 2024-04-25T18:59:11.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.001+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30741a7d80 con 0x7f3074109eb0 2024-04-25T18:59:11.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.002+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074069730 con 0x7f3074109eb0 2024-04-25T18:59:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.003+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f306c007980 con 0x7f3074109eb0 2024-04-25T18:59:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.005+0000 7f3072000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f305c0a3c40 0x7f305c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.005+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f306c00f020 con 0x7f3074109eb0 2024-04-25T18:59:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.005+0000 7f3079c00700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f305c0a3c40 0x7f305c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:11.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.006+0000 7f3079c00700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f305c0a3c40 0x7f305c0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3068009c60 tx=0x7f3068009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.009+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f306c02a020 con 0x7f3074109eb0 2024-04-25T18:59:11.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.284+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f307402df80 con 0x7f3074109eb0 2024-04-25T18:59:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.491+0000 7f3072000700 1 -- 172.21.15.86:0/1192922571 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f306c09e220 con 0x7f3074109eb0 2024-04-25T18:59:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.495+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f305c0a3c40 msgr2=0x7f305c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.495+0000 7f307b000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f305c0a3c40 0x7f305c0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3068009c60 tx=0x7f3068009260 comp rx=0 tx=0).stop 2024-04-25T18:59:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.495+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 msgr2=0x7f307419fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.495+0000 7f307b000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f307419fad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b070 tx=0x7f306c004c80 comp rx=0 tx=0).stop 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 shutdown_connections 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f305c0a3c40 0x7f305c0a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3074109eb0 0x7f307419fad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3074104dd0 0x7f307419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 --2- 172.21.15.86:0/1192922571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3074105780 0x7f307419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 >> 172.21.15.86:0/1192922571 conn(0x7f3074100660 msgr2=0x7f3074102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 shutdown_connections 2024-04-25T18:59:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:11.496+0000 7f307b000700 1 -- 172.21.15.86:0/1192922571 wait complete. 2024-04-25T18:59:11.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] up:active 2024-04-25T18:59:11.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T18:59:11.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:59:11.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: Cluster is now healthy 2024-04-25T18:59:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-04-25T18:59:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T18:59:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-25T18:59:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1192922571' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:59:11.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:11 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:59:11.666 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: 2024-04-25T18:59:11.409+0000 7fa17bfad700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: 2024-04-25T18:59:11.409+0000 7fa17bfad700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: 2024-04-25T18:59:11.409+0000 7fa17bfad700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] up:active 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: Cluster is now healthy 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-04-25T18:59:11.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T18:59:11.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-25T18:59:11.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1192922571' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:59:11.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:11 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:59:11.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] up:active 2024-04-25T18:59:11.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T18:59:11.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:59:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: Cluster is now healthy 2024-04-25T18:59:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-04-25T18:59:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T18:59:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-25T18:59:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1192922571' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:59:11.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:11 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:59:12.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-25T18:59:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:12 smithi086 ceph-mon[27783]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-25T18:59:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:12 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:59:12.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:12 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] up:active 2024-04-25T18:59:12.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:12 smithi086 ceph-mon[27783]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-25T18:59:12.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:12 smithi086 ceph-mon[27783]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T18:59:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:12 smithi183 ceph-mon[32344]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-25T18:59:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:12 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:59:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:12 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] up:active 2024-04-25T18:59:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:12 smithi183 ceph-mon[32344]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-25T18:59:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:12 smithi183 ceph-mon[32344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T18:59:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:12 smithi194 ceph-mon[32740]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-25T18:59:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:12 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:59:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:12 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] up:active 2024-04-25T18:59:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:12 smithi194 ceph-mon[32740]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-25T18:59:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:12 smithi194 ceph-mon[32740]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T18:59:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:14 smithi086 ceph-mon[27783]: pgmap v337: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-04-25T18:59:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:14 smithi183 ceph-mon[32344]: pgmap v337: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-04-25T18:59:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:14 smithi194 ceph-mon[32740]: pgmap v337: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-04-25T18:59:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 -- 172.21.15.86:0/2940190010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84103310 msgr2=0x7f6c8410f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 --2- 172.21.15.86:0/2940190010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84103310 0x7f6c8410f820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00b0a0 tx=0x7f6c7c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 -- 172.21.15.86:0/2940190010 shutdown_connections 2024-04-25T18:59:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 --2- 172.21.15.86:0/2940190010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84103310 0x7f6c8410f820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 --2- 172.21.15.86:0/2940190010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84108950 0x7f6c84108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 --2- 172.21.15.86:0/2940190010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 0x7f6c84102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 -- 172.21.15.86:0/2940190010 >> 172.21.15.86:0/2940190010 conn(0x7f6c840fe460 msgr2=0x7f6c84100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.032+0000 7f6c89800700 1 -- 172.21.15.86:0/2940190010 shutdown_connections 2024-04-25T18:59:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.033+0000 7f6c89800700 1 -- 172.21.15.86:0/2940190010 wait complete. 2024-04-25T18:59:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 Processor -- start 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 -- start start 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 0x7f6c8419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84103310 0x7f6c8419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84108950 0x7f6c8419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84112b60 con 0x7f6c84108950 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841129e0 con 0x7f6c84103310 2024-04-25T18:59:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.034+0000 7f6c89800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84112ce0 con 0x7f6c84102950 2024-04-25T18:59:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 0x7f6c8419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c88e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84108950 0x7f6c8419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84103310 0x7f6c8419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 0x7f6c8419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:49286/0 (socket says 172.21.15.86:49286) 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83e00700 1 -- 172.21.15.86:0/3336871762 learned_addr learned my addr 172.21.15.86:0/3336871762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 -- 172.21.15.86:0/3336871762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 msgr2=0x7f6c8419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 0x7f6c8419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 -- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84108950 msgr2=0x7f6c8419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84108950 0x7f6c8419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 -- 172.21.15.86:0/3336871762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c841a01a0 con 0x7f6c84103310 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.035+0000 7f6c83400700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84103310 0x7f6c8419b6d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000e9f0 tx=0x7f6c7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.036+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004220 con 0x7f6c84103310 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.036+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c700043c0 con 0x7f6c84103310 2024-04-25T18:59:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.036+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c841a0490 con 0x7f6c84103310 2024-04-25T18:59:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.036+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70005020 con 0x7f6c84103310 2024-04-25T18:59:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.036+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a7d10 con 0x7f6c84103310 2024-04-25T18:59:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.037+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c84069730 con 0x7f6c84103310 2024-04-25T18:59:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.039+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c70004750 con 0x7f6c84103310 2024-04-25T18:59:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.040+0000 7f6c80c00700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c680a3c40 0x7f6c680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.041+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f6c700d02d0 con 0x7f6c84103310 2024-04-25T18:59:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.041+0000 7f6c83e00700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c680a3c40 0x7f6c680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.041+0000 7f6c83e00700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c680a3c40 0x7f6c680a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009ca0 tx=0x7f6c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.045+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c7009cd60 con 0x7f6c84103310 2024-04-25T18:59:15.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.317+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6c841a7fa0 con 0x7f6c84103310 2024-04-25T18:59:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.510+0000 7f6c80c00700 1 -- 172.21.15.86:0/3336871762 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f6c7009c660 con 0x7f6c84103310 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.514+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c680a3c40 msgr2=0x7f6c680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.514+0000 7f6c89800700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c680a3c40 0x7f6c680a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009ca0 tx=0x7f6c74009260 comp rx=0 tx=0).stop 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.514+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84103310 msgr2=0x7f6c8419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.514+0000 7f6c89800700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84103310 0x7f6c8419b6d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000e9f0 tx=0x7f6c7000ed00 comp rx=0 tx=0).stop 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 shutdown_connections 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f6c680a3c40 0x7f6c680a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c84108950 0x7f6c8419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c84102950 0x7f6c8419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 --2- 172.21.15.86:0/3336871762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c84103310 0x7f6c8419b6d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:15.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 >> 172.21.15.86:0/3336871762 conn(0x7f6c840fe460 msgr2=0x7f6c8410e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:15.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 shutdown_connections 2024-04-25T18:59:15.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:15.515+0000 7f6c89800700 1 -- 172.21.15.86:0/3336871762 wait complete. 2024-04-25T18:59:15.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:15 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3336871762' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:59:15.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:15 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:59:15.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:15 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3336871762' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:59:15.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:15 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:59:15.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:15 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3336871762' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:59:15.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:15 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:59:16.461 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T18:59:16.461 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-25T18:59:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:16 smithi086 ceph-mon[27783]: pgmap v338: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T18:59:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:16 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:59:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:16 smithi086 ceph-mon[27783]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T18:59:16.905 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:16 smithi183 ceph-mon[32344]: pgmap v338: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T18:59:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:16 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:59:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:16 smithi183 ceph-mon[32344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T18:59:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:16 smithi194 ceph-mon[32740]: pgmap v338: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T18:59:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:16 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:59:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:16 smithi194 ceph-mon[32740]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T18:59:18.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:18 smithi086 ceph-mon[27783]: pgmap v339: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T18:59:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:18 smithi183 ceph-mon[32344]: pgmap v339: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T18:59:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:18 smithi194 ceph-mon[32740]: pgmap v339: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T18:59:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 -- 172.21.15.86:0/557417343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714105480 msgr2=0x7f2714105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 --2- 172.21.15.86:0/557417343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714105480 0x7f2714105910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f270c009990 tx=0x7f270c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 -- 172.21.15.86:0/557417343 shutdown_connections 2024-04-25T18:59:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 --2- 172.21.15.86:0/557417343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2714105480 0x7f2714105910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 --2- 172.21.15.86:0/557417343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2714107970 0x7f271406a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 --2- 172.21.15.86:0/557417343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f271406ac90 0x7f271406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 -- 172.21.15.86:0/557417343 >> 172.21.15.86:0/557417343 conn(0x7f27140788f0 msgr2=0x7f2714078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 -- 172.21.15.86:0/557417343 shutdown_connections 2024-04-25T18:59:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.053+0000 7f2719c00700 1 -- 172.21.15.86:0/557417343 wait complete. 2024-04-25T18:59:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.054+0000 7f2719c00700 1 Processor -- start 2024-04-25T18:59:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.054+0000 7f2719c00700 1 -- start start 2024-04-25T18:59:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2719c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271406ac90 0x7f271419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2719c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2714105480 0x7f271419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2719c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2714107970 0x7f27141a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2719c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2714119230 con 0x7f271406ac90 2024-04-25T18:59:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2719c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27141190b0 con 0x7f2714105480 2024-04-25T18:59:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2719c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27141193b0 con 0x7f2714107970 2024-04-25T18:59:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2712a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271406ac90 0x7f271419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f270a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2714105480 0x7f271419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2712a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271406ac90 0x7f271419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43312/0 (socket says 172.21.15.86:43312) 2024-04-25T18:59:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2712a00700 1 -- 172.21.15.86:0/3314434227 learned_addr learned my addr 172.21.15.86:0/3314434227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f2713400700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2714107970 0x7f27141a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f270a000700 1 -- 172.21.15.86:0/3314434227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2714107970 msgr2=0x7f27141a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f270a000700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2714107970 0x7f27141a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f270a000700 1 -- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271406ac90 msgr2=0x7f271419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f270a000700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271406ac90 0x7f271419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.055+0000 7f270a000700 1 -- 172.21.15.86:0/3314434227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141a46d0 con 0x7f2714105480 2024-04-25T18:59:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.056+0000 7f270a000700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2714105480 0x7f271419fc00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e9f0 tx=0x7f26fc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.056+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00c9c0 con 0x7f2714105480 2024-04-25T18:59:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.056+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc00cb60 con 0x7f2714105480 2024-04-25T18:59:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.056+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc018680 con 0x7f2714105480 2024-04-25T18:59:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.056+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27141a49c0 con 0x7f2714105480 2024-04-25T18:59:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.056+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141ac240 con 0x7f2714105480 2024-04-25T18:59:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.057+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2714069730 con 0x7f2714105480 2024-04-25T18:59:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.059+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26fc010430 con 0x7f2714105480 2024-04-25T18:59:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.060+0000 7f270be00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26f80a3cc0 0x7f26f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.061+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f26fc0d0750 con 0x7f2714105480 2024-04-25T18:59:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.061+0000 7f2712a00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26f80a3cc0 0x7f26f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.061+0000 7f2712a00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26f80a3cc0 0x7f26f80a6180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2704009c10 tx=0x7f270400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.063+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26fc09d1e0 con 0x7f2714105480 2024-04-25T18:59:19.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.321+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2714104b70 con 0x7f26f80a3cc0 2024-04-25T18:59:19.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.363+0000 7f270be00700 1 -- 172.21.15.86:0/3314434227 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2714104b70 con 0x7f26f80a3cc0 2024-04-25T18:59:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26f80a3cc0 msgr2=0x7f26f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26f80a3cc0 0x7f26f80a6180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2704009c10 tx=0x7f270400b040 comp rx=0 tx=0).stop 2024-04-25T18:59:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2714105480 msgr2=0x7f271419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2714105480 0x7f271419fc00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00e9f0 tx=0x7f26fc00ed00 comp rx=0 tx=0).stop 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 shutdown_connections 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26f80a3cc0 0x7f26f80a6180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2714107970 0x7f27141a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271406ac90 0x7f271419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 --2- 172.21.15.86:0/3314434227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2714105480 0x7f271419fc00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 >> 172.21.15.86:0/3314434227 conn(0x7f27140788f0 msgr2=0x7f2714104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 shutdown_connections 2024-04-25T18:59:19.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:19.366+0000 7f2719c00700 1 -- 172.21.15.86:0/3314434227 wait complete. 2024-04-25T18:59:19.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-mon[27783]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:19.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-mon[27783]: pgmap v340: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T18:59:19.773 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:59:19.330+0000 7fa097a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.773 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:59:19.330+0000 7fa097a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.773 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:59:19.330+0000 7fa097a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.773 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:59:19.330+0000 7fa097a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.774 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 18:59:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T18:59:19.330+0000 7fa097a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:19 smithi183 ceph-mon[32344]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:19 smithi183 ceph-mon[32344]: pgmap v340: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T18:59:19.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:19 smithi194 ceph-mon[32740]: from='client.24599 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:19.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:19 smithi194 ceph-mon[32740]: pgmap v340: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T18:59:20.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T18:59:20.614 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:20.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:20 smithi086 ceph-mon[27783]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:59:20.883 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:20 smithi194 ceph-mon[32740]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:59:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:20 smithi183 ceph-mon[32344]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:59:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:21 smithi086 ceph-mon[27783]: pgmap v341: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.7 KiB/s wr, 22 op/s 2024-04-25T18:59:21.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:21 smithi183 ceph-mon[32344]: pgmap v341: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.7 KiB/s wr, 22 op/s 2024-04-25T18:59:21.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:21 smithi194 ceph-mon[32740]: pgmap v341: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.7 KiB/s wr, 22 op/s 2024-04-25T18:59:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.750+0000 7f4cb5800700 1 -- 172.21.15.86:0/3879283419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 msgr2=0x7f4cb0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.750+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3879283419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb0108e30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009990 tx=0x7f4ca0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 -- 172.21.15.86:0/3879283419 shutdown_connections 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3879283419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0103450 0x7f4cb010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3879283419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb0108e30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3879283419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0102a90 0x7f4cb0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 -- 172.21.15.86:0/3879283419 >> 172.21.15.86:0/3879283419 conn(0x7f4cb0079ba0 msgr2=0x7f4cb0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 -- 172.21.15.86:0/3879283419 shutdown_connections 2024-04-25T18:59:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.751+0000 7f4cb5800700 1 -- 172.21.15.86:0/3879283419 wait complete. 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 Processor -- start 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 -- start start 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0102a90 0x7f4cb019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 0x7f4cb019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0112be0 con 0x7f4cb0108a50 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0112a60 con 0x7f4cb0102a90 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.752+0000 7f4cb5800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0112d60 con 0x7f4cb0103450 2024-04-25T18:59:22.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4caf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 0x7f4cb019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4caf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb019fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33914/0 (socket says 172.21.15.86:33914) 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4caf400700 1 -- 172.21.15.86:0/3601166431 learned_addr learned my addr 172.21.15.86:0/3601166431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 0x7f4cb019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33674/0 (socket says 172.21.15.86:33674) 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4caea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0102a90 0x7f4cb019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 -- 172.21.15.86:0/3601166431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0102a90 msgr2=0x7f4cb019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0102a90 0x7f4cb019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 -- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 msgr2=0x7f4cb019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4ca6000700 1 -- 172.21.15.86:0/3601166431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb01a0310 con 0x7f4cb0103450 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4caf400700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.753+0000 7f4caea00700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0102a90 0x7f4cb019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:22.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.754+0000 7f4ca6000700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 0x7f4cb019b840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800e9f0 tx=0x7f4c9800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.754+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9800cb70 con 0x7f4cb0103450 2024-04-25T18:59:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.754+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9800ee90 con 0x7f4cb0103450 2024-04-25T18:59:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.754+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98003d40 con 0x7f4cb0103450 2024-04-25T18:59:22.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.754+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb01a0600 con 0x7f4cb0103450 2024-04-25T18:59:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.754+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb01a7e80 con 0x7f4cb0103450 2024-04-25T18:59:22.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.755+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb0069730 con 0x7f4cb0103450 2024-04-25T18:59:22.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.757+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4c98003ee0 con 0x7f4cb0103450 2024-04-25T18:59:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.759+0000 7f4ca7e00700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c940a3cd0 0x7f4c940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.759+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f4c980d0810 con 0x7f4cb0103450 2024-04-25T18:59:22.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.759+0000 7f4caea00700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c940a3cd0 0x7f4c940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:22.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.760+0000 7f4caea00700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c940a3cd0 0x7f4c940a6190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00098e0 tx=0x7f4ca0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:22.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:22.761+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c98053ff0 con 0x7f4cb0103450 2024-04-25T18:59:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.020+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4cb010f620 con 0x7f4c940a3cd0 2024-04-25T18:59:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.061+0000 7f4ca7e00700 1 -- 172.21.15.86:0/3601166431 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cb010f620 con 0x7f4c940a3cd0 2024-04-25T18:59:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c940a3cd0 msgr2=0x7f4c940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c940a3cd0 0x7f4c940a6190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00098e0 tx=0x7f4ca0005960 comp rx=0 tx=0).stop 2024-04-25T18:59:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 msgr2=0x7f4cb019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 0x7f4cb019b840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800e9f0 tx=0x7f4c9800ed00 comp rx=0 tx=0).stop 2024-04-25T18:59:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 shutdown_connections 2024-04-25T18:59:23.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c940a3cd0 0x7f4c940a6190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cb0108a50 0x7f4cb019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb0102a90 0x7f4cb019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 --2- 172.21.15.86:0/3601166431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0103450 0x7f4cb019b840 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.064+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 >> 172.21.15.86:0/3601166431 conn(0x7f4cb0079ba0 msgr2=0x7f4cb010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.065+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 shutdown_connections 2024-04-25T18:59:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:23.065+0000 7f4cb5800700 1 -- 172.21.15.86:0/3601166431 wait complete. 2024-04-25T18:59:23.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T18:59:24.363 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:24 smithi183 ceph-mon[32344]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:24 smithi183 ceph-mon[32344]: pgmap v342: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.0 KiB/s wr, 19 op/s 2024-04-25T18:59:24.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:24 smithi086 ceph-mon[27783]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:24.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:24 smithi086 ceph-mon[27783]: pgmap v342: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.0 KiB/s wr, 19 op/s 2024-04-25T18:59:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:24 smithi194 ceph-mon[32740]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:24 smithi194 ceph-mon[32740]: pgmap v342: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.0 KiB/s wr, 19 op/s 2024-04-25T18:59:26.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 -- 172.21.15.86:0/3580222352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20105870 msgr2=0x7f5a201098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:26.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 --2- 172.21.15.86:0/3580222352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20105870 0x7f5a201098c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009990 tx=0x7f5a0c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 -- 172.21.15.86:0/3580222352 shutdown_connections 2024-04-25T18:59:26.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 --2- 172.21.15.86:0/3580222352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20109fa0 0x7f5a20111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 --2- 172.21.15.86:0/3580222352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20104ec0 0x7f5a201052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 --2- 172.21.15.86:0/3580222352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20105870 0x7f5a201098c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 -- 172.21.15.86:0/3580222352 >> 172.21.15.86:0/3580222352 conn(0x7f5a20100750 msgr2=0x7f5a20102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:26.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 -- 172.21.15.86:0/3580222352 shutdown_connections 2024-04-25T18:59:26.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.519+0000 7f5a28200700 1 -- 172.21.15.86:0/3580222352 wait complete. 2024-04-25T18:59:26.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:26 smithi086 ceph-mon[27783]: pgmap v343: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.520+0000 7f5a28200700 1 Processor -- start 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 -- start start 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20104ec0 0x7f5a2019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20105870 0x7f5a2019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 0x7f5a2019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20114e60 con 0x7f5a20105870 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20114ce0 con 0x7f5a20104ec0 2024-04-25T18:59:26.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.521+0000 7f5a28200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20114fe0 con 0x7f5a20109fa0 2024-04-25T18:59:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.522+0000 7f5a26200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 0x7f5a2019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.522+0000 7f5a24e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20105870 0x7f5a2019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.522+0000 7f5a25800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20104ec0 0x7f5a2019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.522+0000 7f5a26200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 0x7f5a2019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33692/0 (socket says 172.21.15.86:33692) 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.522+0000 7f5a26200700 1 -- 172.21.15.86:0/3796701888 learned_addr learned my addr 172.21.15.86:0/3796701888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a26200700 1 -- 172.21.15.86:0/3796701888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20104ec0 msgr2=0x7f5a2019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a26200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20104ec0 0x7f5a2019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a26200700 1 -- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20105870 msgr2=0x7f5a2019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a26200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20105870 0x7f5a2019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a26200700 1 -- 172.21.15.86:0/3796701888 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a201a0380 con 0x7f5a20109fa0 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a25800700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20104ec0 0x7f5a2019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a24e00700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20105870 0x7f5a2019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a26200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 0x7f5a2019fc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800ef30 tx=0x7f5a1800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.523+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180092e0 con 0x7f5a20109fa0 2024-04-25T18:59:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.524+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a18004bb0 con 0x7f5a20109fa0 2024-04-25T18:59:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.524+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a201a0610 con 0x7f5a20109fa0 2024-04-25T18:59:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.524+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18004280 con 0x7f5a20109fa0 2024-04-25T18:59:26.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.524+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201a7ef0 con 0x7f5a20109fa0 2024-04-25T18:59:26.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.525+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a20069730 con 0x7f5a20109fa0 2024-04-25T18:59:26.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.527+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5a1801b030 con 0x7f5a20109fa0 2024-04-25T18:59:26.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.528+0000 7f5a1e000700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5a080a3d50 0x7f5a080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:26.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.528+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f5a180d0260 con 0x7f5a20109fa0 2024-04-25T18:59:26.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.529+0000 7f5a25800700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5a080a3d50 0x7f5a080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:26.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.529+0000 7f5a25800700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5a080a3d50 0x7f5a080a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009cb0 tx=0x7f5a14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:26.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.531+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a18022020 con 0x7f5a20109fa0 2024-04-25T18:59:26.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:26 smithi183 ceph-mon[32344]: pgmap v343: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:26 smithi194 ceph-mon[32740]: pgmap v343: 97 pgs: 97 active+clean; 592 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:26.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.796+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5a20062e80 con 0x7f5a080a3d50 2024-04-25T18:59:26.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.808+0000 7f5a1e000700 1 -- 172.21.15.86:0/3796701888 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5a20062e80 con 0x7f5a080a3d50 2024-04-25T18:59:26.810 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/fc0a9234-5bcc-4298-b705-4470b7775988 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5a080a3d50 msgr2=0x7f5a080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5a080a3d50 0x7f5a080a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009cb0 tx=0x7f5a14009260 comp rx=0 tx=0).stop 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 msgr2=0x7f5a2019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 0x7f5a2019fc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800ef30 tx=0x7f5a1800c3a0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 shutdown_connections 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5a080a3d50 0x7f5a080a6210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a20109fa0 0x7f5a2019fc40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a20104ec0 0x7f5a2019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 --2- 172.21.15.86:0/3796701888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a20105870 0x7f5a2019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 >> 172.21.15.86:0/3796701888 conn(0x7f5a20100750 msgr2=0x7f5a20102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 shutdown_connections 2024-04-25T18:59:26.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:26.811+0000 7f5a28200700 1 -- 172.21.15.86:0/3796701888 wait complete. 2024-04-25T18:59:27.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T18:59:28.076 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:28.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:28 smithi086 ceph-mon[27783]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:28.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:28 smithi086 ceph-mon[27783]: pgmap v344: 97 pgs: 97 active+clean; 606 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-25T18:59:28.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:28 smithi183 ceph-mon[32344]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:28.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:28 smithi183 ceph-mon[32344]: pgmap v344: 97 pgs: 97 active+clean; 606 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-25T18:59:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:28 smithi194 ceph-mon[32740]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:28 smithi194 ceph-mon[32740]: pgmap v344: 97 pgs: 97 active+clean; 606 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.193+0000 7f7dc5400700 1 -- 172.21.15.86:0/766438931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc0074e70 msgr2=0x7f7dc0075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.193+0000 7f7dc5400700 1 --2- 172.21.15.86:0/766438931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc0074e70 0x7f7dc0075250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7db00098f0 tx=0x7f7db0009c00 comp rx=0 tx=0).stop 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 -- 172.21.15.86:0/766438931 shutdown_connections 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 --2- 172.21.15.86:0/766438931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc010f1b0 0x7f7dc0111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 --2- 172.21.15.86:0/766438931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc0074e70 0x7f7dc0075250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 --2- 172.21.15.86:0/766438931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dc0075790 0x7f7dc010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 -- 172.21.15.86:0/766438931 >> 172.21.15.86:0/766438931 conn(0x7f7dc00fe4c0 msgr2=0x7f7dc01008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 -- 172.21.15.86:0/766438931 shutdown_connections 2024-04-25T18:59:30.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.194+0000 7f7dc5400700 1 -- 172.21.15.86:0/766438931 wait complete. 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.195+0000 7f7dc5400700 1 Processor -- start 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.195+0000 7f7dc5400700 1 -- start start 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.195+0000 7f7dc5400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dc0074e70 0x7f7dc019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.195+0000 7f7dc5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 0x7f7dc019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7dc5400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc010f1b0 0x7f7dc019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7dc5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0114da0 con 0x7f7dc0075790 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7dc5400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0114c20 con 0x7f7dc010f1b0 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7dc5400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0114f20 con 0x7f7dc0074e70 2024-04-25T18:59:30.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 0x7f7dc019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7dbfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dc0074e70 0x7f7dc019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 0x7f7dc019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51286/0 (socket says 172.21.15.86:51286) 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 -- 172.21.15.86:0/3969562782 learned_addr learned my addr 172.21.15.86:0/3969562782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7dc4a00700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc010f1b0 0x7f7dc019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 -- 172.21.15.86:0/3969562782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dc0074e70 msgr2=0x7f7dc019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dc0074e70 0x7f7dc019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 -- 172.21.15.86:0/3969562782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc010f1b0 msgr2=0x7f7dc019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc010f1b0 0x7f7dc019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.196+0000 7f7db7400700 1 -- 172.21.15.86:0/3969562782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc01a0340 con 0x7f7dc0075790 2024-04-25T18:59:30.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.197+0000 7f7db7400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 0x7f7dc019b870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e9c0 tx=0x7f7da800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.197+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da800cbd0 con 0x7f7dc0075790 2024-04-25T18:59:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.197+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc01a0630 con 0x7f7dc0075790 2024-04-25T18:59:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.197+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da800ee60 con 0x7f7dc0075790 2024-04-25T18:59:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.197+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80055c0 con 0x7f7dc0075790 2024-04-25T18:59:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.197+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc01a7eb0 con 0x7f7dc0075790 2024-04-25T18:59:30.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.198+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc0069730 con 0x7f7dc0075790 2024-04-25T18:59:30.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.200+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7da8004750 con 0x7f7dc0075790 2024-04-25T18:59:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.202+0000 7f7dbd600700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7da40a3cd0 0x7f7da40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.202+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f7da80d0690 con 0x7f7dc0075790 2024-04-25T18:59:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.202+0000 7f7dbfe00700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7da40a3cd0 0x7f7da40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:30.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.203+0000 7f7dbfe00700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7da40a3cd0 0x7f7da40a6190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7db00098c0 tx=0x7f7db0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.207+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7da809d0c0 con 0x7f7dc0075790 2024-04-25T18:59:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.465+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7dc019c6e0 con 0x7f7da40a3cd0 2024-04-25T18:59:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.498+0000 7f7dbd600700 1 -- 172.21.15.86:0/3969562782 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7db000fd20 con 0x7f7da40a3cd0 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7da40a3cd0 msgr2=0x7f7da40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7da40a3cd0 0x7f7da40a6190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7db00098c0 tx=0x7f7db0019040 comp rx=0 tx=0).stop 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 msgr2=0x7f7dc019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 0x7f7dc019b870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e9c0 tx=0x7f7da800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 shutdown_connections 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f7da40a3cd0 0x7f7da40a6190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dc010f1b0 0x7f7dc019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dc0074e70 0x7f7dc019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 --2- 172.21.15.86:0/3969562782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0075790 0x7f7dc019b870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.501+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 >> 172.21.15.86:0/3969562782 conn(0x7f7dc00fe4c0 msgr2=0x7f7dc00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.502+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 shutdown_connections 2024-04-25T18:59:30.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:30.502+0000 7f7dc5400700 1 -- 172.21.15.86:0/3969562782 wait complete. 2024-04-25T18:59:30.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:30 smithi183 ceph-mon[32344]: pgmap v345: 97 pgs: 97 active+clean; 606 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-25T18:59:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:30 smithi194 ceph-mon[32740]: pgmap v345: 97 pgs: 97 active+clean; 606 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-25T18:59:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:30 smithi086 ceph-mon[27783]: pgmap v345: 97 pgs: 97 active+clean; 606 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-25T18:59:31.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T18:59:31.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:31 smithi086 ceph-mon[27783]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:31.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:31 smithi183 ceph-mon[32344]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:31.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:31 smithi194 ceph-mon[32740]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:32 smithi183 ceph-mon[32344]: pgmap v346: 97 pgs: 97 active+clean; 611 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-25T18:59:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:32 smithi194 ceph-mon[32740]: pgmap v346: 97 pgs: 97 active+clean; 611 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-25T18:59:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:32 smithi086 ceph-mon[27783]: pgmap v346: 97 pgs: 97 active+clean; 611 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-25T18:59:34.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.046+0000 7fbbe5200700 1 -- 172.21.15.86:0/1623394149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0109fa0 msgr2=0x7fbbe0111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.046+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1623394149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0109fa0 0x7fbbe0111b70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800b0d0 tx=0x7fbbd800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 -- 172.21.15.86:0/1623394149 shutdown_connections 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1623394149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0109fa0 0x7fbbe0111b70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1623394149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0104ec0 0x7fbbe01052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1623394149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0105870 0x7fbbe01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 -- 172.21.15.86:0/1623394149 >> 172.21.15.86:0/1623394149 conn(0x7fbbe0100750 msgr2=0x7fbbe0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 -- 172.21.15.86:0/1623394149 shutdown_connections 2024-04-25T18:59:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.047+0000 7fbbe5200700 1 -- 172.21.15.86:0/1623394149 wait complete. 2024-04-25T18:59:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 Processor -- start 2024-04-25T18:59:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 -- start start 2024-04-25T18:59:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0104ec0 0x7fbbe019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 0x7fbbe019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:34.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0109fa0 0x7fbbe019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0114e80 con 0x7fbbe0105870 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0114d00 con 0x7fbbe0109fa0 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.048+0000 7fbbe5200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0115000 con 0x7fbbe0104ec0 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 0x7fbbe019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0104ec0 0x7fbbe019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 0x7fbbe019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51308/0 (socket says 172.21.15.86:51308) 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdd400700 1 -- 172.21.15.86:0/1059710454 learned_addr learned my addr 172.21.15.86:0/1059710454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbde800700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0109fa0 0x7fbbe019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 -- 172.21.15.86:0/1059710454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0109fa0 msgr2=0x7fbbe019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0109fa0 0x7fbbe019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 -- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 msgr2=0x7fbbe019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 0x7fbbe019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 -- 172.21.15.86:0/1059710454 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe01a0300 con 0x7fbbe0104ec0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdd400700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 0x7fbbe019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.049+0000 7fbbdde00700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0104ec0 0x7fbbe019b2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e9f0 tx=0x7fbbd000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.050+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000c9c0 con 0x7fbbe0104ec0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.050+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd000cb60 con 0x7fbbe0104ec0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.050+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe01a05f0 con 0x7fbbe0104ec0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.050+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0018680 con 0x7fbbe0104ec0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.050+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe01a7e70 con 0x7fbbe0104ec0 2024-04-25T18:59:34.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.051+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe0069730 con 0x7fbbe0104ec0 2024-04-25T18:59:34.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.053+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbd0010430 con 0x7fbbe0104ec0 2024-04-25T18:59:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.054+0000 7fbbd6a00700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbc40a3cd0 0x7fbbc40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.055+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fbbd00d0980 con 0x7fbbe0104ec0 2024-04-25T18:59:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.055+0000 7fbbdd400700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbc40a3cd0 0x7fbbc40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:34.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.055+0000 7fbbdd400700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbc40a3cd0 0x7fbbc40a6190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbbe010b510 tx=0x7fbbc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:34.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.057+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbd009d2b0 con 0x7fbbe0104ec0 2024-04-25T18:59:34.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.323+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbbe0062e80 con 0x7fbbc40a3cd0 2024-04-25T18:59:34.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.338+0000 7fbbd6a00700 1 -- 172.21.15.86:0/1059710454 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbbe0062e80 con 0x7fbbc40a3cd0 2024-04-25T18:59:34.339 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/e156f586-1b61-4077-95bf-7820cb4279c7 2024-04-25T18:59:34.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbc40a3cd0 msgr2=0x7fbbc40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbc40a3cd0 0x7fbbc40a6190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbbe010b510 tx=0x7fbbc8009260 comp rx=0 tx=0).stop 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0104ec0 msgr2=0x7fbbe019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0104ec0 0x7fbbe019b2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e9f0 tx=0x7fbbd000ed00 comp rx=0 tx=0).stop 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 shutdown_connections 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fbbc40a3cd0 0x7fbbc40a6190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbe0109fa0 0x7fbbe019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbbe0104ec0 0x7fbbe019b2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 --2- 172.21.15.86:0/1059710454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe0105870 0x7fbbe019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.340+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 >> 172.21.15.86:0/1059710454 conn(0x7fbbe0100750 msgr2=0x7fbbe0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.341+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 shutdown_connections 2024-04-25T18:59:34.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:34.341+0000 7fbbe5200700 1 -- 172.21.15.86:0/1059710454 wait complete. 2024-04-25T18:59:34.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:34 smithi183 ceph-mon[32344]: pgmap v347: 97 pgs: 97 active+clean; 612 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T18:59:34.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:34 smithi194 ceph-mon[32740]: pgmap v347: 97 pgs: 97 active+clean; 612 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T18:59:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:34 smithi086 ceph-mon[27783]: pgmap v347: 97 pgs: 97 active+clean; 612 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T18:59:35.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:59:35.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:35 smithi183 ceph-mon[32344]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:35.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:35 smithi194 ceph-mon[32740]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:35.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:35 smithi086 ceph-mon[27783]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:36.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:36 smithi194 ceph-mon[32740]: pgmap v348: 97 pgs: 97 active+clean; 612 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-25T18:59:36.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:36 smithi086 ceph-mon[27783]: pgmap v348: 97 pgs: 97 active+clean; 612 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-25T18:59:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:36 smithi183 ceph-mon[32344]: pgmap v348: 97 pgs: 97 active+clean; 612 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-25T18:59:37.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T18:59:37.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T18:59:37.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T18:59:37.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T18:59:37.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T18:59:37.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T18:59:37.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T18:59:37.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T18:59:37.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T18:59:37.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-25T18:59:37.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T18:59:37.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T18:59:37.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T18:59:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 -- 172.21.15.86:0/2615952704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 msgr2=0x7fa8681052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 --2- 172.21.15.86:0/2615952704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa8681052a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8580099c0 tx=0x7fa858009cd0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 -- 172.21.15.86:0/2615952704 shutdown_connections 2024-04-25T18:59:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 --2- 172.21.15.86:0/2615952704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa868109fa0 0x7fa868111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 --2- 172.21.15.86:0/2615952704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa8681052a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 --2- 172.21.15.86:0/2615952704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa868105870 0x7fa8681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 -- 172.21.15.86:0/2615952704 >> 172.21.15.86:0/2615952704 conn(0x7fa868100750 msgr2=0x7fa868102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.784+0000 7fa86f000700 1 -- 172.21.15.86:0/2615952704 shutdown_connections 2024-04-25T18:59:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.785+0000 7fa86f000700 1 -- 172.21.15.86:0/2615952704 wait complete. 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 Processor -- start 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 -- start start 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa86819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa868105870 0x7fa86819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa868109fa0 0x7fa86819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa868114e80 con 0x7fa868104ec0 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa868114d00 con 0x7fa868109fa0 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa86f000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa868115000 con 0x7fa868105870 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa867e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa86819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa867e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa86819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51340/0 (socket says 172.21.15.86:51340) 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa867e00700 1 -- 172.21.15.86:0/470849657 learned_addr learned my addr 172.21.15.86:0/470849657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.786+0000 7fa867400700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa868105870 0x7fa86819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa86d000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa868109fa0 0x7fa86819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa867e00700 1 -- 172.21.15.86:0/470849657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa868105870 msgr2=0x7fa86819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa867e00700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa868105870 0x7fa86819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa867e00700 1 -- 172.21.15.86:0/470849657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa868109fa0 msgr2=0x7fa86819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa867e00700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa868109fa0 0x7fa86819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa867e00700 1 -- 172.21.15.86:0/470849657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8681a03b0 con 0x7fa868104ec0 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.787+0000 7fa867e00700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa86819b3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa858005910 tx=0x7fa858004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.788+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858026070 con 0x7fa868104ec0 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.788+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8681a0640 con 0x7fa868104ec0 2024-04-25T18:59:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.788+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa858003c70 con 0x7fa868104ec0 2024-04-25T18:59:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.788+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8681a7f20 con 0x7fa868104ec0 2024-04-25T18:59:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.788+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85800f9e0 con 0x7fa868104ec0 2024-04-25T18:59:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.789+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8681063a0 con 0x7fa868104ec0 2024-04-25T18:59:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.791+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa858029070 con 0x7fa868104ec0 2024-04-25T18:59:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.792+0000 7fa864c00700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa84c0a3cd0 0x7fa84c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.792+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa8580d0c60 con 0x7fa868104ec0 2024-04-25T18:59:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.793+0000 7fa867400700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa84c0a3cd0 0x7fa84c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.793+0000 7fa867400700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa84c0a3cd0 0x7fa84c0a6190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa86810b600 tx=0x7fa854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:37.797+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa858013070 con 0x7fa868104ec0 2024-04-25T18:59:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.069+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa86802df80 con 0x7fa868104ec0 2024-04-25T18:59:38.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.071+0000 7fa864c00700 1 -- 172.21.15.86:0/470849657 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11937 (secure 0 0 0) 0x7fa85809d500 con 0x7fa868104ec0 2024-04-25T18:59:38.074 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:59:38.074 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T18:59:15.320078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24478,24527,24493,24494,24509]},"id":1}]} 2024-04-25T18:59:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa84c0a3cd0 msgr2=0x7fa84c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa84c0a3cd0 0x7fa84c0a6190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa86810b600 tx=0x7fa854009260 comp rx=0 tx=0).stop 2024-04-25T18:59:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 msgr2=0x7fa86819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa86819b3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa858005910 tx=0x7fa858004270 comp rx=0 tx=0).stop 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 shutdown_connections 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa84c0a3cd0 0x7fa84c0a6190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa868109fa0 0x7fa86819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa868104ec0 0x7fa86819b3f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 --2- 172.21.15.86:0/470849657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa868105870 0x7fa86819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 >> 172.21.15.86:0/470849657 conn(0x7fa868100750 msgr2=0x7fa868102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 shutdown_connections 2024-04-25T18:59:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:38.074+0000 7fa86f000700 1 -- 172.21.15.86:0/470849657 wait complete. 2024-04-25T18:59:38.078 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T18:59:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: pgmap v350: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/470849657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:38 smithi183 ceph-mon[32344]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:59:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T18:59:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T18:59:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-04-25T18:59:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T18:59:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T18:59:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: pgmap v350: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/470849657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:38.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:38 smithi194 ceph-mon[32740]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:59:38.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T18:59:38.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T18:59:38.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: pgmap v350: 97 pgs: 97 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/470849657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:38.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:38 smithi086 ceph-mon[27783]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:59:38.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:59:39.311 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:40 smithi183 ceph-mon[32344]: pgmap v352: 97 pgs: 1 activating, 3 peering, 93 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-25T18:59:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:40 smithi194 ceph-mon[32740]: pgmap v352: 97 pgs: 1 activating, 3 peering, 93 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-25T18:59:40.811 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:40 smithi086 ceph-mon[27783]: pgmap v352: 97 pgs: 1 activating, 3 peering, 93 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-25T18:59:41.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 -- 172.21.15.86:0/63858659 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec1016c0 msgr2=0x7febec101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 --2- 172.21.15.86:0/63858659 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec1016c0 0x7febec101aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febdc0098f0 tx=0x7febdc009c00 comp rx=0 tx=0).stop 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 -- 172.21.15.86:0/63858659 shutdown_connections 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 --2- 172.21.15.86:0/63858659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febec10f270 0x7febec111660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 --2- 172.21.15.86:0/63858659 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec1016c0 0x7febec101aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 --2- 172.21.15.86:0/63858659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec102070 0x7febec10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.689+0000 7febeb200700 1 -- 172.21.15.86:0/63858659 >> 172.21.15.86:0/63858659 conn(0x7febec0fd340 msgr2=0x7febec0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.690+0000 7febeb200700 1 -- 172.21.15.86:0/63858659 shutdown_connections 2024-04-25T18:59:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.690+0000 7febeb200700 1 -- 172.21.15.86:0/63858659 wait complete. 2024-04-25T18:59:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 Processor -- start 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 -- start start 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 0x7febec19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febec102070 0x7febec19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec10f270 0x7febec19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febec114d80 con 0x7febec102070 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febec114c00 con 0x7febec1016c0 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.691+0000 7febeb200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febec114f00 con 0x7febec10f270 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febec102070 0x7febec19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 0x7febec19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 0x7febec19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:35210/0 (socket says 172.21.15.86:35210) 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 -- 172.21.15.86:0/2212258734 learned_addr learned my addr 172.21.15.86:0/2212258734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febea800700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec10f270 0x7febec19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 -- 172.21.15.86:0/2212258734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec10f270 msgr2=0x7febec19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec10f270 0x7febec19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 -- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febec102070 msgr2=0x7febec19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febec102070 0x7febec19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 -- 172.21.15.86:0/2212258734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec1a02c0 con 0x7febec1016c0 2024-04-25T18:59:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.692+0000 7febe9e00700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 0x7febec19b2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febdc00f990 tx=0x7febdc00f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.693+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc027070 con 0x7febec1016c0 2024-04-25T18:59:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.693+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febdc005570 con 0x7febec1016c0 2024-04-25T18:59:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.693+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec1a0550 con 0x7febec1016c0 2024-04-25T18:59:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.693+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc004930 con 0x7febec1016c0 2024-04-25T18:59:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.693+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec1a7e30 con 0x7febec1016c0 2024-04-25T18:59:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.694+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec06b890 con 0x7febec1016c0 2024-04-25T18:59:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.696+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7febdc02a070 con 0x7febec1016c0 2024-04-25T18:59:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.697+0000 7febe2a00700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7febd00a3cd0 0x7febd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.698+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7febdc0d1670 con 0x7febec1016c0 2024-04-25T18:59:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.698+0000 7febe3e00700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7febd00a3cd0 0x7febd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.698+0000 7febe3e00700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7febd00a3cd0 0x7febd00a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febec103120 tx=0x7febd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.700+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febdc031070 con 0x7febec1016c0 2024-04-25T18:59:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.932+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febec02df80 con 0x7febec1016c0 2024-04-25T18:59:41.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.934+0000 7febe2a00700 1 -- 172.21.15.86:0/2212258734 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22726 (secure 0 0 0) 0x7febdc013030 con 0x7febec1016c0 2024-04-25T18:59:41.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:59:41.938 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T18:59:38.305268+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:59:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.937+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7febd00a3cd0 msgr2=0x7febd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.937+0000 7febeb200700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7febd00a3cd0 0x7febd00a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febec103120 tx=0x7febd4009260 comp rx=0 tx=0).stop 2024-04-25T18:59:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.937+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 msgr2=0x7febec19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.937+0000 7febeb200700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 0x7febec19b2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febdc00f990 tx=0x7febdc00f9c0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 shutdown_connections 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7febd00a3cd0 0x7febd00a6190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febec10f270 0x7febec19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febec1016c0 0x7febec19b2b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 --2- 172.21.15.86:0/2212258734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febec102070 0x7febec19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 >> 172.21.15.86:0/2212258734 conn(0x7febec0fd340 msgr2=0x7febec0ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 shutdown_connections 2024-04-25T18:59:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:41.938+0000 7febeb200700 1 -- 172.21.15.86:0/2212258734 wait complete. 2024-04-25T18:59:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:42 smithi183 ceph-mon[32344]: pgmap v353: 97 pgs: 1 activating, 3 peering, 93 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-04-25T18:59:42.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:42 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2212258734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:59:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:42 smithi194 ceph-mon[32740]: pgmap v353: 97 pgs: 1 activating, 3 peering, 93 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-04-25T18:59:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:42 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2212258734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:59:42.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:42 smithi086 ceph-mon[27783]: pgmap v353: 97 pgs: 1 activating, 3 peering, 93 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-04-25T18:59:42.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:42 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2212258734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:59:42.847 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:59:42.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:59:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:43.660 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:43 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:43.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:43 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:43 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:44.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:44 smithi086 ceph-mon[27783]: pgmap v354: 97 pgs: 1 activating, 96 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T18:59:44.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:44 smithi183 ceph-mon[32344]: pgmap v354: 97 pgs: 1 activating, 96 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T18:59:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:44 smithi194 ceph-mon[32740]: pgmap v354: 97 pgs: 1 activating, 96 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T18:59:45.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:45 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.613 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:45 smithi086 ceph-mon[27783]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1019 B/s wr, 8 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 -- 172.21.15.86:0/537150777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 msgr2=0x7fcb7810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 --2- 172.21.15.86:0/537150777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 0x7fcb7810dcd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098f0 tx=0x7fcb64009c00 comp rx=0 tx=0).stop 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 -- 172.21.15.86:0/537150777 shutdown_connections 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 --2- 172.21.15.86:0/537150777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb78115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 --2- 172.21.15.86:0/537150777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb781092d0 0x7fcb781096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 --2- 172.21.15.86:0/537150777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 0x7fcb7810dcd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 -- 172.21.15.86:0/537150777 >> 172.21.15.86:0/537150777 conn(0x7fcb78076210 msgr2=0x7fcb78107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 -- 172.21.15.86:0/537150777 shutdown_connections 2024-04-25T18:59:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.609+0000 7fcb7e600700 1 -- 172.21.15.86:0/537150777 wait complete. 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.610+0000 7fcb7e600700 1 Processor -- start 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 -- start start 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb781092d0 0x7fcb781a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 0x7fcb781a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb7819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78119290 con 0x7fcb781092d0 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78119110 con 0x7fcb78109c80 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.611+0000 7fcb7e600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78119410 con 0x7fcb7810e3b0 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb781092d0 0x7fcb781a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb7819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb7819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36724/0 (socket says 172.21.15.86:36724) 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 -- 172.21.15.86:0/883016933 learned_addr learned my addr 172.21.15.86:0/883016933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb76a00700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 0x7fcb781a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:45.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 -- 172.21.15.86:0/883016933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 msgr2=0x7fcb781a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 0x7fcb781a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 -- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb781092d0 msgr2=0x7fcb781a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb781092d0 0x7fcb781a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 -- 172.21.15.86:0/883016933 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7819fd90 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.612+0000 7fcb77e00700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb7819f630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000ef30 tx=0x7fcb7000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.613+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb700092e0 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.613+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb781a0020 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.613+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb70004bb0 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.613+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70004020 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.613+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb781ac2f0 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.614+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb78069730 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.615+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb7001b030 con 0x7fcb7810e3b0 2024-04-25T18:59:45.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.617+0000 7fcb6fe00700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb5c0a3cd0 0x7fcb5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:45.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.617+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fcb700cffe0 con 0x7fcb7810e3b0 2024-04-25T18:59:45.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.617+0000 7fcb77400700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb5c0a3cd0 0x7fcb5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:45.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.618+0000 7fcb77400700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb5c0a3cd0 0x7fcb5c0a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009750 tx=0x7fcb68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:45.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.621+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb70022020 con 0x7fcb7810e3b0 2024-04-25T18:59:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.847+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb781a0b10 con 0x7fcb5c0a3cd0 2024-04-25T18:59:45.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.857+0000 7fcb6fe00700 1 -- 172.21.15.86:0/883016933 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259059 (secure 0 0 0) 0x7fcb781a0b10 con 0x7fcb5c0a3cd0 2024-04-25T18:59:45.861 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:59:45.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb5c0a3cd0 msgr2=0x7fcb5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb5c0a3cd0 0x7fcb5c0a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009750 tx=0x7fcb68009310 comp rx=0 tx=0).stop 2024-04-25T18:59:45.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 msgr2=0x7fcb7819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb7819f630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000ef30 tx=0x7fcb7000c3a0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 shutdown_connections 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcb5c0a3cd0 0x7fcb5c0a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb7810e3b0 0x7fcb7819f630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb781092d0 0x7fcb781a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 --2- 172.21.15.86:0/883016933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb78109c80 0x7fcb781a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.865+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 >> 172.21.15.86:0/883016933 conn(0x7fcb78076210 msgr2=0x7fcb781086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.866+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 shutdown_connections 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:45.866+0000 7fcb7e600700 1 -- 172.21.15.86:0/883016933 wait complete. 2024-04-25T18:59:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T18:59:45.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:45 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:45 smithi183 ceph-mon[32344]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1019 B/s wr, 8 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T18:59:45.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:45 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:45.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:45 smithi194 ceph-mon[32740]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1019 B/s wr, 8 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T18:59:46.897 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-04-25T18:59:45.342427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654287,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":476,"num_read_kb":359,"num_write":429,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":296,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361448,"kb_used_data":12236,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580376,"statfs":{"total":1151940427776,"available":1151570305024,"internally_reserved":0,"allocated":12529664,"data_stored":7761003,"data_compressed":176001,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5395,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.033468"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312739+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T18:59:38.312739+0000","last_peered":"2024-04-25T18:59:38.312739+0000","last_clean":"2024-04-25T18:59:38.312739+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T18:59:38.312739+0000","last_undegraded":"2024-04-25T18:59:38.312739+0000","last_fullsized":"2024-04-25T18:59:38.312739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.318737+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T18:59:38.318737+0000","last_peered":"2024-04-25T18:59:38.318737+0000","last_clean":"2024-04-25T18:59:38.318737+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T18:59:38.318737+0000","last_undegraded":"2024-04-25T18:59:38.318737+0000","last_fullsized":"2024-04-25T18:59:38.318737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319829+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T18:59:38.319829+0000","last_peered":"2024-04-25T18:59:38.319829+0000","last_clean":"2024-04-25T18:59:38.319829+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T18:59:38.319829+0000","last_undegraded":"2024-04-25T18:59:38.319829+0000","last_fullsized":"2024-04-25T18:59:38.319829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.578015+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T18:59:38.578015+0000","last_peered":"2024-04-25T18:59:38.578015+0000","last_clean":"2024-04-25T18:59:38.578015+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T18:59:38.578015+0000","last_undegraded":"2024-04-25T18:59:38.578015+0000","last_fullsized":"2024-04-25T18:59:38.578015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315769+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T18:59:38.315769+0000","last_peered":"2024-04-25T18:59:38.315769+0000","last_clean":"2024-04-25T18:59:38.315769+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T18:59:38.315769+0000","last_undegraded":"2024-04-25T18:59:38.315769+0000","last_fullsized":"2024-04-25T18:59:38.315769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.317249+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T18:59:38.317249+0000","last_peered":"2024-04-25T18:59:38.317249+0000","last_clean":"2024-04-25T18:59:38.317249+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T18:59:38.317249+0000","last_undegraded":"2024-04-25T18:59:38.317249+0000","last_fullsized":"2024-04-25T18:59:38.317249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312808+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T18:59:38.312808+0000","last_peered":"2024-04-25T18:59:38.312808+0000","last_clean":"2024-04-25T18:59:38.312808+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T18:59:38.312808+0000","last_undegraded":"2024-04-25T18:59:38.312808+0000","last_fullsized":"2024-04-25T18:59:38.312808+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714041+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T18:59:38.714041+0000","last_peered":"2024-04-25T18:59:38.714041+0000","last_clean":"2024-04-25T18:59:38.714041+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T18:59:38.714041+0000","last_undegraded":"2024-04-25T18:59:38.714041+0000","last_fullsized":"2024-04-25T18:59:38.714041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316336+0000","last_change":"2024-04-25T18:58:48.555800+0000","last_active":"2024-04-25T18:59:38.316336+0000","last_peered":"2024-04-25T18:59:38.316336+0000","last_clean":"2024-04-25T18:59:38.316336+0000","last_became_active":"2024-04-25T18:58:48.555442+0000","last_became_peered":"2024-04-25T18:58:48.555442+0000","last_unstale":"2024-04-25T18:59:38.316336+0000","last_undegraded":"2024-04-25T18:59:38.316336+0000","last_fullsized":"2024-04-25T18:59:38.316336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:46.232944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312105+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T18:59:38.312105+0000","last_peered":"2024-04-25T18:59:38.312105+0000","last_clean":"2024-04-25T18:59:38.312105+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T18:59:38.312105+0000","last_undegraded":"2024-04-25T18:59:38.312105+0000","last_fullsized":"2024-04-25T18:59:38.312105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.314097+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T18:59:38.314097+0000","last_peered":"2024-04-25T18:59:38.314097+0000","last_clean":"2024-04-25T18:59:38.314097+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T18:59:38.314097+0000","last_undegraded":"2024-04-25T18:59:38.314097+0000","last_fullsized":"2024-04-25T18:59:38.314097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577842+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T18:59:38.577842+0000","last_peered":"2024-04-25T18:59:38.577842+0000","last_clean":"2024-04-25T18:59:38.577842+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T18:59:38.577842+0000","last_undegraded":"2024-04-25T18:59:38.577842+0000","last_fullsized":"2024-04-25T18:59:38.577842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573683+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T18:59:38.573683+0000","last_peered":"2024-04-25T18:59:38.573683+0000","last_clean":"2024-04-25T18:59:38.573683+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T18:59:38.573683+0000","last_undegraded":"2024-04-25T18:59:38.573683+0000","last_fullsized":"2024-04-25T18:59:38.573683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.317529+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T18:59:38.317529+0000","last_peered":"2024-04-25T18:59:38.317529+0000","last_clean":"2024-04-25T18:59:38.317529+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T18:59:38.317529+0000","last_undegraded":"2024-04-25T18:59:38.317529+0000","last_fullsized":"2024-04-25T18:59:38.317529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316619+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T18:59:38.316619+0000","last_peered":"2024-04-25T18:59:38.316619+0000","last_clean":"2024-04-25T18:59:38.316619+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T18:59:38.316619+0000","last_undegraded":"2024-04-25T18:59:38.316619+0000","last_fullsized":"2024-04-25T18:59:38.316619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315668+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T18:59:38.315668+0000","last_peered":"2024-04-25T18:59:38.315668+0000","last_clean":"2024-04-25T18:59:38.315668+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T18:59:38.315668+0000","last_undegraded":"2024-04-25T18:59:38.315668+0000","last_fullsized":"2024-04-25T18:59:38.315668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313263+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T18:59:38.313263+0000","last_peered":"2024-04-25T18:59:38.313263+0000","last_clean":"2024-04-25T18:59:38.313263+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T18:59:38.313263+0000","last_undegraded":"2024-04-25T18:59:38.313263+0000","last_fullsized":"2024-04-25T18:59:38.313263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312805+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T18:59:38.312805+0000","last_peered":"2024-04-25T18:59:38.312805+0000","last_clean":"2024-04-25T18:59:38.312805+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T18:59:38.312805+0000","last_undegraded":"2024-04-25T18:59:38.312805+0000","last_fullsized":"2024-04-25T18:59:38.312805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310743+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T18:59:38.310743+0000","last_peered":"2024-04-25T18:59:38.310743+0000","last_clean":"2024-04-25T18:59:38.310743+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T18:59:38.310743+0000","last_undegraded":"2024-04-25T18:59:38.310743+0000","last_fullsized":"2024-04-25T18:59:38.310743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:40.573939+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T18:59:40.573939+0000","last_peered":"2024-04-25T18:59:40.573939+0000","last_clean":"2024-04-25T18:59:40.573939+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T18:59:40.573939+0000","last_undegraded":"2024-04-25T18:59:40.573939+0000","last_fullsized":"2024-04-25T18:59:40.573939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.716737+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T18:59:38.716737+0000","last_peered":"2024-04-25T18:59:38.716737+0000","last_clean":"2024-04-25T18:59:38.716737+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T18:59:38.716737+0000","last_undegraded":"2024-04-25T18:59:38.716737+0000","last_fullsized":"2024-04-25T18:59:38.716737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.716858+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T18:59:38.716858+0000","last_peered":"2024-04-25T18:59:38.716858+0000","last_clean":"2024-04-25T18:59:38.716858+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T18:59:38.716858+0000","last_undegraded":"2024-04-25T18:59:38.716858+0000","last_fullsized":"2024-04-25T18:59:38.716858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319662+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T18:59:38.319662+0000","last_peered":"2024-04-25T18:59:38.319662+0000","last_clean":"2024-04-25T18:59:38.319662+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T18:59:38.319662+0000","last_undegraded":"2024-04-25T18:59:38.319662+0000","last_fullsized":"2024-04-25T18:59:38.319662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.597805+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T18:59:38.597805+0000","last_peered":"2024-04-25T18:59:38.597805+0000","last_clean":"2024-04-25T18:59:38.597805+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T18:59:38.597805+0000","last_undegraded":"2024-04-25T18:59:38.597805+0000","last_fullsized":"2024-04-25T18:59:38.597805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315844+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T18:59:38.315844+0000","last_peered":"2024-04-25T18:59:38.315844+0000","last_clean":"2024-04-25T18:59:38.315844+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T18:59:38.315844+0000","last_undegraded":"2024-04-25T18:59:38.315844+0000","last_fullsized":"2024-04-25T18:59:38.315844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316289+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T18:59:38.316289+0000","last_peered":"2024-04-25T18:59:38.316289+0000","last_clean":"2024-04-25T18:59:38.316289+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T18:59:38.316289+0000","last_undegraded":"2024-04-25T18:59:38.316289+0000","last_fullsized":"2024-04-25T18:59:38.316289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.446104+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T18:59:38.446104+0000","last_peered":"2024-04-25T18:59:38.446104+0000","last_clean":"2024-04-25T18:59:38.446104+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T18:59:38.446104+0000","last_undegraded":"2024-04-25T18:59:38.446104+0000","last_fullsized":"2024-04-25T18:59:38.446104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:39.547768+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T18:59:39.547768+0000","last_peered":"2024-04-25T18:59:39.547768+0000","last_clean":"2024-04-25T18:59:39.547768+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T18:59:39.547768+0000","last_undegraded":"2024-04-25T18:59:39.547768+0000","last_fullsized":"2024-04-25T18:59:39.547768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315801+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T18:59:38.315801+0000","last_peered":"2024-04-25T18:59:38.315801+0000","last_clean":"2024-04-25T18:59:38.315801+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T18:59:38.315801+0000","last_undegraded":"2024-04-25T18:59:38.315801+0000","last_fullsized":"2024-04-25T18:59:38.315801+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.311758+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T18:59:38.311758+0000","last_peered":"2024-04-25T18:59:38.311758+0000","last_clean":"2024-04-25T18:59:38.311758+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T18:59:38.311758+0000","last_undegraded":"2024-04-25T18:59:38.311758+0000","last_fullsized":"2024-04-25T18:59:38.311758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445876+0000","last_change":"2024-04-25T18:58:48.565609+0000","last_active":"2024-04-25T18:59:38.445876+0000","last_peered":"2024-04-25T18:59:38.445876+0000","last_clean":"2024-04-25T18:59:38.445876+0000","last_became_active":"2024-04-25T18:58:48.565263+0000","last_became_peered":"2024-04-25T18:58:48.565263+0000","last_unstale":"2024-04-25T18:59:38.445876+0000","last_undegraded":"2024-04-25T18:59:38.445876+0000","last_fullsized":"2024-04-25T18:59:38.445876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:36.491798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573348+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T18:59:38.573348+0000","last_peered":"2024-04-25T18:59:38.573348+0000","last_clean":"2024-04-25T18:59:38.573348+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T18:59:38.573348+0000","last_undegraded":"2024-04-25T18:59:38.573348+0000","last_fullsized":"2024-04-25T18:59:38.573348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714155+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T18:59:38.714155+0000","last_peered":"2024-04-25T18:59:38.714155+0000","last_clean":"2024-04-25T18:59:38.714155+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T18:59:38.714155+0000","last_undegraded":"2024-04-25T18:59:38.714155+0000","last_fullsized":"2024-04-25T18:59:38.714155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445734+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T18:59:38.445734+0000","last_peered":"2024-04-25T18:59:38.445734+0000","last_clean":"2024-04-25T18:59:38.445734+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T18:59:38.445734+0000","last_undegraded":"2024-04-25T18:59:38.445734+0000","last_fullsized":"2024-04-25T18:59:38.445734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319372+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T18:59:38.319372+0000","last_peered":"2024-04-25T18:59:38.319372+0000","last_clean":"2024-04-25T18:59:38.319372+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T18:59:38.319372+0000","last_undegraded":"2024-04-25T18:59:38.319372+0000","last_fullsized":"2024-04-25T18:59:38.319372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:39.546782+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T18:59:39.546782+0000","last_peered":"2024-04-25T18:59:39.546782+0000","last_clean":"2024-04-25T18:59:39.546782+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T18:59:39.546782+0000","last_undegraded":"2024-04-25T18:59:39.546782+0000","last_fullsized":"2024-04-25T18:59:39.546782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316285+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T18:59:38.316285+0000","last_peered":"2024-04-25T18:59:38.316285+0000","last_clean":"2024-04-25T18:59:38.316285+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T18:59:38.316285+0000","last_undegraded":"2024-04-25T18:59:38.316285+0000","last_fullsized":"2024-04-25T18:59:38.316285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.600224+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T18:59:38.600224+0000","last_peered":"2024-04-25T18:59:38.600224+0000","last_clean":"2024-04-25T18:59:38.600224+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T18:59:38.600224+0000","last_undegraded":"2024-04-25T18:59:38.600224+0000","last_fullsized":"2024-04-25T18:59:38.600224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319691+0000","last_change":"2024-04-25T18:58:48.576775+0000","last_active":"2024-04-25T18:59:38.319691+0000","last_peered":"2024-04-25T18:59:38.319691+0000","last_clean":"2024-04-25T18:59:38.319691+0000","last_became_active":"2024-04-25T18:58:48.576115+0000","last_became_peered":"2024-04-25T18:58:48.576115+0000","last_unstale":"2024-04-25T18:59:38.319691+0000","last_undegraded":"2024-04-25T18:59:38.319691+0000","last_fullsized":"2024-04-25T18:59:38.319691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:56.752009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.446149+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T18:59:38.446149+0000","last_peered":"2024-04-25T18:59:38.446149+0000","last_clean":"2024-04-25T18:59:38.446149+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T18:59:38.446149+0000","last_undegraded":"2024-04-25T18:59:38.446149+0000","last_fullsized":"2024-04-25T18:59:38.446149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.576644+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T18:59:38.576644+0000","last_peered":"2024-04-25T18:59:38.576644+0000","last_clean":"2024-04-25T18:59:38.576644+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T18:59:38.576644+0000","last_undegraded":"2024-04-25T18:59:38.576644+0000","last_fullsized":"2024-04-25T18:59:38.576644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'30","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:39.546733+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T18:59:39.546733+0000","last_peered":"2024-04-25T18:59:39.546733+0000","last_clean":"2024-04-25T18:59:39.546733+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T18:59:39.546733+0000","last_undegraded":"2024-04-25T18:59:39.546733+0000","last_fullsized":"2024-04-25T18:59:39.546733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56090,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":87,"num_write":32,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316026+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T18:59:38.316026+0000","last_peered":"2024-04-25T18:59:38.316026+0000","last_clean":"2024-04-25T18:59:38.316026+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T18:59:38.316026+0000","last_undegraded":"2024-04-25T18:59:38.316026+0000","last_fullsized":"2024-04-25T18:59:38.316026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310457+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T18:59:38.310457+0000","last_peered":"2024-04-25T18:59:38.310457+0000","last_clean":"2024-04-25T18:59:38.310457+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T18:59:38.310457+0000","last_undegraded":"2024-04-25T18:59:38.310457+0000","last_fullsized":"2024-04-25T18:59:38.310457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573737+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T18:59:38.573737+0000","last_peered":"2024-04-25T18:59:38.573737+0000","last_clean":"2024-04-25T18:59:38.573737+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T18:59:38.573737+0000","last_undegraded":"2024-04-25T18:59:38.573737+0000","last_fullsized":"2024-04-25T18:59:38.573737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445702+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T18:59:38.445702+0000","last_peered":"2024-04-25T18:59:38.445702+0000","last_clean":"2024-04-25T18:59:38.445702+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T18:59:38.445702+0000","last_undegraded":"2024-04-25T18:59:38.445702+0000","last_fullsized":"2024-04-25T18:59:38.445702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313883+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T18:59:38.313883+0000","last_peered":"2024-04-25T18:59:38.313883+0000","last_clean":"2024-04-25T18:59:38.313883+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T18:59:38.313883+0000","last_undegraded":"2024-04-25T18:59:38.313883+0000","last_fullsized":"2024-04-25T18:59:38.313883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577573+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T18:59:38.577573+0000","last_peered":"2024-04-25T18:59:38.577573+0000","last_clean":"2024-04-25T18:59:38.577573+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T18:59:38.577573+0000","last_undegraded":"2024-04-25T18:59:38.577573+0000","last_fullsized":"2024-04-25T18:59:38.577573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312818+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T18:59:38.312818+0000","last_peered":"2024-04-25T18:59:38.312818+0000","last_clean":"2024-04-25T18:59:38.312818+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T18:59:38.312818+0000","last_undegraded":"2024-04-25T18:59:38.312818+0000","last_fullsized":"2024-04-25T18:59:38.312818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315504+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T18:59:38.315504+0000","last_peered":"2024-04-25T18:59:38.315504+0000","last_clean":"2024-04-25T18:59:38.315504+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T18:59:38.315504+0000","last_undegraded":"2024-04-25T18:59:38.315504+0000","last_fullsized":"2024-04-25T18:59:38.315504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445733+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T18:59:38.445733+0000","last_peered":"2024-04-25T18:59:38.445733+0000","last_clean":"2024-04-25T18:59:38.445733+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T18:59:38.445733+0000","last_undegraded":"2024-04-25T18:59:38.445733+0000","last_fullsized":"2024-04-25T18:59:38.445733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316086+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T18:59:38.316086+0000","last_peered":"2024-04-25T18:59:38.316086+0000","last_clean":"2024-04-25T18:59:38.316086+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T18:59:38.316086+0000","last_undegraded":"2024-04-25T18:59:38.316086+0000","last_fullsized":"2024-04-25T18:59:38.316086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.578855+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T18:59:38.578855+0000","last_peered":"2024-04-25T18:59:38.578855+0000","last_clean":"2024-04-25T18:59:38.578855+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T18:59:38.578855+0000","last_undegraded":"2024-04-25T18:59:38.578855+0000","last_fullsized":"2024-04-25T18:59:38.578855+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312363+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T18:59:38.312363+0000","last_peered":"2024-04-25T18:59:38.312363+0000","last_clean":"2024-04-25T18:59:38.312363+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T18:59:38.312363+0000","last_undegraded":"2024-04-25T18:59:38.312363+0000","last_fullsized":"2024-04-25T18:59:38.312363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.311954+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T18:59:38.311954+0000","last_peered":"2024-04-25T18:59:38.311954+0000","last_clean":"2024-04-25T18:59:38.311954+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T18:59:38.311954+0000","last_undegraded":"2024-04-25T18:59:38.311954+0000","last_fullsized":"2024-04-25T18:59:38.311954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:39.572452+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T18:59:39.572452+0000","last_peered":"2024-04-25T18:59:39.572452+0000","last_clean":"2024-04-25T18:59:39.572452+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T18:59:39.572452+0000","last_undegraded":"2024-04-25T18:59:39.572452+0000","last_fullsized":"2024-04-25T18:59:39.572452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.311758+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T18:59:38.311758+0000","last_peered":"2024-04-25T18:59:38.311758+0000","last_clean":"2024-04-25T18:59:38.311758+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T18:59:38.311758+0000","last_undegraded":"2024-04-25T18:59:38.311758+0000","last_fullsized":"2024-04-25T18:59:38.311758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.578321+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T18:59:38.578321+0000","last_peered":"2024-04-25T18:59:38.578321+0000","last_clean":"2024-04-25T18:59:38.578321+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T18:59:38.578321+0000","last_undegraded":"2024-04-25T18:59:38.578321+0000","last_fullsized":"2024-04-25T18:59:38.578321+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:43.588845+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T18:59:43.588845+0000","last_peered":"2024-04-25T18:59:43.588845+0000","last_clean":"2024-04-25T18:59:43.588845+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T18:59:43.588845+0000","last_undegraded":"2024-04-25T18:59:43.588845+0000","last_fullsized":"2024-04-25T18:59:43.588845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.715608+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T18:59:38.715608+0000","last_peered":"2024-04-25T18:59:38.715608+0000","last_clean":"2024-04-25T18:59:38.715608+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T18:59:38.715608+0000","last_undegraded":"2024-04-25T18:59:38.715608+0000","last_fullsized":"2024-04-25T18:59:38.715608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577583+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T18:59:38.577583+0000","last_peered":"2024-04-25T18:59:38.577583+0000","last_clean":"2024-04-25T18:59:38.577583+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T18:59:38.577583+0000","last_undegraded":"2024-04-25T18:59:38.577583+0000","last_fullsized":"2024-04-25T18:59:38.577583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313250+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T18:59:38.313250+0000","last_peered":"2024-04-25T18:59:38.313250+0000","last_clean":"2024-04-25T18:59:38.313250+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T18:59:38.313250+0000","last_undegraded":"2024-04-25T18:59:38.313250+0000","last_fullsized":"2024-04-25T18:59:38.313250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573152+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T18:59:38.573152+0000","last_peered":"2024-04-25T18:59:38.573152+0000","last_clean":"2024-04-25T18:59:38.573152+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T18:59:38.573152+0000","last_undegraded":"2024-04-25T18:59:38.573152+0000","last_fullsized":"2024-04-25T18:59:38.573152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313159+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T18:59:38.313159+0000","last_peered":"2024-04-25T18:59:38.313159+0000","last_clean":"2024-04-25T18:59:38.313159+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T18:59:38.313159+0000","last_undegraded":"2024-04-25T18:59:38.313159+0000","last_fullsized":"2024-04-25T18:59:38.313159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.329411+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T18:59:38.329411+0000","last_peered":"2024-04-25T18:59:38.329411+0000","last_clean":"2024-04-25T18:59:38.329411+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T18:59:38.329411+0000","last_undegraded":"2024-04-25T18:59:38.329411+0000","last_fullsized":"2024-04-25T18:59:38.329411+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573699+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T18:59:38.573699+0000","last_peered":"2024-04-25T18:59:38.573699+0000","last_clean":"2024-04-25T18:59:38.573699+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T18:59:38.573699+0000","last_undegraded":"2024-04-25T18:59:38.573699+0000","last_fullsized":"2024-04-25T18:59:38.573699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313143+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T18:59:38.313143+0000","last_peered":"2024-04-25T18:59:38.313143+0000","last_clean":"2024-04-25T18:59:38.313143+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T18:59:38.313143+0000","last_undegraded":"2024-04-25T18:59:38.313143+0000","last_fullsized":"2024-04-25T18:59:38.313143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.717054+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T18:59:38.717054+0000","last_peered":"2024-04-25T18:59:38.717054+0000","last_clean":"2024-04-25T18:59:38.717054+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T18:59:38.717054+0000","last_undegraded":"2024-04-25T18:59:38.717054+0000","last_fullsized":"2024-04-25T18:59:38.717054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577811+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T18:59:38.577811+0000","last_peered":"2024-04-25T18:59:38.577811+0000","last_clean":"2024-04-25T18:59:38.577811+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T18:59:38.577811+0000","last_undegraded":"2024-04-25T18:59:38.577811+0000","last_fullsized":"2024-04-25T18:59:38.577811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.318862+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T18:59:38.318862+0000","last_peered":"2024-04-25T18:59:38.318862+0000","last_clean":"2024-04-25T18:59:38.318862+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T18:59:38.318862+0000","last_undegraded":"2024-04-25T18:59:38.318862+0000","last_fullsized":"2024-04-25T18:59:38.318862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315691+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T18:59:38.315691+0000","last_peered":"2024-04-25T18:59:38.315691+0000","last_clean":"2024-04-25T18:59:38.315691+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T18:59:38.315691+0000","last_undegraded":"2024-04-25T18:59:38.315691+0000","last_fullsized":"2024-04-25T18:59:38.315691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315680+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T18:59:38.315680+0000","last_peered":"2024-04-25T18:59:38.315680+0000","last_clean":"2024-04-25T18:59:38.315680+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T18:59:38.315680+0000","last_undegraded":"2024-04-25T18:59:38.315680+0000","last_fullsized":"2024-04-25T18:59:38.315680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714907+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T18:59:38.714907+0000","last_peered":"2024-04-25T18:59:38.714907+0000","last_clean":"2024-04-25T18:59:38.714907+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T18:59:38.714907+0000","last_undegraded":"2024-04-25T18:59:38.714907+0000","last_fullsized":"2024-04-25T18:59:38.714907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312210+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T18:59:38.312210+0000","last_peered":"2024-04-25T18:59:38.312210+0000","last_clean":"2024-04-25T18:59:38.312210+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T18:59:38.312210+0000","last_undegraded":"2024-04-25T18:59:38.312210+0000","last_fullsized":"2024-04-25T18:59:38.312210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310572+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T18:59:38.310572+0000","last_peered":"2024-04-25T18:59:38.310572+0000","last_clean":"2024-04-25T18:59:38.310572+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T18:59:38.310572+0000","last_undegraded":"2024-04-25T18:59:38.310572+0000","last_fullsized":"2024-04-25T18:59:38.310572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.715878+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T18:59:38.715878+0000","last_peered":"2024-04-25T18:59:38.715878+0000","last_clean":"2024-04-25T18:59:38.715878+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T18:59:38.715878+0000","last_undegraded":"2024-04-25T18:59:38.715878+0000","last_fullsized":"2024-04-25T18:59:38.715878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316099+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T18:59:38.316099+0000","last_peered":"2024-04-25T18:59:38.316099+0000","last_clean":"2024-04-25T18:59:38.316099+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T18:59:38.316099+0000","last_undegraded":"2024-04-25T18:59:38.316099+0000","last_fullsized":"2024-04-25T18:59:38.316099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313512+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T18:59:38.313512+0000","last_peered":"2024-04-25T18:59:38.313512+0000","last_clean":"2024-04-25T18:59:38.313512+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T18:59:38.313512+0000","last_undegraded":"2024-04-25T18:59:38.313512+0000","last_fullsized":"2024-04-25T18:59:38.313512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:41.550689+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T18:59:41.550689+0000","last_peered":"2024-04-25T18:59:41.550689+0000","last_clean":"2024-04-25T18:59:41.550689+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T18:59:41.550689+0000","last_undegraded":"2024-04-25T18:59:41.550689+0000","last_fullsized":"2024-04-25T18:59:41.550689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.314401+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T18:59:38.314401+0000","last_peered":"2024-04-25T18:59:38.314401+0000","last_clean":"2024-04-25T18:59:38.314401+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T18:59:38.314401+0000","last_undegraded":"2024-04-25T18:59:38.314401+0000","last_fullsized":"2024-04-25T18:59:38.314401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445972+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T18:59:38.445972+0000","last_peered":"2024-04-25T18:59:38.445972+0000","last_clean":"2024-04-25T18:59:38.445972+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T18:59:38.445972+0000","last_undegraded":"2024-04-25T18:59:38.445972+0000","last_fullsized":"2024-04-25T18:59:38.445972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313856+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T18:59:38.313856+0000","last_peered":"2024-04-25T18:59:38.313856+0000","last_clean":"2024-04-25T18:59:38.313856+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T18:59:38.313856+0000","last_undegraded":"2024-04-25T18:59:38.313856+0000","last_fullsized":"2024-04-25T18:59:38.313856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573746+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T18:59:38.573746+0000","last_peered":"2024-04-25T18:59:38.573746+0000","last_clean":"2024-04-25T18:59:38.573746+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T18:59:38.573746+0000","last_undegraded":"2024-04-25T18:59:38.573746+0000","last_fullsized":"2024-04-25T18:59:38.573746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.314001+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T18:59:38.314001+0000","last_peered":"2024-04-25T18:59:38.314001+0000","last_clean":"2024-04-25T18:59:38.314001+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T18:59:38.314001+0000","last_undegraded":"2024-04-25T18:59:38.314001+0000","last_fullsized":"2024-04-25T18:59:38.314001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312268+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T18:59:38.312268+0000","last_peered":"2024-04-25T18:59:38.312268+0000","last_clean":"2024-04-25T18:59:38.312268+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T18:59:38.312268+0000","last_undegraded":"2024-04-25T18:59:38.312268+0000","last_fullsized":"2024-04-25T18:59:38.312268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310371+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T18:59:38.310371+0000","last_peered":"2024-04-25T18:59:38.310371+0000","last_clean":"2024-04-25T18:59:38.310371+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T18:59:38.310371+0000","last_undegraded":"2024-04-25T18:59:38.310371+0000","last_fullsized":"2024-04-25T18:59:38.310371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310299+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T18:59:38.310299+0000","last_peered":"2024-04-25T18:59:38.310299+0000","last_clean":"2024-04-25T18:59:38.310299+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T18:59:38.310299+0000","last_undegraded":"2024-04-25T18:59:38.310299+0000","last_fullsized":"2024-04-25T18:59:38.310299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.575702+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T18:59:38.575702+0000","last_peered":"2024-04-25T18:59:38.575702+0000","last_clean":"2024-04-25T18:59:38.575702+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T18:59:38.575702+0000","last_undegraded":"2024-04-25T18:59:38.575702+0000","last_fullsized":"2024-04-25T18:59:38.575702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573803+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T18:59:38.573803+0000","last_peered":"2024-04-25T18:59:38.573803+0000","last_clean":"2024-04-25T18:59:38.573803+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T18:59:38.573803+0000","last_undegraded":"2024-04-25T18:59:38.573803+0000","last_fullsized":"2024-04-25T18:59:38.573803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313511+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T18:59:38.313511+0000","last_peered":"2024-04-25T18:59:38.313511+0000","last_clean":"2024-04-25T18:59:38.313511+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T18:59:38.313511+0000","last_undegraded":"2024-04-25T18:59:38.313511+0000","last_fullsized":"2024-04-25T18:59:38.313511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310766+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T18:59:38.310766+0000","last_peered":"2024-04-25T18:59:38.310766+0000","last_clean":"2024-04-25T18:59:38.310766+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T18:59:38.310766+0000","last_undegraded":"2024-04-25T18:59:38.310766+0000","last_fullsized":"2024-04-25T18:59:38.310766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.716756+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T18:59:38.716756+0000","last_peered":"2024-04-25T18:59:38.716756+0000","last_clean":"2024-04-25T18:59:38.716756+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T18:59:38.716756+0000","last_undegraded":"2024-04-25T18:59:38.716756+0000","last_fullsized":"2024-04-25T18:59:38.716756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445661+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T18:59:38.445661+0000","last_peered":"2024-04-25T18:59:38.445661+0000","last_clean":"2024-04-25T18:59:38.445661+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T18:59:38.445661+0000","last_undegraded":"2024-04-25T18:59:38.445661+0000","last_fullsized":"2024-04-25T18:59:38.445661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714678+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T18:59:38.714678+0000","last_peered":"2024-04-25T18:59:38.714678+0000","last_clean":"2024-04-25T18:59:38.714678+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T18:59:38.714678+0000","last_undegraded":"2024-04-25T18:59:38.714678+0000","last_fullsized":"2024-04-25T18:59:38.714678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315904+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T18:59:38.315904+0000","last_peered":"2024-04-25T18:59:38.315904+0000","last_clean":"2024-04-25T18:59:38.315904+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T18:59:38.315904+0000","last_undegraded":"2024-04-25T18:59:38.315904+0000","last_fullsized":"2024-04-25T18:59:38.315904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573957+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T18:59:38.573957+0000","last_peered":"2024-04-25T18:59:38.573957+0000","last_clean":"2024-04-25T18:59:38.573957+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T18:59:38.573957+0000","last_undegraded":"2024-04-25T18:59:38.573957+0000","last_fullsized":"2024-04-25T18:59:38.573957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312474+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T18:59:38.312474+0000","last_peered":"2024-04-25T18:59:38.312474+0000","last_clean":"2024-04-25T18:59:38.312474+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T18:59:38.312474+0000","last_undegraded":"2024-04-25T18:59:38.312474+0000","last_fullsized":"2024-04-25T18:59:38.312474+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279982,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63685,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":340,"num_read_kb":240,"num_write":246,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":199569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":483773,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1146880,"data_stored":1131171,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":483591,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483857,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1077269,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":483699,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":544959,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":484647,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":545179,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":483133,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483560,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":1076165,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:59:46.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:59:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:46 smithi183 ceph-mon[32344]: from='client.24640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:46 smithi194 ceph-mon[32740]: from='client.24640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:46.997 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:46 smithi086 ceph-mon[27783]: from='client.24640 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:47.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:47 smithi086 ceph-mon[27783]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T18:59:47.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:47 smithi183 ceph-mon[32344]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T18:59:47.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:47 smithi194 ceph-mon[32740]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T18:59:49.063 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:49.064 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:48 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:49.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:49.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:48 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:49.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:49.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:48 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.549+0000 7fe8dc800700 1 -- 172.21.15.86:0/664074541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102a90 msgr2=0x7fe8d4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.549+0000 7fe8dc800700 1 --2- 172.21.15.86:0/664074541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102a90 0x7fe8d4102f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009990 tx=0x7fe8c4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 -- 172.21.15.86:0/664074541 shutdown_connections 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 --2- 172.21.15.86:0/664074541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4103450 0x7fe8d410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 --2- 172.21.15.86:0/664074541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8d4108a50 0x7fe8d4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 --2- 172.21.15.86:0/664074541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4102a90 0x7fe8d4102f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 -- 172.21.15.86:0/664074541 >> 172.21.15.86:0/664074541 conn(0x7fe8d4079ba0 msgr2=0x7fe8d4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 -- 172.21.15.86:0/664074541 shutdown_connections 2024-04-25T18:59:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.550+0000 7fe8dc800700 1 -- 172.21.15.86:0/664074541 wait complete. 2024-04-25T18:59:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.551+0000 7fe8dc800700 1 Processor -- start 2024-04-25T18:59:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.551+0000 7fe8dc800700 1 -- start start 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8dc800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102a90 0x7fe8d4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8dc800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 0x7fe8d4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8dc800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8d4108a50 0x7fe8d40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8dc800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4112c50 con 0x7fe8d4103450 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8dc800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4112ad0 con 0x7fe8d4102a90 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8dc800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4112dd0 con 0x7fe8d4108a50 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 0x7fe8d4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8d9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102a90 0x7fe8d4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8d9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 0x7fe8d4075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52984/0 (socket says 172.21.15.86:52984) 2024-04-25T18:59:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.552+0000 7fe8d9400700 1 -- 172.21.15.86:0/2581630241 learned_addr learned my addr 172.21.15.86:0/2581630241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8da800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8d4108a50 0x7fe8d40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9400700 1 -- 172.21.15.86:0/2581630241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8d4108a50 msgr2=0x7fe8d40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9400700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8d4108a50 0x7fe8d40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9400700 1 -- 172.21.15.86:0/2581630241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102a90 msgr2=0x7fe8d4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9400700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102a90 0x7fe8d4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9400700 1 -- 172.21.15.86:0/2581630241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d41a76f0 con 0x7fe8d4103450 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9e00700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102a90 0x7fe8d4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.553+0000 7fe8d9400700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 0x7fe8d4075280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4005910 tx=0x7fe8c400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.554+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4027070 con 0x7fe8d4103450 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.554+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c4022470 con 0x7fe8d4103450 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.554+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41a7980 con 0x7fe8d4103450 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.554+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4017650 con 0x7fe8d4103450 2024-04-25T18:59:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.554+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a7eb0 con 0x7fe8d4103450 2024-04-25T18:59:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.555+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d4069730 con 0x7fe8d4103450 2024-04-25T18:59:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.557+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe8c402a070 con 0x7fe8d4103450 2024-04-25T18:59:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.558+0000 7fe8cea00700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe8bc0a3d50 0x7fe8bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.559+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fe8c40d12c0 con 0x7fe8d4103450 2024-04-25T18:59:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.559+0000 7fe8d9e00700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe8bc0a3d50 0x7fe8bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.560+0000 7fe8d9e00700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe8bc0a3d50 0x7fe8bc0a6210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009fd0 tx=0x7fe8c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:49.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.564+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8c4013070 con 0x7fe8d4103450 2024-04-25T18:59:49.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.784+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8d4062e80 con 0x7fe8bc0a3d50 2024-04-25T18:59:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.810+0000 7fe8cea00700 1 -- 172.21.15.86:0/2581630241 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259037 (secure 0 0 0) 0x7fe8d4062e80 con 0x7fe8bc0a3d50 2024-04-25T18:59:49.813 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:59:49.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe8bc0a3d50 msgr2=0x7fe8bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:49.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe8bc0a3d50 0x7fe8bc0a6210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009fd0 tx=0x7fe8c8009260 comp rx=0 tx=0).stop 2024-04-25T18:59:49.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 msgr2=0x7fe8d4075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:49.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 0x7fe8d4075280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4005910 tx=0x7fe8c400b670 comp rx=0 tx=0).stop 2024-04-25T18:59:49.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 shutdown_connections 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe8bc0a3d50 0x7fe8bc0a6210 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8d4108a50 0x7fe8d40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102a90 0x7fe8d4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 --2- 172.21.15.86:0/2581630241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8d4103450 0x7fe8d4075280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.817+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 >> 172.21.15.86:0/2581630241 conn(0x7fe8d4079ba0 msgr2=0x7fe8d4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.818+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 shutdown_connections 2024-04-25T18:59:49.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:49.818+0000 7fe8dc800700 1 -- 172.21.15.86:0/2581630241 wait complete. 2024-04-25T18:59:49.821 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T18:59:50.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:50.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:50.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:49 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:50.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:49 smithi086 ceph-mon[27783]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T18:59:50.133 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:50.133 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:50.133 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:49 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:50.133 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:49 smithi194 ceph-mon[32740]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T18:59:50.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:49 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T18:59:50.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:49 smithi183 ceph-mon[32344]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T18:59:50.650 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-04-25T18:59:49.343408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658544,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":522,"num_read_kb":384,"num_write":430,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361444,"kb_used_data":12240,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580380,"statfs":{"total":1151940427776,"available":1151570309120,"internally_reserved":0,"allocated":12533760,"data_stored":7765260,"data_compressed":176001,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":58,"num_write":6,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.034350"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312739+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T18:59:38.312739+0000","last_peered":"2024-04-25T18:59:38.312739+0000","last_clean":"2024-04-25T18:59:38.312739+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T18:59:38.312739+0000","last_undegraded":"2024-04-25T18:59:38.312739+0000","last_fullsized":"2024-04-25T18:59:38.312739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.318737+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T18:59:38.318737+0000","last_peered":"2024-04-25T18:59:38.318737+0000","last_clean":"2024-04-25T18:59:38.318737+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T18:59:38.318737+0000","last_undegraded":"2024-04-25T18:59:38.318737+0000","last_fullsized":"2024-04-25T18:59:38.318737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319829+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T18:59:38.319829+0000","last_peered":"2024-04-25T18:59:38.319829+0000","last_clean":"2024-04-25T18:59:38.319829+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T18:59:38.319829+0000","last_undegraded":"2024-04-25T18:59:38.319829+0000","last_fullsized":"2024-04-25T18:59:38.319829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.578015+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T18:59:38.578015+0000","last_peered":"2024-04-25T18:59:38.578015+0000","last_clean":"2024-04-25T18:59:38.578015+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T18:59:38.578015+0000","last_undegraded":"2024-04-25T18:59:38.578015+0000","last_fullsized":"2024-04-25T18:59:38.578015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315769+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T18:59:38.315769+0000","last_peered":"2024-04-25T18:59:38.315769+0000","last_clean":"2024-04-25T18:59:38.315769+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T18:59:38.315769+0000","last_undegraded":"2024-04-25T18:59:38.315769+0000","last_fullsized":"2024-04-25T18:59:38.315769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.317249+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T18:59:38.317249+0000","last_peered":"2024-04-25T18:59:38.317249+0000","last_clean":"2024-04-25T18:59:38.317249+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T18:59:38.317249+0000","last_undegraded":"2024-04-25T18:59:38.317249+0000","last_fullsized":"2024-04-25T18:59:38.317249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312808+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T18:59:38.312808+0000","last_peered":"2024-04-25T18:59:38.312808+0000","last_clean":"2024-04-25T18:59:38.312808+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T18:59:38.312808+0000","last_undegraded":"2024-04-25T18:59:38.312808+0000","last_fullsized":"2024-04-25T18:59:38.312808+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714041+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T18:59:38.714041+0000","last_peered":"2024-04-25T18:59:38.714041+0000","last_clean":"2024-04-25T18:59:38.714041+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T18:59:38.714041+0000","last_undegraded":"2024-04-25T18:59:38.714041+0000","last_fullsized":"2024-04-25T18:59:38.714041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316336+0000","last_change":"2024-04-25T18:58:48.555800+0000","last_active":"2024-04-25T18:59:38.316336+0000","last_peered":"2024-04-25T18:59:38.316336+0000","last_clean":"2024-04-25T18:59:38.316336+0000","last_became_active":"2024-04-25T18:58:48.555442+0000","last_became_peered":"2024-04-25T18:58:48.555442+0000","last_unstale":"2024-04-25T18:59:38.316336+0000","last_undegraded":"2024-04-25T18:59:38.316336+0000","last_fullsized":"2024-04-25T18:59:38.316336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:46.232944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312105+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T18:59:38.312105+0000","last_peered":"2024-04-25T18:59:38.312105+0000","last_clean":"2024-04-25T18:59:38.312105+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T18:59:38.312105+0000","last_undegraded":"2024-04-25T18:59:38.312105+0000","last_fullsized":"2024-04-25T18:59:38.312105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.314097+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T18:59:38.314097+0000","last_peered":"2024-04-25T18:59:38.314097+0000","last_clean":"2024-04-25T18:59:38.314097+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T18:59:38.314097+0000","last_undegraded":"2024-04-25T18:59:38.314097+0000","last_fullsized":"2024-04-25T18:59:38.314097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577842+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T18:59:38.577842+0000","last_peered":"2024-04-25T18:59:38.577842+0000","last_clean":"2024-04-25T18:59:38.577842+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T18:59:38.577842+0000","last_undegraded":"2024-04-25T18:59:38.577842+0000","last_fullsized":"2024-04-25T18:59:38.577842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573683+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T18:59:38.573683+0000","last_peered":"2024-04-25T18:59:38.573683+0000","last_clean":"2024-04-25T18:59:38.573683+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T18:59:38.573683+0000","last_undegraded":"2024-04-25T18:59:38.573683+0000","last_fullsized":"2024-04-25T18:59:38.573683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.317529+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T18:59:38.317529+0000","last_peered":"2024-04-25T18:59:38.317529+0000","last_clean":"2024-04-25T18:59:38.317529+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T18:59:38.317529+0000","last_undegraded":"2024-04-25T18:59:38.317529+0000","last_fullsized":"2024-04-25T18:59:38.317529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316619+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T18:59:38.316619+0000","last_peered":"2024-04-25T18:59:38.316619+0000","last_clean":"2024-04-25T18:59:38.316619+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T18:59:38.316619+0000","last_undegraded":"2024-04-25T18:59:38.316619+0000","last_fullsized":"2024-04-25T18:59:38.316619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315668+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T18:59:38.315668+0000","last_peered":"2024-04-25T18:59:38.315668+0000","last_clean":"2024-04-25T18:59:38.315668+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T18:59:38.315668+0000","last_undegraded":"2024-04-25T18:59:38.315668+0000","last_fullsized":"2024-04-25T18:59:38.315668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313263+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T18:59:38.313263+0000","last_peered":"2024-04-25T18:59:38.313263+0000","last_clean":"2024-04-25T18:59:38.313263+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T18:59:38.313263+0000","last_undegraded":"2024-04-25T18:59:38.313263+0000","last_fullsized":"2024-04-25T18:59:38.313263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312805+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T18:59:38.312805+0000","last_peered":"2024-04-25T18:59:38.312805+0000","last_clean":"2024-04-25T18:59:38.312805+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T18:59:38.312805+0000","last_undegraded":"2024-04-25T18:59:38.312805+0000","last_fullsized":"2024-04-25T18:59:38.312805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310743+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T18:59:38.310743+0000","last_peered":"2024-04-25T18:59:38.310743+0000","last_clean":"2024-04-25T18:59:38.310743+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T18:59:38.310743+0000","last_undegraded":"2024-04-25T18:59:38.310743+0000","last_fullsized":"2024-04-25T18:59:38.310743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:45.584494+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T18:59:45.584494+0000","last_peered":"2024-04-25T18:59:45.584494+0000","last_clean":"2024-04-25T18:59:45.584494+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T18:59:45.584494+0000","last_undegraded":"2024-04-25T18:59:45.584494+0000","last_fullsized":"2024-04-25T18:59:45.584494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.716737+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T18:59:38.716737+0000","last_peered":"2024-04-25T18:59:38.716737+0000","last_clean":"2024-04-25T18:59:38.716737+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T18:59:38.716737+0000","last_undegraded":"2024-04-25T18:59:38.716737+0000","last_fullsized":"2024-04-25T18:59:38.716737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.716858+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T18:59:38.716858+0000","last_peered":"2024-04-25T18:59:38.716858+0000","last_clean":"2024-04-25T18:59:38.716858+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T18:59:38.716858+0000","last_undegraded":"2024-04-25T18:59:38.716858+0000","last_fullsized":"2024-04-25T18:59:38.716858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319662+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T18:59:38.319662+0000","last_peered":"2024-04-25T18:59:38.319662+0000","last_clean":"2024-04-25T18:59:38.319662+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T18:59:38.319662+0000","last_undegraded":"2024-04-25T18:59:38.319662+0000","last_fullsized":"2024-04-25T18:59:38.319662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.597805+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T18:59:38.597805+0000","last_peered":"2024-04-25T18:59:38.597805+0000","last_clean":"2024-04-25T18:59:38.597805+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T18:59:38.597805+0000","last_undegraded":"2024-04-25T18:59:38.597805+0000","last_fullsized":"2024-04-25T18:59:38.597805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315844+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T18:59:38.315844+0000","last_peered":"2024-04-25T18:59:38.315844+0000","last_clean":"2024-04-25T18:59:38.315844+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T18:59:38.315844+0000","last_undegraded":"2024-04-25T18:59:38.315844+0000","last_fullsized":"2024-04-25T18:59:38.315844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316289+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T18:59:38.316289+0000","last_peered":"2024-04-25T18:59:38.316289+0000","last_clean":"2024-04-25T18:59:38.316289+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T18:59:38.316289+0000","last_undegraded":"2024-04-25T18:59:38.316289+0000","last_fullsized":"2024-04-25T18:59:38.316289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.446104+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T18:59:38.446104+0000","last_peered":"2024-04-25T18:59:38.446104+0000","last_clean":"2024-04-25T18:59:38.446104+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T18:59:38.446104+0000","last_undegraded":"2024-04-25T18:59:38.446104+0000","last_fullsized":"2024-04-25T18:59:38.446104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":70,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:44.555215+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T18:59:44.555215+0000","last_peered":"2024-04-25T18:59:44.555215+0000","last_clean":"2024-04-25T18:59:44.555215+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T18:59:44.555215+0000","last_undegraded":"2024-04-25T18:59:44.555215+0000","last_fullsized":"2024-04-25T18:59:44.555215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315801+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T18:59:38.315801+0000","last_peered":"2024-04-25T18:59:38.315801+0000","last_clean":"2024-04-25T18:59:38.315801+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T18:59:38.315801+0000","last_undegraded":"2024-04-25T18:59:38.315801+0000","last_fullsized":"2024-04-25T18:59:38.315801+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.311758+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T18:59:38.311758+0000","last_peered":"2024-04-25T18:59:38.311758+0000","last_clean":"2024-04-25T18:59:38.311758+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T18:59:38.311758+0000","last_undegraded":"2024-04-25T18:59:38.311758+0000","last_fullsized":"2024-04-25T18:59:38.311758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445876+0000","last_change":"2024-04-25T18:58:48.565609+0000","last_active":"2024-04-25T18:59:38.445876+0000","last_peered":"2024-04-25T18:59:38.445876+0000","last_clean":"2024-04-25T18:59:38.445876+0000","last_became_active":"2024-04-25T18:58:48.565263+0000","last_became_peered":"2024-04-25T18:58:48.565263+0000","last_unstale":"2024-04-25T18:59:38.445876+0000","last_undegraded":"2024-04-25T18:59:38.445876+0000","last_fullsized":"2024-04-25T18:59:38.445876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:36.491798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573348+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T18:59:38.573348+0000","last_peered":"2024-04-25T18:59:38.573348+0000","last_clean":"2024-04-25T18:59:38.573348+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T18:59:38.573348+0000","last_undegraded":"2024-04-25T18:59:38.573348+0000","last_fullsized":"2024-04-25T18:59:38.573348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714155+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T18:59:38.714155+0000","last_peered":"2024-04-25T18:59:38.714155+0000","last_clean":"2024-04-25T18:59:38.714155+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T18:59:38.714155+0000","last_undegraded":"2024-04-25T18:59:38.714155+0000","last_fullsized":"2024-04-25T18:59:38.714155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445734+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T18:59:38.445734+0000","last_peered":"2024-04-25T18:59:38.445734+0000","last_clean":"2024-04-25T18:59:38.445734+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T18:59:38.445734+0000","last_undegraded":"2024-04-25T18:59:38.445734+0000","last_fullsized":"2024-04-25T18:59:38.445734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319372+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T18:59:38.319372+0000","last_peered":"2024-04-25T18:59:38.319372+0000","last_clean":"2024-04-25T18:59:38.319372+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T18:59:38.319372+0000","last_undegraded":"2024-04-25T18:59:38.319372+0000","last_fullsized":"2024-04-25T18:59:38.319372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":98,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:44.554368+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T18:59:44.554368+0000","last_peered":"2024-04-25T18:59:44.554368+0000","last_clean":"2024-04-25T18:59:44.554368+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T18:59:44.554368+0000","last_undegraded":"2024-04-25T18:59:44.554368+0000","last_fullsized":"2024-04-25T18:59:44.554368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316285+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T18:59:38.316285+0000","last_peered":"2024-04-25T18:59:38.316285+0000","last_clean":"2024-04-25T18:59:38.316285+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T18:59:38.316285+0000","last_undegraded":"2024-04-25T18:59:38.316285+0000","last_fullsized":"2024-04-25T18:59:38.316285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.600224+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T18:59:38.600224+0000","last_peered":"2024-04-25T18:59:38.600224+0000","last_clean":"2024-04-25T18:59:38.600224+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T18:59:38.600224+0000","last_undegraded":"2024-04-25T18:59:38.600224+0000","last_fullsized":"2024-04-25T18:59:38.600224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.319691+0000","last_change":"2024-04-25T18:58:48.576775+0000","last_active":"2024-04-25T18:59:38.319691+0000","last_peered":"2024-04-25T18:59:38.319691+0000","last_clean":"2024-04-25T18:59:38.319691+0000","last_became_active":"2024-04-25T18:58:48.576115+0000","last_became_peered":"2024-04-25T18:58:48.576115+0000","last_unstale":"2024-04-25T18:59:38.319691+0000","last_undegraded":"2024-04-25T18:59:38.319691+0000","last_fullsized":"2024-04-25T18:59:38.319691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:56.752009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.446149+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T18:59:38.446149+0000","last_peered":"2024-04-25T18:59:38.446149+0000","last_clean":"2024-04-25T18:59:38.446149+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T18:59:38.446149+0000","last_undegraded":"2024-04-25T18:59:38.446149+0000","last_fullsized":"2024-04-25T18:59:38.446149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.576644+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T18:59:38.576644+0000","last_peered":"2024-04-25T18:59:38.576644+0000","last_clean":"2024-04-25T18:59:38.576644+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T18:59:38.576644+0000","last_undegraded":"2024-04-25T18:59:38.576644+0000","last_fullsized":"2024-04-25T18:59:38.576644+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:44.554316+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T18:59:44.554316+0000","last_peered":"2024-04-25T18:59:44.554316+0000","last_clean":"2024-04-25T18:59:44.554316+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T18:59:44.554316+0000","last_undegraded":"2024-04-25T18:59:44.554316+0000","last_fullsized":"2024-04-25T18:59:44.554316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":97,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316026+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T18:59:38.316026+0000","last_peered":"2024-04-25T18:59:38.316026+0000","last_clean":"2024-04-25T18:59:38.316026+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T18:59:38.316026+0000","last_undegraded":"2024-04-25T18:59:38.316026+0000","last_fullsized":"2024-04-25T18:59:38.316026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310457+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T18:59:38.310457+0000","last_peered":"2024-04-25T18:59:38.310457+0000","last_clean":"2024-04-25T18:59:38.310457+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T18:59:38.310457+0000","last_undegraded":"2024-04-25T18:59:38.310457+0000","last_fullsized":"2024-04-25T18:59:38.310457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573737+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T18:59:38.573737+0000","last_peered":"2024-04-25T18:59:38.573737+0000","last_clean":"2024-04-25T18:59:38.573737+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T18:59:38.573737+0000","last_undegraded":"2024-04-25T18:59:38.573737+0000","last_fullsized":"2024-04-25T18:59:38.573737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445702+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T18:59:38.445702+0000","last_peered":"2024-04-25T18:59:38.445702+0000","last_clean":"2024-04-25T18:59:38.445702+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T18:59:38.445702+0000","last_undegraded":"2024-04-25T18:59:38.445702+0000","last_fullsized":"2024-04-25T18:59:38.445702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313883+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T18:59:38.313883+0000","last_peered":"2024-04-25T18:59:38.313883+0000","last_clean":"2024-04-25T18:59:38.313883+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T18:59:38.313883+0000","last_undegraded":"2024-04-25T18:59:38.313883+0000","last_fullsized":"2024-04-25T18:59:38.313883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:43.589823+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T18:59:43.589823+0000","last_peered":"2024-04-25T18:59:43.589823+0000","last_clean":"2024-04-25T18:59:43.589823+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T18:59:43.589823+0000","last_undegraded":"2024-04-25T18:59:43.589823+0000","last_fullsized":"2024-04-25T18:59:43.589823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312818+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T18:59:38.312818+0000","last_peered":"2024-04-25T18:59:38.312818+0000","last_clean":"2024-04-25T18:59:38.312818+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T18:59:38.312818+0000","last_undegraded":"2024-04-25T18:59:38.312818+0000","last_fullsized":"2024-04-25T18:59:38.312818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315504+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T18:59:38.315504+0000","last_peered":"2024-04-25T18:59:38.315504+0000","last_clean":"2024-04-25T18:59:38.315504+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T18:59:38.315504+0000","last_undegraded":"2024-04-25T18:59:38.315504+0000","last_fullsized":"2024-04-25T18:59:38.315504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445733+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T18:59:38.445733+0000","last_peered":"2024-04-25T18:59:38.445733+0000","last_clean":"2024-04-25T18:59:38.445733+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T18:59:38.445733+0000","last_undegraded":"2024-04-25T18:59:38.445733+0000","last_fullsized":"2024-04-25T18:59:38.445733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316086+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T18:59:38.316086+0000","last_peered":"2024-04-25T18:59:38.316086+0000","last_clean":"2024-04-25T18:59:38.316086+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T18:59:38.316086+0000","last_undegraded":"2024-04-25T18:59:38.316086+0000","last_fullsized":"2024-04-25T18:59:38.316086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.578855+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T18:59:38.578855+0000","last_peered":"2024-04-25T18:59:38.578855+0000","last_clean":"2024-04-25T18:59:38.578855+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T18:59:38.578855+0000","last_undegraded":"2024-04-25T18:59:38.578855+0000","last_fullsized":"2024-04-25T18:59:38.578855+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312363+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T18:59:38.312363+0000","last_peered":"2024-04-25T18:59:38.312363+0000","last_clean":"2024-04-25T18:59:38.312363+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T18:59:38.312363+0000","last_undegraded":"2024-04-25T18:59:38.312363+0000","last_fullsized":"2024-04-25T18:59:38.312363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.311954+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T18:59:38.311954+0000","last_peered":"2024-04-25T18:59:38.311954+0000","last_clean":"2024-04-25T18:59:38.311954+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T18:59:38.311954+0000","last_undegraded":"2024-04-25T18:59:38.311954+0000","last_fullsized":"2024-04-25T18:59:38.311954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:44.583753+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T18:59:44.583753+0000","last_peered":"2024-04-25T18:59:44.583753+0000","last_clean":"2024-04-25T18:59:44.583753+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T18:59:44.583753+0000","last_undegraded":"2024-04-25T18:59:44.583753+0000","last_fullsized":"2024-04-25T18:59:44.583753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.311758+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T18:59:38.311758+0000","last_peered":"2024-04-25T18:59:38.311758+0000","last_clean":"2024-04-25T18:59:38.311758+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T18:59:38.311758+0000","last_undegraded":"2024-04-25T18:59:38.311758+0000","last_fullsized":"2024-04-25T18:59:38.311758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.578321+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T18:59:38.578321+0000","last_peered":"2024-04-25T18:59:38.578321+0000","last_clean":"2024-04-25T18:59:38.578321+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T18:59:38.578321+0000","last_undegraded":"2024-04-25T18:59:38.578321+0000","last_fullsized":"2024-04-25T18:59:38.578321+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:43.588845+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T18:59:43.588845+0000","last_peered":"2024-04-25T18:59:43.588845+0000","last_clean":"2024-04-25T18:59:43.588845+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T18:59:43.588845+0000","last_undegraded":"2024-04-25T18:59:43.588845+0000","last_fullsized":"2024-04-25T18:59:43.588845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.715608+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T18:59:38.715608+0000","last_peered":"2024-04-25T18:59:38.715608+0000","last_clean":"2024-04-25T18:59:38.715608+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T18:59:38.715608+0000","last_undegraded":"2024-04-25T18:59:38.715608+0000","last_fullsized":"2024-04-25T18:59:38.715608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577583+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T18:59:38.577583+0000","last_peered":"2024-04-25T18:59:38.577583+0000","last_clean":"2024-04-25T18:59:38.577583+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T18:59:38.577583+0000","last_undegraded":"2024-04-25T18:59:38.577583+0000","last_fullsized":"2024-04-25T18:59:38.577583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313494+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T18:59:38.313494+0000","last_peered":"2024-04-25T18:59:38.313494+0000","last_clean":"2024-04-25T18:59:38.313494+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T18:59:38.313494+0000","last_undegraded":"2024-04-25T18:59:38.313494+0000","last_fullsized":"2024-04-25T18:59:38.313494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573152+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T18:59:38.573152+0000","last_peered":"2024-04-25T18:59:38.573152+0000","last_clean":"2024-04-25T18:59:38.573152+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T18:59:38.573152+0000","last_undegraded":"2024-04-25T18:59:38.573152+0000","last_fullsized":"2024-04-25T18:59:38.573152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313159+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T18:59:38.313159+0000","last_peered":"2024-04-25T18:59:38.313159+0000","last_clean":"2024-04-25T18:59:38.313159+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T18:59:38.313159+0000","last_undegraded":"2024-04-25T18:59:38.313159+0000","last_fullsized":"2024-04-25T18:59:38.313159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.329411+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T18:59:38.329411+0000","last_peered":"2024-04-25T18:59:38.329411+0000","last_clean":"2024-04-25T18:59:38.329411+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T18:59:38.329411+0000","last_undegraded":"2024-04-25T18:59:38.329411+0000","last_fullsized":"2024-04-25T18:59:38.329411+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573699+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T18:59:38.573699+0000","last_peered":"2024-04-25T18:59:38.573699+0000","last_clean":"2024-04-25T18:59:38.573699+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T18:59:38.573699+0000","last_undegraded":"2024-04-25T18:59:38.573699+0000","last_fullsized":"2024-04-25T18:59:38.573699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313143+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T18:59:38.313143+0000","last_peered":"2024-04-25T18:59:38.313143+0000","last_clean":"2024-04-25T18:59:38.313143+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T18:59:38.313143+0000","last_undegraded":"2024-04-25T18:59:38.313143+0000","last_fullsized":"2024-04-25T18:59:38.313143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.717054+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T18:59:38.717054+0000","last_peered":"2024-04-25T18:59:38.717054+0000","last_clean":"2024-04-25T18:59:38.717054+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T18:59:38.717054+0000","last_undegraded":"2024-04-25T18:59:38.717054+0000","last_fullsized":"2024-04-25T18:59:38.717054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.577811+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T18:59:38.577811+0000","last_peered":"2024-04-25T18:59:38.577811+0000","last_clean":"2024-04-25T18:59:38.577811+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T18:59:38.577811+0000","last_undegraded":"2024-04-25T18:59:38.577811+0000","last_fullsized":"2024-04-25T18:59:38.577811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.318862+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T18:59:38.318862+0000","last_peered":"2024-04-25T18:59:38.318862+0000","last_clean":"2024-04-25T18:59:38.318862+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T18:59:38.318862+0000","last_undegraded":"2024-04-25T18:59:38.318862+0000","last_fullsized":"2024-04-25T18:59:38.318862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315691+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T18:59:38.315691+0000","last_peered":"2024-04-25T18:59:38.315691+0000","last_clean":"2024-04-25T18:59:38.315691+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T18:59:38.315691+0000","last_undegraded":"2024-04-25T18:59:38.315691+0000","last_fullsized":"2024-04-25T18:59:38.315691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315680+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T18:59:38.315680+0000","last_peered":"2024-04-25T18:59:38.315680+0000","last_clean":"2024-04-25T18:59:38.315680+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T18:59:38.315680+0000","last_undegraded":"2024-04-25T18:59:38.315680+0000","last_fullsized":"2024-04-25T18:59:38.315680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714907+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T18:59:38.714907+0000","last_peered":"2024-04-25T18:59:38.714907+0000","last_clean":"2024-04-25T18:59:38.714907+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T18:59:38.714907+0000","last_undegraded":"2024-04-25T18:59:38.714907+0000","last_fullsized":"2024-04-25T18:59:38.714907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312210+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T18:59:38.312210+0000","last_peered":"2024-04-25T18:59:38.312210+0000","last_clean":"2024-04-25T18:59:38.312210+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T18:59:38.312210+0000","last_undegraded":"2024-04-25T18:59:38.312210+0000","last_fullsized":"2024-04-25T18:59:38.312210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310572+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T18:59:38.310572+0000","last_peered":"2024-04-25T18:59:38.310572+0000","last_clean":"2024-04-25T18:59:38.310572+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T18:59:38.310572+0000","last_undegraded":"2024-04-25T18:59:38.310572+0000","last_fullsized":"2024-04-25T18:59:38.310572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.715878+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T18:59:38.715878+0000","last_peered":"2024-04-25T18:59:38.715878+0000","last_clean":"2024-04-25T18:59:38.715878+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T18:59:38.715878+0000","last_undegraded":"2024-04-25T18:59:38.715878+0000","last_fullsized":"2024-04-25T18:59:38.715878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.316099+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T18:59:38.316099+0000","last_peered":"2024-04-25T18:59:38.316099+0000","last_clean":"2024-04-25T18:59:38.316099+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T18:59:38.316099+0000","last_undegraded":"2024-04-25T18:59:38.316099+0000","last_fullsized":"2024-04-25T18:59:38.316099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313512+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T18:59:38.313512+0000","last_peered":"2024-04-25T18:59:38.313512+0000","last_clean":"2024-04-25T18:59:38.313512+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T18:59:38.313512+0000","last_undegraded":"2024-04-25T18:59:38.313512+0000","last_fullsized":"2024-04-25T18:59:38.313512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":70,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:46.557681+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T18:59:46.557681+0000","last_peered":"2024-04-25T18:59:46.557681+0000","last_clean":"2024-04-25T18:59:46.557681+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T18:59:46.557681+0000","last_undegraded":"2024-04-25T18:59:46.557681+0000","last_fullsized":"2024-04-25T18:59:46.557681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.314401+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T18:59:38.314401+0000","last_peered":"2024-04-25T18:59:38.314401+0000","last_clean":"2024-04-25T18:59:38.314401+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T18:59:38.314401+0000","last_undegraded":"2024-04-25T18:59:38.314401+0000","last_fullsized":"2024-04-25T18:59:38.314401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445972+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T18:59:38.445972+0000","last_peered":"2024-04-25T18:59:38.445972+0000","last_clean":"2024-04-25T18:59:38.445972+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T18:59:38.445972+0000","last_undegraded":"2024-04-25T18:59:38.445972+0000","last_fullsized":"2024-04-25T18:59:38.445972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313856+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T18:59:38.313856+0000","last_peered":"2024-04-25T18:59:38.313856+0000","last_clean":"2024-04-25T18:59:38.313856+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T18:59:38.313856+0000","last_undegraded":"2024-04-25T18:59:38.313856+0000","last_fullsized":"2024-04-25T18:59:38.313856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573746+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T18:59:38.573746+0000","last_peered":"2024-04-25T18:59:38.573746+0000","last_clean":"2024-04-25T18:59:38.573746+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T18:59:38.573746+0000","last_undegraded":"2024-04-25T18:59:38.573746+0000","last_fullsized":"2024-04-25T18:59:38.573746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.314001+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T18:59:38.314001+0000","last_peered":"2024-04-25T18:59:38.314001+0000","last_clean":"2024-04-25T18:59:38.314001+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T18:59:38.314001+0000","last_undegraded":"2024-04-25T18:59:38.314001+0000","last_fullsized":"2024-04-25T18:59:38.314001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312268+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T18:59:38.312268+0000","last_peered":"2024-04-25T18:59:38.312268+0000","last_clean":"2024-04-25T18:59:38.312268+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T18:59:38.312268+0000","last_undegraded":"2024-04-25T18:59:38.312268+0000","last_fullsized":"2024-04-25T18:59:38.312268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310371+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T18:59:38.310371+0000","last_peered":"2024-04-25T18:59:38.310371+0000","last_clean":"2024-04-25T18:59:38.310371+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T18:59:38.310371+0000","last_undegraded":"2024-04-25T18:59:38.310371+0000","last_fullsized":"2024-04-25T18:59:38.310371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310299+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T18:59:38.310299+0000","last_peered":"2024-04-25T18:59:38.310299+0000","last_clean":"2024-04-25T18:59:38.310299+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T18:59:38.310299+0000","last_undegraded":"2024-04-25T18:59:38.310299+0000","last_fullsized":"2024-04-25T18:59:38.310299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.575702+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T18:59:38.575702+0000","last_peered":"2024-04-25T18:59:38.575702+0000","last_clean":"2024-04-25T18:59:38.575702+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T18:59:38.575702+0000","last_undegraded":"2024-04-25T18:59:38.575702+0000","last_fullsized":"2024-04-25T18:59:38.575702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573803+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T18:59:38.573803+0000","last_peered":"2024-04-25T18:59:38.573803+0000","last_clean":"2024-04-25T18:59:38.573803+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T18:59:38.573803+0000","last_undegraded":"2024-04-25T18:59:38.573803+0000","last_fullsized":"2024-04-25T18:59:38.573803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.313511+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T18:59:38.313511+0000","last_peered":"2024-04-25T18:59:38.313511+0000","last_clean":"2024-04-25T18:59:38.313511+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T18:59:38.313511+0000","last_undegraded":"2024-04-25T18:59:38.313511+0000","last_fullsized":"2024-04-25T18:59:38.313511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.310766+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T18:59:38.310766+0000","last_peered":"2024-04-25T18:59:38.310766+0000","last_clean":"2024-04-25T18:59:38.310766+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T18:59:38.310766+0000","last_undegraded":"2024-04-25T18:59:38.310766+0000","last_fullsized":"2024-04-25T18:59:38.310766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.716756+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T18:59:38.716756+0000","last_peered":"2024-04-25T18:59:38.716756+0000","last_clean":"2024-04-25T18:59:38.716756+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T18:59:38.716756+0000","last_undegraded":"2024-04-25T18:59:38.716756+0000","last_fullsized":"2024-04-25T18:59:38.716756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.445661+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T18:59:38.445661+0000","last_peered":"2024-04-25T18:59:38.445661+0000","last_clean":"2024-04-25T18:59:38.445661+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T18:59:38.445661+0000","last_undegraded":"2024-04-25T18:59:38.445661+0000","last_fullsized":"2024-04-25T18:59:38.445661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.714678+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T18:59:38.714678+0000","last_peered":"2024-04-25T18:59:38.714678+0000","last_clean":"2024-04-25T18:59:38.714678+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T18:59:38.714678+0000","last_undegraded":"2024-04-25T18:59:38.714678+0000","last_fullsized":"2024-04-25T18:59:38.714678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.315904+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T18:59:38.315904+0000","last_peered":"2024-04-25T18:59:38.315904+0000","last_clean":"2024-04-25T18:59:38.315904+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T18:59:38.315904+0000","last_undegraded":"2024-04-25T18:59:38.315904+0000","last_fullsized":"2024-04-25T18:59:38.315904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.573957+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T18:59:38.573957+0000","last_peered":"2024-04-25T18:59:38.573957+0000","last_clean":"2024-04-25T18:59:38.573957+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T18:59:38.573957+0000","last_undegraded":"2024-04-25T18:59:38.573957+0000","last_fullsized":"2024-04-25T18:59:38.573957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T18:59:38.312474+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T18:59:38.312474+0000","last_peered":"2024-04-25T18:59:38.312474+0000","last_clean":"2024-04-25T18:59:38.312474+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T18:59:38.312474+0000","last_undegraded":"2024-04-25T18:59:38.312474+0000","last_fullsized":"2024-04-25T18:59:38.312474+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279982,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67942,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":386,"num_read_kb":265,"num_write":247,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":483773,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1135428,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":483591,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483857,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1077269,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":483699,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":544959,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":484647,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":545179,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":483133,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483560,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":1076165,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:59:50.655 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:59:50.656 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T18:59:50.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T18:59:51.138 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:51.407 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:51 smithi194 ceph-mon[32740]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:51.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:51 smithi086 ceph-mon[27783]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:51.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:51 smithi183 ceph-mon[32344]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:52.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:52 smithi086 ceph-mon[27783]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:59:52.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:52 smithi183 ceph-mon[32344]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:59:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:52 smithi194 ceph-mon[32740]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:59:53.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 -- 172.21.15.86:0/1659840657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 msgr2=0x7fd014105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:53.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 --2- 172.21.15.86:0/1659840657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 0x7fd014105100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0080099c0 tx=0x7fd008009cd0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 -- 172.21.15.86:0/1659840657 shutdown_connections 2024-04-25T18:59:53.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 --2- 172.21.15.86:0/1659840657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd014106880 0x7fd01410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 --2- 172.21.15.86:0/1659840657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 0x7fd014105100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 --2- 172.21.15.86:0/1659840657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd014106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.296+0000 7fd01b800700 1 -- 172.21.15.86:0/1659840657 >> 172.21.15.86:0/1659840657 conn(0x7fd0141002a0 msgr2=0x7fd0141026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:53.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.297+0000 7fd01b800700 1 -- 172.21.15.86:0/1659840657 shutdown_connections 2024-04-25T18:59:53.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.297+0000 7fd01b800700 1 -- 172.21.15.86:0/1659840657 wait complete. 2024-04-25T18:59:53.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.298+0000 7fd01b800700 1 Processor -- start 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.298+0000 7fd01b800700 1 -- start start 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd01b800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 0x7fd01419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd01b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd01419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd01b800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd014106880 0x7fd0141a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd01b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd014110770 con 0x7fd014105ee0 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd01b800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141105f0 con 0x7fd014104ce0 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd01b800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141108f0 con 0x7fd014106880 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd013e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd01419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:53.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd013e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd01419b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40580/0 (socket says 172.21.15.86:40580) 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd013e00700 1 -- 172.21.15.86:0/2342519846 learned_addr learned my addr 172.21.15.86:0/2342519846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd019800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd014106880 0x7fd0141a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.299+0000 7fd018e00700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 0x7fd01419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd013e00700 1 -- 172.21.15.86:0/2342519846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd014106880 msgr2=0x7fd0141a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd013e00700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd014106880 0x7fd0141a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd013e00700 1 -- 172.21.15.86:0/2342519846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 msgr2=0x7fd01419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd013e00700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 0x7fd01419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd013e00700 1 -- 172.21.15.86:0/2342519846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0141a2960 con 0x7fd014105ee0 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd013e00700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd01419b250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd00000eb30 tx=0x7fd00000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:53.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cc90 con 0x7fd014105ee0 2024-04-25T18:59:53.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0000044d0 con 0x7fd014105ee0 2024-04-25T18:59:53.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0141a2c50 con 0x7fd014105ee0 2024-04-25T18:59:53.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000010430 con 0x7fd014105ee0 2024-04-25T18:59:53.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.300+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0141a3190 con 0x7fd014105ee0 2024-04-25T18:59:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.302+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd014050f80 con 0x7fd014105ee0 2024-04-25T18:59:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.304+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd000003ea0 con 0x7fd014105ee0 2024-04-25T18:59:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.305+0000 7fd011600700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcffc0a3d80 0x7fcffc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.305+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fd0000d0870 con 0x7fd014105ee0 2024-04-25T18:59:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.305+0000 7fd018e00700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcffc0a3d80 0x7fcffc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.306+0000 7fd018e00700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcffc0a3d80 0x7fcffc0a6240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd008009990 tx=0x7fd008005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:53.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.310+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd00009d080 con 0x7fd014105ee0 2024-04-25T18:59:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.630+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd01402e0a0 con 0x7fd014105ee0 2024-04-25T18:59:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.635+0000 7fd011600700 1 -- 172.21.15.86:0/2342519846 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd01402e0a0 con 0x7fd014105ee0 2024-04-25T18:59:53.637 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T18:59:53.637 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQApqCpmEs20JRAAvyoYW8Jqy7SqqTv/DuvwDw== 2024-04-25T18:59:53.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcffc0a3d80 msgr2=0x7fcffc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcffc0a3d80 0x7fcffc0a6240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd008009990 tx=0x7fd008005fb0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 msgr2=0x7fd01419b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd01419b250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd00000eb30 tx=0x7fd00000ee40 comp rx=0 tx=0).stop 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 shutdown_connections 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcffc0a3d80 0x7fcffc0a6240 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd014106880 0x7fd0141a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd014104ce0 0x7fd01419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 --2- 172.21.15.86:0/2342519846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd014105ee0 0x7fd01419b250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:53.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 >> 172.21.15.86:0/2342519846 conn(0x7fd0141002a0 msgr2=0x7fd0141091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:53.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 shutdown_connections 2024-04-25T18:59:53.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:53.638+0000 7fd01b800700 1 -- 172.21.15.86:0/2342519846 wait complete. 2024-04-25T18:59:54.526 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T18:59:54.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T18:59:54.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T18:59:54.645 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T18:59:54.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T18:59:54.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T18:59:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:54 smithi183 ceph-mon[32344]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:59:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:54 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2342519846' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:59:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:54 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2342519846' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:59:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:54 smithi194 ceph-mon[32740]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:59:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:54 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2342519846' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:59:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:54 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2342519846' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:59:54.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:54 smithi086 ceph-mon[27783]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:59:54.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:54 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2342519846' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:59:54.761 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:54 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2342519846' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:59:55.125 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:56 smithi183 ceph-mon[32344]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T18:59:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:56 smithi194 ceph-mon[32740]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T18:59:56.819 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:56 smithi086 ceph-mon[27783]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T18:59:57.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 -- 172.21.15.86:0/3562893605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 msgr2=0x7f8048075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:57.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 --2- 172.21.15.86:0/3562893605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f8048075050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8038009990 tx=0x7f8038009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 -- 172.21.15.86:0/3562893605 shutdown_connections 2024-04-25T18:59:57.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 --2- 172.21.15.86:0/3562893605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8048113c10 0x7f8048116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 --2- 172.21.15.86:0/3562893605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f8048075050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 --2- 172.21.15.86:0/3562893605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048075590 0x7f80481136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 -- 172.21.15.86:0/3562893605 >> 172.21.15.86:0/3562893605 conn(0x7f8048079ba0 msgr2=0x7f8048079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:57.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.427+0000 7f804d400700 1 -- 172.21.15.86:0/3562893605 shutdown_connections 2024-04-25T18:59:57.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.428+0000 7f804d400700 1 -- 172.21.15.86:0/3562893605 wait complete. 2024-04-25T18:59:57.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 Processor -- start 2024-04-25T18:59:57.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 -- start start 2024-04-25T18:59:57.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f80481a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:57.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8048075590 0x7f80481a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:57.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048113c10 0x7f804819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8048119260 con 0x7f8048074c70 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80481190e0 con 0x7f8048113c10 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f804d400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80481193e0 con 0x7f8048075590 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f8046800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f80481a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f8046800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f80481a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40596/0 (socket says 172.21.15.86:40596) 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f8046800700 1 -- 172.21.15.86:0/1662525880 learned_addr learned my addr 172.21.15.86:0/1662525880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.429+0000 7f803fe00700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8048075590 0x7f80481a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8047200700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048113c10 0x7f804819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8046800700 1 -- 172.21.15.86:0/1662525880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8048075590 msgr2=0x7f80481a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8046800700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8048075590 0x7f80481a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8046800700 1 -- 172.21.15.86:0/1662525880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048113c10 msgr2=0x7f804819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8046800700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048113c10 0x7f804819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8046800700 1 -- 172.21.15.86:0/1662525880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819ff20 con 0x7f8048074c70 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f8046800700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f80481a5570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8038000c00 tx=0x7f80380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.430+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038027070 con 0x7f8048074c70 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.431+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8038022470 con 0x7f8048074c70 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.431+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80481a01b0 con 0x7f8048074c70 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.431+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038017650 con 0x7f8048074c70 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.431+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481ac320 con 0x7f8048074c70 2024-04-25T18:59:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.432+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8048076360 con 0x7f8048074c70 2024-04-25T18:59:57.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.434+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f803802a070 con 0x7f8048074c70 2024-04-25T18:59:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.435+0000 7f803f400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f802c0a3cd0 0x7f802c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.435+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f80380d1240 con 0x7f8048074c70 2024-04-25T18:59:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.436+0000 7f803fe00700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f802c0a3cd0 0x7f802c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.436+0000 7f803fe00700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f802c0a3cd0 0x7f802c0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8030009da0 tx=0x7f803000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.440+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8038013070 con 0x7f8048074c70 2024-04-25T18:59:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.728+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f80481ac550 con 0x7f8048074c70 2024-04-25T18:59:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.743+0000 7f803f400700 1 -- 172.21.15.86:0/1662525880 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7f803809dae0 con 0x7f8048074c70 2024-04-25T18:59:57.749 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T18:59:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f802c0a3cd0 msgr2=0x7f802c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f802c0a3cd0 0x7f802c0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8030009da0 tx=0x7f803000b040 comp rx=0 tx=0).stop 2024-04-25T18:59:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 msgr2=0x7f80481a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f80481a5570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8038000c00 tx=0x7f80380057c0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 shutdown_connections 2024-04-25T18:59:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f802c0a3cd0 0x7f802c0a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048113c10 0x7f804819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8048074c70 0x7f80481a5570 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 --2- 172.21.15.86:0/1662525880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8048075590 0x7f80481a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.749+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 >> 172.21.15.86:0/1662525880 conn(0x7f8048079ba0 msgr2=0x7f8048112a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.750+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 shutdown_connections 2024-04-25T18:59:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T18:59:57.750+0000 7f804d400700 1 -- 172.21.15.86:0/1662525880 wait complete. 2024-04-25T18:59:58.661 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":237697797},{"type":"v1","addr":"172.21.15.86:6801","nonce":237697797}]},"active_addr":"172.21.15.86:6801/237697797","active_change":"2024-04-25T18:50:37.175636+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2093097457}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":324236586}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1310908896}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1288701391}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3898956201}]}]} 2024-04-25T18:59:58.666 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T18:59:58.667 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T18:59:58.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:59:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:58 smithi183 ceph-mon[32344]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T18:59:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 18:59:58 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1662525880' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:59:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:58 smithi194 ceph-mon[32740]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T18:59:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 18:59:58 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1662525880' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:59:58.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:58 smithi086 ceph-mon[27783]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T18:59:58.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 18:59:58 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1662525880' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:59:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:00 smithi194 ceph-mon[32740]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:00 smithi194 ceph-mon[32740]: overall HEALTH_OK 2024-04-25T19:00:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:00 smithi086 ceph-mon[27783]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:00.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:00 smithi086 ceph-mon[27783]: overall HEALTH_OK 2024-04-25T19:00:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:00 smithi183 ceph-mon[32344]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:00 smithi183 ceph-mon[32344]: overall HEALTH_OK 2024-04-25T19:00:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.392+0000 7f5935400700 1 -- 172.21.15.86:0/1167251352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930113920 msgr2=0x7f5930115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.392+0000 7f5935400700 1 --2- 172.21.15.86:0/1167251352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930113920 0x7f5930115d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f592800b0a0 tx=0x7f592800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 -- 172.21.15.86:0/1167251352 shutdown_connections 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 --2- 172.21.15.86:0/1167251352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930113920 0x7f5930115d10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 --2- 172.21.15.86:0/1167251352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5930076f70 0x7f5930077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 --2- 172.21.15.86:0/1167251352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930077890 0x7f59301133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 -- 172.21.15.86:0/1167251352 >> 172.21.15.86:0/1167251352 conn(0x7f5930100770 msgr2=0x7f5930102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 -- 172.21.15.86:0/1167251352 shutdown_connections 2024-04-25T19:00:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.393+0000 7f5935400700 1 -- 172.21.15.86:0/1167251352 wait complete. 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 Processor -- start 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 -- start start 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 0x7f593019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5930077890 0x7f593019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930113920 0x7f59301a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5930119280 con 0x7f5930076f70 2024-04-25T19:00:01.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5930119100 con 0x7f5930077890 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.394+0000 7f5935400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5930119400 con 0x7f5930113920 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 0x7f593019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 0x7f593019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45744/0 (socket says 172.21.15.86:45744) 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 -- 172.21.15.86:0/809421397 learned_addr learned my addr 172.21.15.86:0/809421397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f5925e00700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5930077890 0x7f593019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592f200700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930113920 0x7f59301a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 -- 172.21.15.86:0/809421397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930113920 msgr2=0x7f59301a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930113920 0x7f59301a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 -- 172.21.15.86:0/809421397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5930077890 msgr2=0x7f593019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5930077890 0x7f593019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 -- 172.21.15.86:0/809421397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59301a3720 con 0x7f5930076f70 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592f200700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930113920 0x7f59301a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:01.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.395+0000 7f592e800700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 0x7f593019f7d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f592000e9a0 tx=0x7f592000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.396+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592000cbd0 con 0x7f5930076f70 2024-04-25T19:00:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.396+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592000cd70 con 0x7f5930076f70 2024-04-25T19:00:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.396+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920018680 con 0x7f5930076f70 2024-04-25T19:00:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.396+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59301a3a10 con 0x7f5930076f70 2024-04-25T19:00:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.396+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59301ac380 con 0x7f5930076f70 2024-04-25T19:00:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.397+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5930069730 con 0x7f5930076f70 2024-04-25T19:00:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.399+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5920010430 con 0x7f5930076f70 2024-04-25T19:00:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.400+0000 7f5927e00700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f59140a3d50 0x7f59140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.401+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f59200d07a0 con 0x7f5930076f70 2024-04-25T19:00:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.401+0000 7f5925e00700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f59140a3d50 0x7f59140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.401+0000 7f5925e00700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f59140a3d50 0x7f59140a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5930079d70 tx=0x7f5918009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.405+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f592009cfb0 con 0x7f5930076f70 2024-04-25T19:00:01.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.633+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f593002df80 con 0x7f5930076f70 2024-04-25T19:00:01.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.637+0000 7f5927e00700 1 -- 172.21.15.86:0/809421397 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22726 (secure 0 0 0) 0x7f592009c8b0 con 0x7f5930076f70 2024-04-25T19:00:01.639 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:00:01.639 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T18:59:38.305268+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f59140a3d50 msgr2=0x7f59140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f59140a3d50 0x7f59140a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5930079d70 tx=0x7f5918009260 comp rx=0 tx=0).stop 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 msgr2=0x7f593019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 0x7f593019f7d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f592000e9a0 tx=0x7f592000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 shutdown_connections 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f59140a3d50 0x7f59140a6210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5930113920 0x7f59301a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5930076f70 0x7f593019f7d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 --2- 172.21.15.86:0/809421397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5930077890 0x7f593019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.640+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 >> 172.21.15.86:0/809421397 conn(0x7f5930100770 msgr2=0x7f5930102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.641+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 shutdown_connections 2024-04-25T19:00:01.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:01.641+0000 7f5935400700 1 -- 172.21.15.86:0/809421397 wait complete. 2024-04-25T19:00:02.582 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T19:00:02.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T19:00:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:02 smithi194 ceph-mon[32740]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:02 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/809421397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:00:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:02 smithi086 ceph-mon[27783]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:02 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/809421397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:00:02.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:02 smithi183 ceph-mon[32344]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:02 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/809421397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:00:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:04 smithi194 ceph-mon[32740]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:04.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:04 smithi086 ceph-mon[27783]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:04.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:04 smithi183 ceph-mon[32344]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:05.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 -- 172.21.15.86:0/4149014006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 msgr2=0x7f780c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:05.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 --2- 172.21.15.86:0/4149014006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c1098c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009920 tx=0x7f77f8009c30 comp rx=0 tx=0).stop 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 -- 172.21.15.86:0/4149014006 shutdown_connections 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 --2- 172.21.15.86:0/4149014006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 0x7f780c111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 --2- 172.21.15.86:0/4149014006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 0x7f780c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 --2- 172.21.15.86:0/4149014006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c1098c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 -- 172.21.15.86:0/4149014006 >> 172.21.15.86:0/4149014006 conn(0x7f780c100750 msgr2=0x7f780c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 -- 172.21.15.86:0/4149014006 shutdown_connections 2024-04-25T19:00:05.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.235+0000 7f7811400700 1 -- 172.21.15.86:0/4149014006 wait complete. 2024-04-25T19:00:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 Processor -- start 2024-04-25T19:00:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 -- start start 2024-04-25T19:00:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 0x7f780c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 0x7f780c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f780c114e30 con 0x7f780c105870 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f780c114cb0 con 0x7f780c109fa0 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.236+0000 7f7811400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f780c114fb0 con 0x7f780c104ec0 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45762/0 (socket says 172.21.15.86:45762) 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 -- 172.21.15.86:0/1515372633 learned_addr learned my addr 172.21.15.86:0/1515372633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f780a800700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 0x7f780c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f780b200700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 0x7f780c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 -- 172.21.15.86:0/1515372633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 msgr2=0x7f780c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 0x7f780c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 -- 172.21.15.86:0/1515372633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 msgr2=0x7f780c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 0x7f780c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f7809e00700 1 -- 172.21.15.86:0/1515372633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c1a03f0 con 0x7f780c105870 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f780a800700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 0x7f780c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.237+0000 7f780b200700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 0x7f780c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.238+0000 7f7809e00700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c19b970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009230 tx=0x7f77f800f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.238+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8027070 con 0x7f780c105870 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.238+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f80054f0 con 0x7f780c105870 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.238+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c1a0680 con 0x7f780c105870 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.238+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8004930 con 0x7f780c105870 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.238+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c1a7f60 con 0x7f780c105870 2024-04-25T19:00:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.239+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c1063a0 con 0x7f780c105870 2024-04-25T19:00:05.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.241+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f77f802a070 con 0x7f780c105870 2024-04-25T19:00:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.244+0000 7f7803400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77f00a3cd0 0x7f77f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.244+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f77f80d0f00 con 0x7f780c105870 2024-04-25T19:00:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.244+0000 7f780a800700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77f00a3cd0 0x7f77f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.245+0000 7f780a800700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77f00a3cd0 0x7f77f00a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77fc011490 tx=0x7f77fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:05.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.249+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f805eff0 con 0x7f780c105870 2024-04-25T19:00:05.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.474+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f780c02df80 con 0x7f780c105870 2024-04-25T19:00:05.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.476+0000 7f7803400700 1 -- 172.21.15.86:0/1515372633 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22726 (secure 0 0 0) 0x7f77f8013030 con 0x7f780c105870 2024-04-25T19:00:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:00:05.478 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T18:59:38.305268+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:00:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.480+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77f00a3cd0 msgr2=0x7f77f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.480+0000 7f7811400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77f00a3cd0 0x7f77f00a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77fc011490 tx=0x7f77fc009260 comp rx=0 tx=0).stop 2024-04-25T19:00:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.480+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 msgr2=0x7f780c19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.480+0000 7f7811400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c19b970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009230 tx=0x7f77f800f950 comp rx=0 tx=0).stop 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 shutdown_connections 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f77f00a3cd0 0x7f77f00a6190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f780c109fa0 0x7f780c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f780c104ec0 0x7f780c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 --2- 172.21.15.86:0/1515372633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f780c105870 0x7f780c19b970 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 >> 172.21.15.86:0/1515372633 conn(0x7f780c100750 msgr2=0x7f780c101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 shutdown_connections 2024-04-25T19:00:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:05.481+0000 7f7811400700 1 -- 172.21.15.86:0/1515372633 wait complete. 2024-04-25T19:00:06.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T19:00:06.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T19:00:06.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T19:00:06.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T19:00:06.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T19:00:06.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T19:00:06.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T19:00:06.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T19:00:06.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T19:00:06.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T19:00:06.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T19:00:06.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T19:00:06.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:06 smithi086 ceph-mon[27783]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:06.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:06 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1515372633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:00:06.719 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:06 smithi183 ceph-mon[32344]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:06.720 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:06 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1515372633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:00:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:06 smithi194 ceph-mon[32740]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:06 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1515372633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:00:07.431 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:07.461 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:07.561 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:07.880 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:08 smithi086 ceph-mon[27783]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:08 smithi194 ceph-mon[32740]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:08 smithi183 ceph-mon[32344]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:10.703 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:10 smithi086 ceph-mon[27783]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:10 smithi194 ceph-mon[32740]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:10 smithi183 ceph-mon[32344]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:12 smithi086 ceph-mon[27783]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:12 smithi194 ceph-mon[32740]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:12 smithi183 ceph-mon[32344]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:14.712 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:14 smithi086 ceph-mon[27783]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:14 smithi194 ceph-mon[32740]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:14 smithi183 ceph-mon[32344]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 -- 172.21.15.86:0/1375501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4103450 msgr2=0x7f3dd410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/1375501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4103450 0x7f3dd410f8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00b0a0 tx=0x7f3dcc00b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 -- 172.21.15.86:0/1375501325 shutdown_connections 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/1375501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4103450 0x7f3dd410f8b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/1375501325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dd4108a50 0x7f3dd4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/1375501325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4102a90 0x7f3dd4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 -- 172.21.15.86:0/1375501325 >> 172.21.15.86:0/1375501325 conn(0x7f3dd4079ba0 msgr2=0x7f3dd4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 -- 172.21.15.86:0/1375501325 shutdown_connections 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.466+0000 7f3dd9a00700 1 -- 172.21.15.86:0/1375501325 wait complete. 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.467+0000 7f3dd9a00700 1 Processor -- start 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 -- start start 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dd4102a90 0x7f3dd419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4103450 0x7f3dd419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4108a50 0x7f3dd419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112c70 con 0x7f3dd4108a50 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112af0 con 0x7f3dd4102a90 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd9a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112df0 con 0x7f3dd4103450 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4108a50 0x7f3dd419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4108a50 0x7f3dd419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56224/0 (socket says 172.21.15.86:56224) 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd3400700 1 -- 172.21.15.86:0/692967597 learned_addr learned my addr 172.21.15.86:0/692967597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd2a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dd4102a90 0x7f3dd419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.468+0000 7f3dd2000700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4103450 0x7f3dd419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd2a00700 1 -- 172.21.15.86:0/692967597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4103450 msgr2=0x7f3dd419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd2a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4103450 0x7f3dd419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd2a00700 1 -- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4108a50 msgr2=0x7f3dd419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd2a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4108a50 0x7f3dd419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd2a00700 1 -- 172.21.15.86:0/692967597 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd41a0310 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd3400700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd4108a50 0x7f3dd419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd2a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dd4102a90 0x7f3dd419b300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e9f0 tx=0x7f3dc400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400c9c0 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc400cb60 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4018680 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.469+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41a0600 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.470+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41a7e80 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.471+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3dd410f8f0 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.473+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3dc4010430 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.474+0000 7f3dcb400700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3db80a3d50 0x7f3db80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.474+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f3dc40d0620 con 0x7f3dd4102a90 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.474+0000 7f3dd2000700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3db80a3d50 0x7f3db80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.474+0000 7f3dcb400700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] conn(0x7f3db80ad910 0x7f3db80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.474+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 --> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3db80b0480 con 0x7f3db80ad910 2024-04-25T19:00:15.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.474+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3dc40d7050 con 0x7f3dd4102a90 2024-04-25T19:00:15.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.475+0000 7f3dd3400700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] conn(0x7f3db80ad910 0x7f3db80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.475+0000 7f3dd2000700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3db80a3d50 0x7f3db80a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3dd407a4a0 tx=0x7f3dc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.475+0000 7f3dd3400700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] conn(0x7f3db80ad910 0x7f3db80afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.476+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== osd.0 v2:172.21.15.86:6802/2684261173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3db80b0480 con 0x7f3db80ad910 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.494+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 --> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3dd4104180 con 0x7f3db80ad910 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.494+0000 7f3dcb400700 1 -- 172.21.15.86:0/692967597 <== osd.0 v2:172.21.15.86:6802/2684261173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3dd4104180 con 0x7f3db80ad910 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] conn(0x7f3db80ad910 msgr2=0x7f3db80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] conn(0x7f3db80ad910 0x7f3db80afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3db80a3d50 msgr2=0x7f3db80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3db80a3d50 0x7f3db80a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3dd407a4a0 tx=0x7f3dc0009260 comp rx=0 tx=0).stop 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dd4102a90 msgr2=0x7f3dd419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 --2- 172.21.15.86:0/692967597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dd4102a90 0x7f3dd419b300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e9f0 tx=0x7f3dc400ed00 comp rx=0 tx=0).stop 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd3400700 1 -- 172.21.15.86:0/692967597 reap_dead start 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 shutdown_connections 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 >> 172.21.15.86:0/692967597 conn(0x7f3dd4079ba0 msgr2=0x7f3dd410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.495+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 shutdown_connections 2024-04-25T19:00:15.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:14.496+0000 7f3dd9a00700 1 -- 172.21.15.86:0/692967597 wait complete. 2024-04-25T19:00:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:16 smithi194 ceph-mon[32740]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:16 smithi086 ceph-mon[27783]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:16 smithi183 ceph-mon[32344]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:18.522 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:18 smithi194 ceph-mon[32740]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:18.756 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:18 smithi086 ceph-mon[27783]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:18 smithi183 ceph-mon[32344]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:19.086 INFO:teuthology.orchestra.run.smithi086.stdout:34359738465 2024-04-25T19:00:19.087 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T19:00:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 -- 172.21.15.86:0/391227150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 msgr2=0x7f355410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 --2- 172.21.15.86:0/391227150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f355410f8b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f354c00b0a0 tx=0x7f354c00b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 -- 172.21.15.86:0/391227150 shutdown_connections 2024-04-25T19:00:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 --2- 172.21.15.86:0/391227150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f355410f8b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 --2- 172.21.15.86:0/391227150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554108a50 0x7f3554108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 --2- 172.21.15.86:0/391227150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3554102a90 0x7f3554102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 -- 172.21.15.86:0/391227150 >> 172.21.15.86:0/391227150 conn(0x7f3554079ba0 msgr2=0x7f3554079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 -- 172.21.15.86:0/391227150 shutdown_connections 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.771+0000 7f355a600700 1 -- 172.21.15.86:0/391227150 wait complete. 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.772+0000 7f355a600700 1 Processor -- start 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.772+0000 7f355a600700 1 -- start start 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f355a600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554102a90 0x7f3554199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f355a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f3554199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f355a600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3554108a50 0x7f355419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f355a600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3554112c50 con 0x7f3554103450 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f355a600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3554112ad0 con 0x7f3554102a90 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f355a600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3554112dd0 con 0x7f3554108a50 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3552a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f3554199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3553400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554102a90 0x7f3554199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3553400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554102a90 0x7f3554199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:52552/0 (socket says 172.21.15.86:52552) 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3552a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f3554199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56258/0 (socket says 172.21.15.86:56258) 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3553400700 1 -- 172.21.15.86:0/4089507669 learned_addr learned my addr 172.21.15.86:0/4089507669 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3553e00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3554108a50 0x7f355419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3553400700 1 -- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3554108a50 msgr2=0x7f355419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.773+0000 7f3553400700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3554108a50 0x7f355419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f3553400700 1 -- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 msgr2=0x7f3554199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f3553400700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f3554199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f3553400700 1 -- 172.21.15.86:0/4089507669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355419e110 con 0x7f3554102a90 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f3552a00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3554103450 0x7f3554199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f3553400700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554102a90 0x7f3554199100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9a0 tx=0x7f354400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354400ca10 con 0x7f3554102a90 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354400cbb0 con 0x7f3554102a90 2024-04-25T19:00:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355419e400 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.774+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544018680 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.775+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35541a5c80 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.775+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35540631d0 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.778+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3544010430 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.779+0000 7f354be00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35380a3cd0 0x7f35380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.779+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f35440d0fa0 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.779+0000 7f3552a00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35380a3cd0 0x7f35380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.779+0000 7f354be00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] conn(0x7f35380ad890 0x7f35380afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.779+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 --> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35380b0400 con 0x7f35380ad890 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.779+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f35440d1350 con 0x7f3554102a90 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.780+0000 7f3553e00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] conn(0x7f35380ad890 0x7f35380afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.780+0000 7f3552a00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35380a3cd0 0x7f35380a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f355407a210 tx=0x7f3540009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.781+0000 7f3553e00700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] conn(0x7f35380ad890 0x7f35380afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.782+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== osd.5 v2:172.21.15.194:6808/1967519671 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35380b0400 con 0x7f35380ad890 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.800+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 --> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3554063650 con 0x7f35380ad890 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.801+0000 7f354be00700 1 -- 172.21.15.86:0/4089507669 <== osd.5 v2:172.21.15.194:6808/1967519671 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3554063650 con 0x7f35380ad890 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] conn(0x7f35380ad890 msgr2=0x7f35380afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] conn(0x7f35380ad890 0x7f35380afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35380a3cd0 msgr2=0x7f35380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35380a3cd0 0x7f35380a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f355407a210 tx=0x7f3540009260 comp rx=0 tx=0).stop 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554102a90 msgr2=0x7f3554199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 --2- 172.21.15.86:0/4089507669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554102a90 0x7f3554199100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9a0 tx=0x7f354400ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f3553e00700 1 -- 172.21.15.86:0/4089507669 reap_dead start 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 shutdown_connections 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 >> 172.21.15.86:0/4089507669 conn(0x7f3554079ba0 msgr2=0x7f3554101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.802+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 shutdown_connections 2024-04-25T19:00:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:19.803+0000 7f355a600700 1 -- 172.21.15.86:0/4089507669 wait complete. 2024-04-25T19:00:20.497 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:20 smithi086 ceph-mon[27783]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:20 smithi194 ceph-mon[32740]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:20 smithi183 ceph-mon[32344]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 -- 172.21.15.86:0/759739817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 msgr2=0x7f92dc115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 --2- 172.21.15.86:0/759739817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc115f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b0d0 tx=0x7f92d400b3e0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 -- 172.21.15.86:0/759739817 shutdown_connections 2024-04-25T19:00:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 --2- 172.21.15.86:0/759739817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc115f80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 --2- 172.21.15.86:0/759739817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92dc1092d0 0x7f92dc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 --2- 172.21.15.86:0/759739817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92dc109c80 0x7f92dc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 -- 172.21.15.86:0/759739817 >> 172.21.15.86:0/759739817 conn(0x7f92dc076210 msgr2=0x7f92dc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:21.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 -- 172.21.15.86:0/759739817 shutdown_connections 2024-04-25T19:00:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.941+0000 7f92e1000700 1 -- 172.21.15.86:0/759739817 wait complete. 2024-04-25T19:00:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.942+0000 7f92e1000700 1 Processor -- start 2024-04-25T19:00:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.942+0000 7f92e1000700 1 -- start start 2024-04-25T19:00:21.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92e1000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92dc1092d0 0x7f92dc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92e1000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92dc109c80 0x7f92dc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92e1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92e1000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1192a0 con 0x7f92dc10e3b0 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92e1000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc119120 con 0x7f92dc1092d0 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92e1000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc119420 con 0x7f92dc109c80 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92d9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92dc1092d0 0x7f92dc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc1a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33870/0 (socket says 172.21.15.86:33870) 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 -- 172.21.15.86:0/3645288945 learned_addr learned my addr 172.21.15.86:0/3645288945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92d9400700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92dc109c80 0x7f92dc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:21.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 -- 172.21.15.86:0/3645288945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92dc109c80 msgr2=0x7f92dc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92dc109c80 0x7f92dc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 -- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92dc1092d0 msgr2=0x7f92dc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92dc1092d0 0x7f92dc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.943+0000 7f92da800700 1 -- 172.21.15.86:0/3645288945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc1a46e0 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.944+0000 7f92da800700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc1a3fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f92d40079a0 tx=0x7f92d40079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.944+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d401f070 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.944+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc1a4970 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.944+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d4003c00 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.944+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1ac250 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.944+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4005a50 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.945+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92dc0631d0 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.947+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f92d4022070 con 0x7f92dc10e3b0 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.949+0000 7f92d2a00700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f92c00a3e70 0x7f92c00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:21.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.949+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f92d40d1850 con 0x7f92dc10e3b0 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.949+0000 7f92d9e00700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f92c00a3e70 0x7f92c00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.949+0000 7f92d2a00700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] conn(0x7f92c00ada30 0x7f92c00afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.949+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 --> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92d4005a50 con 0x7f92c00ada30 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.949+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f92d40d1c00 con 0x7f92dc10e3b0 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.950+0000 7f92d9e00700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f92c00a3e70 0x7f92c00a6330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f92cc007be0 tx=0x7f92cc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.950+0000 7f92d9400700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] conn(0x7f92c00ada30 0x7f92c00afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.950+0000 7f92d9400700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] conn(0x7f92c00ada30 0x7f92c00afed0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.951+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== osd.7 v2:172.21.15.183:6816/2750054847 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92d4005a50 con 0x7f92c00ada30 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.970+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 --> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92dc0635d0 con 0x7f92c00ada30 2024-04-25T19:00:21.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.970+0000 7f92d2a00700 1 -- 172.21.15.86:0/3645288945 <== osd.7 v2:172.21.15.183:6816/2750054847 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f92dc0635d0 con 0x7f92c00ada30 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] conn(0x7f92c00ada30 msgr2=0x7f92c00afed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92e1000700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] conn(0x7f92c00ada30 0x7f92c00afed0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f92c00a3e70 msgr2=0x7f92c00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92e1000700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f92c00a3e70 0x7f92c00a6330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f92cc007be0 tx=0x7f92cc0073d0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 msgr2=0x7f92dc1a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92e1000700 1 --2- 172.21.15.86:0/3645288945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92dc10e3b0 0x7f92dc1a3fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f92d40079a0 tx=0x7f92d40079d0 comp rx=0 tx=0).stop 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.971+0000 7f92da800700 1 -- 172.21.15.86:0/3645288945 reap_dead start 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.972+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 shutdown_connections 2024-04-25T19:00:21.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.972+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 >> 172.21.15.86:0/3645288945 conn(0x7f92dc076210 msgr2=0x7f92dc076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:21.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.972+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 shutdown_connections 2024-04-25T19:00:21.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:20.972+0000 7f92e1000700 1 -- 172.21.15.86:0/3645288945 wait complete. 2024-04-25T19:00:22.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:22 smithi086 ceph-mon[27783]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:22 smithi194 ceph-mon[32740]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:22 smithi183 ceph-mon[32344]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:00:23.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 -- 172.21.15.86:0/3205130459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 msgr2=0x7f163c111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 --2- 172.21.15.86:0/3205130459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c111830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f163400b0a0 tx=0x7f163400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 -- 172.21.15.86:0/3205130459 shutdown_connections 2024-04-25T19:00:23.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 --2- 172.21.15.86:0/3205130459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c111830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 --2- 172.21.15.86:0/3205130459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f163c074cb0 0x7f163c075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 --2- 172.21.15.86:0/3205130459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163c0755d0 0x7f163c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 -- 172.21.15.86:0/3205130459 >> 172.21.15.86:0/3205130459 conn(0x7f163c079c30 msgr2=0x7f163c07a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:23.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 -- 172.21.15.86:0/3205130459 shutdown_connections 2024-04-25T19:00:23.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.761+0000 7f1644600700 1 -- 172.21.15.86:0/3205130459 wait complete. 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.762+0000 7f1644600700 1 Processor -- start 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 -- start start 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f163c074cb0 0x7f163c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163c0755d0 0x7f163c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f163c114eb0 con 0x7f163c10f440 2024-04-25T19:00:23.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f163c114d30 con 0x7f163c0755d0 2024-04-25T19:00:23.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1644600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f163c115030 con 0x7f163c074cb0 2024-04-25T19:00:23.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1642600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.763+0000 7f1641c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f163c074cb0 0x7f163c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163c0755d0 0x7f163c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1642600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33914/0 (socket says 172.21.15.86:33914) 2024-04-25T19:00:23.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1642600700 1 -- 172.21.15.86:0/1896214692 learned_addr learned my addr 172.21.15.86:0/1896214692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 -- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f163c074cb0 msgr2=0x7f163c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f163c074cb0 0x7f163c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 -- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 msgr2=0x7f163c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 -- 172.21.15.86:0/1896214692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c1a03d0 con 0x7f163c0755d0 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1642600700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c10f440 0x7f163c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:23.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.764+0000 7f1641200700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163c0755d0 0x7f163c19b900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f162800e9f0 tx=0x7f162800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:23.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.765+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162800cb70 con 0x7f163c0755d0 2024-04-25T19:00:23.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.765+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c1a06c0 con 0x7f163c0755d0 2024-04-25T19:00:23.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.765+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162800ee90 con 0x7f163c0755d0 2024-04-25T19:00:23.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.765+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628003d30 con 0x7f163c0755d0 2024-04-25T19:00:23.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.765+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c1a7f40 con 0x7f163c0755d0 2024-04-25T19:00:23.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.766+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f163c063150 con 0x7f163c0755d0 2024-04-25T19:00:23.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.768+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1628004020 con 0x7f163c0755d0 2024-04-25T19:00:23.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.769+0000 7f163a000700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16240a3d50 0x7f16240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.769+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f16280d0a20 con 0x7f163c0755d0 2024-04-25T19:00:23.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.769+0000 7f1641c00700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16240a3d50 0x7f16240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.770+0000 7f163a000700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] conn(0x7f16240ad910 0x7f16240afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.770+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 --> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16280090b0 con 0x7f16240ad910 2024-04-25T19:00:23.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.770+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f16280d0dd0 con 0x7f163c0755d0 2024-04-25T19:00:23.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.770+0000 7f1642600700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] conn(0x7f16240ad910 0x7f16240afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.770+0000 7f1641c00700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16240a3d50 0x7f16240a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1630005950 tx=0x7f163000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:23.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.771+0000 7f1642600700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] conn(0x7f16240ad910 0x7f16240afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:23.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.772+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== osd.11 v2:172.21.15.194:6824/745573568 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16280090b0 con 0x7f16240ad910 2024-04-25T19:00:23.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.790+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 --> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f163c0635d0 con 0x7f16240ad910 2024-04-25T19:00:23.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.790+0000 7f163a000700 1 -- 172.21.15.86:0/1896214692 <== osd.11 v2:172.21.15.194:6824/745573568 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f163c0635d0 con 0x7f16240ad910 2024-04-25T19:00:23.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] conn(0x7f16240ad910 msgr2=0x7f16240afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] conn(0x7f16240ad910 0x7f16240afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16240a3d50 msgr2=0x7f16240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16240a3d50 0x7f16240a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1630005950 tx=0x7f163000b410 comp rx=0 tx=0).stop 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163c0755d0 msgr2=0x7f163c19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 --2- 172.21.15.86:0/1896214692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f163c0755d0 0x7f163c19b900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f162800e9f0 tx=0x7f162800ed00 comp rx=0 tx=0).stop 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1642600700 1 -- 172.21.15.86:0/1896214692 reap_dead start 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 shutdown_connections 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 >> 172.21.15.86:0/1896214692 conn(0x7f163c079c30 msgr2=0x7f163c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 shutdown_connections 2024-04-25T19:00:23.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.791+0000 7f1644600700 1 -- 172.21.15.86:0/1896214692 wait complete. 2024-04-25T19:00:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 -- 172.21.15.86:0/4145425116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109310 msgr2=0x7f81041096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 --2- 172.21.15.86:0/4145425116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109310 0x7f81041096f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80f40099c0 tx=0x7f80f4009cd0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 -- 172.21.15.86:0/4145425116 shutdown_connections 2024-04-25T19:00:24.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 --2- 172.21.15.86:0/4145425116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f810410e3a0 0x7f8104115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 --2- 172.21.15.86:0/4145425116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109310 0x7f81041096f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 --2- 172.21.15.86:0/4145425116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8104109cc0 0x7f810410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 -- 172.21.15.86:0/4145425116 >> 172.21.15.86:0/4145425116 conn(0x7f8104076210 msgr2=0x7f8104078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 -- 172.21.15.86:0/4145425116 shutdown_connections 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.306+0000 7f810a800700 1 -- 172.21.15.86:0/4145425116 wait complete. 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.307+0000 7f810a800700 1 Processor -- start 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.307+0000 7f810a800700 1 -- start start 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f810a800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8104109310 0x7f810419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f810a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109cc0 0x7f810419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f810a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f810410e3a0 0x7f81041a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f810a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8104119230 con 0x7f8104109cc0 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f810a800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81041190b0 con 0x7f810410e3a0 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f810a800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f81041193b0 con 0x7f8104109310 2024-04-25T19:00:24.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f80fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109cc0 0x7f810419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f8103400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8104109310 0x7f810419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f8103400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8104109310 0x7f810419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:54348/0 (socket says 172.21.15.86:54348) 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f8103400700 1 -- 172.21.15.86:0/3062451988 learned_addr learned my addr 172.21.15.86:0/3062451988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f80fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109cc0 0x7f810419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33900/0 (socket says 172.21.15.86:33900) 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f8103e00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f810410e3a0 0x7f81041a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.308+0000 7f8103400700 1 -- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f810410e3a0 msgr2=0x7f81041a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8103400700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f810410e3a0 0x7f81041a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8103400700 1 -- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109cc0 msgr2=0x7f810419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8103400700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8104109cc0 0x7f810419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8103400700 1 -- 172.21.15.86:0/3062451988 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81041a4730 con 0x7f8104109310 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8103400700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8104109310 0x7f810419f720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f80f4006010 tx=0x7f80f4005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4026070 con 0x7f8104109310 2024-04-25T19:00:24.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81041a49c0 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80f4003c70 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f400fbb0 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.309+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041ac2a0 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.310+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81040631d0 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.312+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f80f4029070 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.314+0000 7f8100c00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80e80a3cd0 0x7f80e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.314+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f80f40d13e0 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.314+0000 7f80fbe00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80e80a3cd0 0x7f80e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.314+0000 7f8100c00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] conn(0x7f80e80ad890 0x7f80e80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.314+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 --> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80e80b0400 con 0x7f80e80ad890 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.314+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f80f40d1790 con 0x7f8104109310 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.315+0000 7f8103e00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] conn(0x7f80e80ad890 0x7f80e80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:24.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.315+0000 7f80fbe00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80e80a3cd0 0x7f80e80a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009680 tx=0x7f80ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.315+0000 7f8103e00700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] conn(0x7f80e80ad890 0x7f80e80afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.316+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== osd.10 v2:172.21.15.183:6824/291756027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80e80b0400 con 0x7f80e80ad890 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.334+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 --> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81040635d0 con 0x7f80e80ad890 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.334+0000 7f8100c00700 1 -- 172.21.15.86:0/3062451988 <== osd.10 v2:172.21.15.183:6824/291756027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f81040635d0 con 0x7f80e80ad890 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] conn(0x7f80e80ad890 msgr2=0x7f80e80afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] conn(0x7f80e80ad890 0x7f80e80afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80e80a3cd0 msgr2=0x7f80e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:24.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f80e80a3cd0 0x7f80e80a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009680 tx=0x7f80ec009260 comp rx=0 tx=0).stop 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8104109310 msgr2=0x7f810419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 --2- 172.21.15.86:0/3062451988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8104109310 0x7f810419f720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f80f4006010 tx=0x7f80f4005280 comp rx=0 tx=0).stop 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f8103e00700 1 -- 172.21.15.86:0/3062451988 reap_dead start 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 shutdown_connections 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.335+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 >> 172.21.15.86:0/3062451988 conn(0x7f8104076210 msgr2=0x7f8104077600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.336+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 shutdown_connections 2024-04-25T19:00:24.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:23.336+0000 7f810a800700 1 -- 172.21.15.86:0/3062451988 wait complete. 2024-04-25T19:00:24.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:24 smithi086 ceph-mon[27783]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-25T19:00:24.709 INFO:teuthology.orchestra.run.smithi086.stdout:167503724613 2024-04-25T19:00:24.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T19:00:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:24 smithi194 ceph-mon[32740]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-25T19:00:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:24 smithi183 ceph-mon[32344]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-25T19:00:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 -- 172.21.15.86:0/3492113507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 msgr2=0x7f4254102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 --2- 172.21.15.86:0/3492113507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254102f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f423c009990 tx=0x7f423c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 -- 172.21.15.86:0/3492113507 shutdown_connections 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 --2- 172.21.15.86:0/3492113507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4254103450 0x7f425410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 --2- 172.21.15.86:0/3492113507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4254108a50 0x7f4254108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 --2- 172.21.15.86:0/3492113507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254102f10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 -- 172.21.15.86:0/3492113507 >> 172.21.15.86:0/3492113507 conn(0x7f4254079ba0 msgr2=0x7f4254079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 -- 172.21.15.86:0/3492113507 shutdown_connections 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.408+0000 7f425a600700 1 -- 172.21.15.86:0/3492113507 wait complete. 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.409+0000 7f425a600700 1 Processor -- start 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.409+0000 7f425a600700 1 -- start start 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f425a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f425a600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4254103450 0x7f42541996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f425a600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4254108a50 0x7f425419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f425a600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112c50 con 0x7f4254102a90 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f425a600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112ad0 con 0x7f4254103450 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f425a600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112dd0 con 0x7f4254108a50 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56246/0 (socket says 172.21.15.86:56246) 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253400700 1 -- 172.21.15.86:0/2696831762 learned_addr learned my addr 172.21.15.86:0/2696831762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f424be00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4254103450 0x7f42541996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253e00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4254108a50 0x7f425419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253400700 1 -- 172.21.15.86:0/2696831762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4254108a50 msgr2=0x7f425419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253400700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4254108a50 0x7f425419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.410+0000 7f4253400700 1 -- 172.21.15.86:0/2696831762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4254103450 msgr2=0x7f42541996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f4253400700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4254103450 0x7f42541996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f4253400700 1 -- 172.21.15.86:0/2696831762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425419e190 con 0x7f4254102a90 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f424be00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4254103450 0x7f42541996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f4253400700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254199180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f424400e9a0 tx=0x7f424400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424400ca10 con 0x7f4254102a90 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f424400cbb0 con 0x7f4254102a90 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244018680 con 0x7f4254102a90 2024-04-25T19:00:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425419e480 con 0x7f4254102a90 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.411+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42541a5d00 con 0x7f4254102a90 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.412+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42540631d0 con 0x7f4254102a90 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.415+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4244010430 con 0x7f4254102a90 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.416+0000 7f4250c00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f42380a3d50 0x7f42380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f42440d0be0 con 0x7f4254102a90 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f424be00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f42380a3d50 0x7f42380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f4250c00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] conn(0x7f42380ad910 0x7f42380afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 --> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4244019ec0 con 0x7f42380ad910 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f42440d0f90 con 0x7f4254102a90 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f4253e00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] conn(0x7f42380ad910 0x7f42380afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.417+0000 7f424be00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f42380a3d50 0x7f42380a6210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f425407a2f0 tx=0x7f423c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.418+0000 7f4253e00700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] conn(0x7f42380ad910 0x7f42380afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.419+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== osd.9 v2:172.21.15.86:6826/3110000530 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4244019ec0 con 0x7f42380ad910 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.438+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 --> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42540636a0 con 0x7f42380ad910 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f4250c00700 1 -- 172.21.15.86:0/2696831762 <== osd.9 v2:172.21.15.86:6826/3110000530 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f42540636a0 con 0x7f42380ad910 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] conn(0x7f42380ad910 msgr2=0x7f42380afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f425a600700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] conn(0x7f42380ad910 0x7f42380afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f42380a3d50 msgr2=0x7f42380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f425a600700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f42380a3d50 0x7f42380a6210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f425407a2f0 tx=0x7f423c00b260 comp rx=0 tx=0).stop 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 msgr2=0x7f4254199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.439+0000 7f425a600700 1 --2- 172.21.15.86:0/2696831762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4254102a90 0x7f4254199180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f424400e9a0 tx=0x7f424400ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.440+0000 7f4253e00700 1 -- 172.21.15.86:0/2696831762 reap_dead start 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.440+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 shutdown_connections 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.440+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 >> 172.21.15.86:0/2696831762 conn(0x7f4254079ba0 msgr2=0x7f4254101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.440+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 shutdown_connections 2024-04-25T19:00:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:17.440+0000 7f425a600700 1 -- 172.21.15.86:0/2696831762 wait complete. 2024-04-25T19:00:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 -- 172.21.15.86:0/3242456712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248010e3b0 msgr2=0x7f2480115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 --2- 172.21.15.86:0/3242456712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248010e3b0 0x7f2480115f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f247800b0d0 tx=0x7f247800b3e0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 -- 172.21.15.86:0/3242456712 shutdown_connections 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 --2- 172.21.15.86:0/3242456712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248010e3b0 0x7f2480115f80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 --2- 172.21.15.86:0/3242456712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24801092d0 0x7f24801096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 --2- 172.21.15.86:0/3242456712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2480109c80 0x7f248010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.566+0000 7f2487e00700 1 -- 172.21.15.86:0/3242456712 >> 172.21.15.86:0/3242456712 conn(0x7f2480076210 msgr2=0x7f2480107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.567+0000 7f2487e00700 1 -- 172.21.15.86:0/3242456712 shutdown_connections 2024-04-25T19:00:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.567+0000 7f2487e00700 1 -- 172.21.15.86:0/3242456712 wait complete. 2024-04-25T19:00:25.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.567+0000 7f2487e00700 1 Processor -- start 2024-04-25T19:00:25.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 -- start start 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24801092d0 0x7f248019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2480109c80 0x7f248019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f248010e3b0 0x7f24801a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24801192a0 con 0x7f24801092d0 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2480119120 con 0x7f248010e3b0 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2487e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2480119420 con 0x7f2480109c80 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2485400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24801092d0 0x7f248019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2485400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24801092d0 0x7f248019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33938/0 (socket says 172.21.15.86:33938) 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2485400700 1 -- 172.21.15.86:0/1184953076 learned_addr learned my addr 172.21.15.86:0/1184953076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.568+0000 7f2485e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f248010e3b0 0x7f24801a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2484a00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2480109c80 0x7f248019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2485e00700 1 -- 172.21.15.86:0/1184953076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2480109c80 msgr2=0x7f248019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2485e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2480109c80 0x7f248019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2485e00700 1 -- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24801092d0 msgr2=0x7f248019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2485e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24801092d0 0x7f248019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2485e00700 1 -- 172.21.15.86:0/1184953076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24801a46e0 con 0x7f248010e3b0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.569+0000 7f2485e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f248010e3b0 0x7f24801a3fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f247800b770 tx=0x7f2478007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.570+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247801f070 con 0x7f248010e3b0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.570+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2478007dc0 con 0x7f248010e3b0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.570+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24801a4970 con 0x7f248010e3b0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.570+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247801b430 con 0x7f248010e3b0 2024-04-25T19:00:25.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.570+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24801ac250 con 0x7f248010e3b0 2024-04-25T19:00:25.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.571+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24800631d0 con 0x7f248010e3b0 2024-04-25T19:00:25.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.573+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2478052050 con 0x7f248010e3b0 2024-04-25T19:00:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.574+0000 7f247e000700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f24680a3d50 0x7f24680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.574+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f247800f020 con 0x7f248010e3b0 2024-04-25T19:00:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.574+0000 7f2485400700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f24680a3d50 0x7f24680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.575+0000 7f247e000700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] conn(0x7f24680ad7e0 0x7f24680afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.575+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 --> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f247801de80 con 0x7f24680ad7e0 2024-04-25T19:00:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.575+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f24780d1db0 con 0x7f248010e3b0 2024-04-25T19:00:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.575+0000 7f2484a00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] conn(0x7f24680ad7e0 0x7f24680afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.575+0000 7f2485400700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f24680a3d50 0x7f24680a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2474009d20 tx=0x7f2474009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.576+0000 7f2484a00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] conn(0x7f24680ad7e0 0x7f24680afc80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:25.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.577+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== osd.3 v2:172.21.15.86:6810/2203765843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f247801de80 con 0x7f24680ad7e0 2024-04-25T19:00:25.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.594+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 --> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24800635d0 con 0x7f24680ad7e0 2024-04-25T19:00:25.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.595+0000 7f247e000700 1 -- 172.21.15.86:0/1184953076 <== osd.3 v2:172.21.15.86:6810/2203765843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f24800635d0 con 0x7f24680ad7e0 2024-04-25T19:00:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.595+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] conn(0x7f24680ad7e0 msgr2=0x7f24680afc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.595+0000 7f2487e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] conn(0x7f24680ad7e0 0x7f24680afc80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f24680a3d50 msgr2=0x7f24680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f24680a3d50 0x7f24680a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2474009d20 tx=0x7f2474009260 comp rx=0 tx=0).stop 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f248010e3b0 msgr2=0x7f24801a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 --2- 172.21.15.86:0/1184953076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f248010e3b0 0x7f24801a3fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f247800b770 tx=0x7f2478007980 comp rx=0 tx=0).stop 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2485e00700 1 -- 172.21.15.86:0/1184953076 reap_dead start 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 shutdown_connections 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 >> 172.21.15.86:0/1184953076 conn(0x7f2480076210 msgr2=0x7f2480076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 shutdown_connections 2024-04-25T19:00:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:25.596+0000 7f2487e00700 1 -- 172.21.15.86:0/1184953076 wait complete. 2024-04-25T19:00:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:26 smithi194 ceph-mon[32740]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-25T19:00:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:26 smithi086 ceph-mon[27783]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-25T19:00:26.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:26 smithi183 ceph-mon[32344]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-25T19:00:27.549 INFO:teuthology.orchestra.run.smithi086.stdout:223338299449 2024-04-25T19:00:27.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T19:00:28.095 INFO:teuthology.orchestra.run.smithi086.stdout:347892351013 2024-04-25T19:00:28.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 -- 172.21.15.86:0/3719532839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 msgr2=0x7f0380075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 --2- 172.21.15.86:0/3719532839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 0x7f0380075050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0374009990 tx=0x7f0374009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 -- 172.21.15.86:0/3719532839 shutdown_connections 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 --2- 172.21.15.86:0/3719532839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038010f4a0 0x7f0380111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 --2- 172.21.15.86:0/3719532839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 0x7f0380075050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 --2- 172.21.15.86:0/3719532839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0380075590 0x7f038010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 -- 172.21.15.86:0/3719532839 >> 172.21.15.86:0/3719532839 conn(0x7f0380079ba0 msgr2=0x7f0380079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 -- 172.21.15.86:0/3719532839 shutdown_connections 2024-04-25T19:00:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.455+0000 7f0387400700 1 -- 172.21.15.86:0/3719532839 wait complete. 2024-04-25T19:00:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.456+0000 7f0387400700 1 Processor -- start 2024-04-25T19:00:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 -- start start 2024-04-25T19:00:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 0x7f038019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0380075590 0x7f038019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f038010f4a0 0x7f038019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0380114eb0 con 0x7f0380074c70 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0380114d30 con 0x7f0380075590 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.457+0000 7f0387400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0380115030 con 0x7f038010f4a0 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f037fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0380075590 0x7f038019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f037fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0380075590 0x7f038019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:36518/0 (socket says 172.21.15.86:36518) 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f037fe00700 1 -- 172.21.15.86:0/4162605854 learned_addr learned my addr 172.21.15.86:0/4162605854 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f0385400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f038010f4a0 0x7f038019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f037fe00700 1 -- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f038010f4a0 msgr2=0x7f038019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f037fe00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f038010f4a0 0x7f038019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.458+0000 7f037fe00700 1 -- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 msgr2=0x7f038019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f0384a00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 0x7f038019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f037fe00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 0x7f038019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f037fe00700 1 -- 172.21.15.86:0/4162605854 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03801a0300 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f0385400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f038010f4a0 0x7f038019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f0384a00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0380074c70 0x7f038019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f037fe00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0380075590 0x7f038019b880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f036c00e9c0 tx=0x7f036c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c00cbd0 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c00cd70 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.459+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c005610 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.460+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03801a05f0 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.460+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03801a7e70 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.461+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0380063150 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.463+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f036c004750 con 0x7f0380075590 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.464+0000 7f037d600700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f03680a3d50 0x7f03680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.464+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f036c0d0ca0 con 0x7f0380075590 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.465+0000 7f0384a00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f03680a3d50 0x7f03680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.465+0000 7f037d600700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] conn(0x7f03680ad910 0x7f03680afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.465+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 --> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f036c01ee50 con 0x7f03680ad910 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.465+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f036c0d1050 con 0x7f0380075590 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.465+0000 7f0384a00700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f03680a3d50 0x7f03680a6210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f037400b2e0 tx=0x7f0374005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.465+0000 7f0385400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] conn(0x7f03680ad910 0x7f03680afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.466+0000 7f0385400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] conn(0x7f03680ad910 0x7f03680afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.467+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== osd.2 v2:172.21.15.194:6800/1582539385 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f036c01ee50 con 0x7f03680ad910 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.485+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 --> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03800635d0 con 0x7f03680ad910 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.485+0000 7f037d600700 1 -- 172.21.15.86:0/4162605854 <== osd.2 v2:172.21.15.194:6800/1582539385 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f03800635d0 con 0x7f03680ad910 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] conn(0x7f03680ad910 msgr2=0x7f03680afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] conn(0x7f03680ad910 0x7f03680afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f03680a3d50 msgr2=0x7f03680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f03680a3d50 0x7f03680a6210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f037400b2e0 tx=0x7f0374005fb0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0380075590 msgr2=0x7f038019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 --2- 172.21.15.86:0/4162605854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0380075590 0x7f038019b880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f036c00e9c0 tx=0x7f036c00ecd0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0385400700 1 -- 172.21.15.86:0/4162605854 reap_dead start 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 shutdown_connections 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 >> 172.21.15.86:0/4162605854 conn(0x7f0380079ba0 msgr2=0x7f03800ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 shutdown_connections 2024-04-25T19:00:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:27.486+0000 7f0387400700 1 -- 172.21.15.86:0/4162605854 wait complete. 2024-04-25T19:00:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:28 smithi086 ceph-mon[27783]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:28 smithi183 ceph-mon[32344]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:28 smithi194 ceph-mon[32740]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:29.252 INFO:teuthology.orchestra.run.smithi086.stdout:77309411416 2024-04-25T19:00:29.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T19:00:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:30 smithi086 ceph-mon[27783]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:30.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:30 smithi183 ceph-mon[32344]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:30 smithi194 ceph-mon[32740]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:00:31.082 INFO:teuthology.orchestra.run.smithi086.stdout:292057776173 2024-04-25T19:00:31.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T19:00:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 -- 172.21.15.86:0/3769971969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c108a70 msgr2=0x7f355c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 --2- 172.21.15.86:0/3769971969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c108a70 0x7f355c108e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f354c009990 tx=0x7f354c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 -- 172.21.15.86:0/3769971969 shutdown_connections 2024-04-25T19:00:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 --2- 172.21.15.86:0/3769971969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f355c103460 0x7f355c10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 --2- 172.21.15.86:0/3769971969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c108a70 0x7f355c108e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 --2- 172.21.15.86:0/3769971969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 0x7f355c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 -- 172.21.15.86:0/3769971969 >> 172.21.15.86:0/3769971969 conn(0x7f355c079bf0 msgr2=0x7f355c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 -- 172.21.15.86:0/3769971969 shutdown_connections 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.086+0000 7f3562a00700 1 -- 172.21.15.86:0/3769971969 wait complete. 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.087+0000 7f3562a00700 1 Processor -- start 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.087+0000 7f3562a00700 1 -- start start 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3562a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 0x7f355c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3562a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c103460 0x7f355c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3562a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f355c108a70 0x7f355c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3562a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f355c112c50 con 0x7f355c102ac0 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3562a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f355c112ad0 con 0x7f355c108a70 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3562a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f355c112dd0 con 0x7f355c103460 2024-04-25T19:00:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f355be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 0x7f355c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f355be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 0x7f355c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36794/0 (socket says 172.21.15.86:36794) 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f355be00700 1 -- 172.21.15.86:0/2175941170 learned_addr learned my addr 172.21.15.86:0/2175941170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f355b400700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c103460 0x7f355c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.088+0000 7f3560a00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f355c108a70 0x7f355c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f355be00700 1 -- 172.21.15.86:0/2175941170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c103460 msgr2=0x7f355c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f355be00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f355c103460 0x7f355c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f355be00700 1 -- 172.21.15.86:0/2175941170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f355c108a70 msgr2=0x7f355c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f355be00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f355c108a70 0x7f355c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f355be00700 1 -- 172.21.15.86:0/2175941170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1a0440 con 0x7f355c102ac0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f355be00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 0x7f355c19b430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f354c005910 tx=0x7f354c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c027070 con 0x7f355c102ac0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c1a06d0 con 0x7f355c102ac0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354c022470 con 0x7f355c102ac0 2024-04-25T19:00:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c017650 con 0x7f355c102ac0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.089+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a7fb0 con 0x7f355c102ac0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.090+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f355c101290 con 0x7f355c102ac0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.092+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f354c02a070 con 0x7f355c102ac0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.094+0000 7f3558c00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35400a3d50 0x7f35400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.094+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f354c0d1680 con 0x7f355c102ac0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.094+0000 7f355b400700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35400a3d50 0x7f35400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.094+0000 7f3558c00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] conn(0x7f35400ad910 0x7f35400afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.094+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 --> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f354c0095d0 con 0x7f35400ad910 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.094+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f354c0d1a30 con 0x7f355c102ac0 2024-04-25T19:00:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.095+0000 7f3560a00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] conn(0x7f35400ad910 0x7f35400afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.095+0000 7f355b400700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35400a3d50 0x7f35400a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3548009680 tx=0x7f3548009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.095+0000 7f3560a00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] conn(0x7f35400ad910 0x7f35400afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.097+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== osd.6 v2:172.21.15.86:6818/1838185370 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f354c0095d0 con 0x7f35400ad910 2024-04-25T19:00:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.114+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 --> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f355c1041b0 con 0x7f35400ad910 2024-04-25T19:00:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.115+0000 7f3558c00700 1 -- 172.21.15.86:0/2175941170 <== osd.6 v2:172.21.15.86:6818/1838185370 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f355c1041b0 con 0x7f35400ad910 2024-04-25T19:00:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.115+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] conn(0x7f35400ad910 msgr2=0x7f35400afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.115+0000 7f3562a00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] conn(0x7f35400ad910 0x7f35400afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.115+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35400a3d50 msgr2=0x7f35400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.115+0000 7f3562a00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f35400a3d50 0x7f35400a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3548009680 tx=0x7f3548009260 comp rx=0 tx=0).stop 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 msgr2=0x7f355c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3562a00700 1 --2- 172.21.15.86:0/2175941170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c102ac0 0x7f355c19b430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f354c005910 tx=0x7f354c00b670 comp rx=0 tx=0).stop 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3560a00700 1 -- 172.21.15.86:0/2175941170 reap_dead start 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 shutdown_connections 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 >> 172.21.15.86:0/2175941170 conn(0x7f355c079bf0 msgr2=0x7f355c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 shutdown_connections 2024-04-25T19:00:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.116+0000 7f3562a00700 1 -- 172.21.15.86:0/2175941170 wait complete. 2024-04-25T19:00:32.192 INFO:teuthology.orchestra.run.smithi086.stdout:111669149777 2024-04-25T19:00:32.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T19:00:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 -- 172.21.15.86:0/1317229618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 msgr2=0x7f4f0406ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 --2- 172.21.15.86:0/1317229618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f0406ab30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009990 tx=0x7f4ef8009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 -- 172.21.15.86:0/1317229618 shutdown_connections 2024-04-25T19:00:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 --2- 172.21.15.86:0/1317229618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f0410f4f0 0x7f4f041118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 --2- 172.21.15.86:0/1317229618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f0406ab30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 --2- 172.21.15.86:0/1317229618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f0406b070 0x7f4f0410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 -- 172.21.15.86:0/1317229618 >> 172.21.15.86:0/1317229618 conn(0x7f4f040788f0 msgr2=0x7f4f04078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 -- 172.21.15.86:0/1317229618 shutdown_connections 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.884+0000 7f4f0be00700 1 -- 172.21.15.86:0/1317229618 wait complete. 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.885+0000 7f4f0be00700 1 Processor -- start 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 -- start start 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f04103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f0406b070 0x7f4f04104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f0410f4f0 0x7f4f041046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04114e00 con 0x7f4f0406a750 2024-04-25T19:00:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04114c80 con 0x7f4f0410f4f0 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f0be00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04114f80 con 0x7f4f0406b070 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.886+0000 7f4f09400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f04103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f04103c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36818/0 (socket says 172.21.15.86:36818) 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 -- 172.21.15.86:0/768106197 learned_addr learned my addr 172.21.15.86:0/768106197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f08a00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f0406b070 0x7f4f04104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09e00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f0410f4f0 0x7f4f041046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 -- 172.21.15.86:0/768106197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f0406b070 msgr2=0x7f4f04104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f0406b070 0x7f4f04104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 -- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f0410f4f0 msgr2=0x7f4f041046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f0410f4f0 0x7f4f041046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 -- 172.21.15.86:0/768106197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f041ab9c0 con 0x7f4f0406a750 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09e00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f0410f4f0 0x7f4f041046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f09400700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f04103c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80092e0 tx=0x7f4ef80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.887+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8026070 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.888+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef8021410 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.888+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041abc50 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.888+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8017650 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.888+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041ac0a0 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.889+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f040631d0 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.891+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ef8029070 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.892+0000 7f4f02000700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4eec0a3d00 0x7f4eec0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f4ef80d1d00 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f08a00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4eec0a3d00 0x7f4eec0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f02000700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] conn(0x7f4eec0ad8c0 0x7f4eec0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 --> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eec0b0430 con 0x7f4eec0ad8c0 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4ef80d20b0 con 0x7f4f0406a750 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f09e00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] conn(0x7f4eec0ad8c0 0x7f4eec0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.893+0000 7f4f08a00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4eec0a3d00 0x7f4eec0a61c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009680 tx=0x7f4ef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.894+0000 7f4f09e00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] conn(0x7f4eec0ad8c0 0x7f4eec0afd60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.895+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== osd.1 v2:172.21.15.183:6800/4147786843 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec0b0430 con 0x7f4eec0ad8c0 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.913+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 --> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f04063650 con 0x7f4eec0ad8c0 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.914+0000 7f4f02000700 1 -- 172.21.15.86:0/768106197 <== osd.1 v2:172.21.15.183:6800/4147786843 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f04063650 con 0x7f4eec0ad8c0 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.914+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] conn(0x7f4eec0ad8c0 msgr2=0x7f4eec0afd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.914+0000 7f4f0be00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] conn(0x7f4eec0ad8c0 0x7f4eec0afd60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.914+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4eec0a3d00 msgr2=0x7f4eec0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4eec0a3d00 0x7f4eec0a61c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009680 tx=0x7f4ef0009260 comp rx=0 tx=0).stop 2024-04-25T19:00:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 msgr2=0x7f4f04103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 --2- 172.21.15.86:0/768106197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0406a750 0x7f4f04103c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80092e0 tx=0x7f4ef80047b0 comp rx=0 tx=0).stop 2024-04-25T19:00:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f09e00700 1 -- 172.21.15.86:0/768106197 reap_dead start 2024-04-25T19:00:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 shutdown_connections 2024-04-25T19:00:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 >> 172.21.15.86:0/768106197 conn(0x7f4f040788f0 msgr2=0x7f4f041109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 shutdown_connections 2024-04-25T19:00:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:31.915+0000 7f4f0be00700 1 -- 172.21.15.86:0/768106197 wait complete. 2024-04-25T19:00:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:32 smithi086 ceph-mon[27783]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T19:00:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:32 smithi183 ceph-mon[32344]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T19:00:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:32 smithi194 ceph-mon[32740]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T19:00:33.390 INFO:teuthology.orchestra.run.smithi086.stdout:313532612650 2024-04-25T19:00:33.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T19:00:33.709 INFO:teuthology.orchestra.run.smithi086.stdout:55834574941 2024-04-25T19:00:33.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T19:00:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 -- 172.21.15.86:0/3790320388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064103450 msgr2=0x7f706410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 --2- 172.21.15.86:0/3790320388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064103450 0x7f706410f8b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f705c009f90 tx=0x7f705c00b330 comp rx=0 tx=0).stop 2024-04-25T19:00:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 -- 172.21.15.86:0/3790320388 shutdown_connections 2024-04-25T19:00:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 --2- 172.21.15.86:0/3790320388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064103450 0x7f706410f8b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 --2- 172.21.15.86:0/3790320388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7064108a50 0x7f7064108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 --2- 172.21.15.86:0/3790320388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064102a90 0x7f7064102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.923+0000 7f7068e00700 1 -- 172.21.15.86:0/3790320388 >> 172.21.15.86:0/3790320388 conn(0x7f7064079ba0 msgr2=0x7f7064079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.924+0000 7f7068e00700 1 -- 172.21.15.86:0/3790320388 shutdown_connections 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.924+0000 7f7068e00700 1 -- 172.21.15.86:0/3790320388 wait complete. 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.925+0000 7f7068e00700 1 Processor -- start 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.925+0000 7f7068e00700 1 -- start start 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7068e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064102a90 0x7f706419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7068e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064103450 0x7f706419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7068e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7064108a50 0x7f706419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7068e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7064112be0 con 0x7f7064103450 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7068e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7064112a60 con 0x7f7064102a90 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7068e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7064112d60 con 0x7f7064108a50 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7061400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064103450 0x7f706419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7061400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064103450 0x7f706419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36836/0 (socket says 172.21.15.86:36836) 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7061400700 1 -- 172.21.15.86:0/1996901003 learned_addr learned my addr 172.21.15.86:0/1996901003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7062800700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7064108a50 0x7f706419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.926+0000 7f7061e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064102a90 0x7f706419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7062800700 1 -- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064102a90 msgr2=0x7f706419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7062800700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064102a90 0x7f706419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7062800700 1 -- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064103450 msgr2=0x7f706419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7062800700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064103450 0x7f706419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7062800700 1 -- 172.21.15.86:0/1996901003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70641a0300 con 0x7f7064108a50 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7061400700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7064103450 0x7f706419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7061e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7064102a90 0x7f706419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7062800700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7064108a50 0x7f706419fbc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f705c009790 tx=0x7f705c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c021070 con 0x7f7064108a50 2024-04-25T19:00:34.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f705c007ac0 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c005040 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.927+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641a0590 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.928+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641a7e70 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.928+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f706410f930 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.931+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f705c024070 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.932+0000 7f705aa00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f70480a3d50 0x7f70480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.932+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f705c0d16b0 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.932+0000 7f7061e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f70480a3d50 0x7f70480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.933+0000 7f705aa00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] conn(0x7f70480ad7e0 0x7f70480afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.933+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 --> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f705c01fe80 con 0x7f70480ad7e0 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.933+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f705c0d1a60 con 0x7f7064108a50 2024-04-25T19:00:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.933+0000 7f7061400700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] conn(0x7f70480ad7e0 0x7f70480afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.933+0000 7f7061e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f70480a3d50 0x7f70480a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7054009fd0 tx=0x7f7054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.934+0000 7f7061400700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] conn(0x7f70480ad7e0 0x7f70480afc80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.934+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== osd.4 v2:172.21.15.183:6808/3004862386 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f705c01fe80 con 0x7f70480ad7e0 2024-04-25T19:00:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.954+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 --> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7064104180 con 0x7f70480ad7e0 2024-04-25T19:00:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.955+0000 7f705aa00700 1 -- 172.21.15.86:0/1996901003 <== osd.4 v2:172.21.15.183:6808/3004862386 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7064104180 con 0x7f70480ad7e0 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.955+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] conn(0x7f70480ad7e0 msgr2=0x7f70480afc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.955+0000 7f7068e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] conn(0x7f70480ad7e0 0x7f70480afc80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f70480a3d50 msgr2=0x7f70480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f70480a3d50 0x7f70480a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7054009fd0 tx=0x7f7054009260 comp rx=0 tx=0).stop 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7064108a50 msgr2=0x7f706419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 --2- 172.21.15.86:0/1996901003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7064108a50 0x7f706419fbc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f705c009790 tx=0x7f705c00ccb0 comp rx=0 tx=0).stop 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7062800700 1 -- 172.21.15.86:0/1996901003 reap_dead start 2024-04-25T19:00:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 shutdown_connections 2024-04-25T19:00:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 >> 172.21.15.86:0/1996901003 conn(0x7f7064079ba0 msgr2=0x7f706410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 shutdown_connections 2024-04-25T19:00:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:33.956+0000 7f7068e00700 1 -- 172.21.15.86:0/1996901003 wait complete. 2024-04-25T19:00:34.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:34 smithi086 ceph-mon[27783]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T19:00:34.884 INFO:teuthology.orchestra.run.smithi086.stdout:201863462978 2024-04-25T19:00:34.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T19:00:34.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:34 smithi183 ceph-mon[32344]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T19:00:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:34 smithi194 ceph-mon[32740]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T19:00:35.543 INFO:teuthology.orchestra.run.smithi086.stdout:146028888141 2024-04-25T19:00:35.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 -- 172.21.15.86:0/3692988501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5898109c80 msgr2=0x7f589810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 --2- 172.21.15.86:0/3692988501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5898109c80 0x7f589810dcd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5884009920 tx=0x7f5884009c30 comp rx=0 tx=0).stop 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 -- 172.21.15.86:0/3692988501 shutdown_connections 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 --2- 172.21.15.86:0/3692988501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589810e3b0 0x7f5898115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 --2- 172.21.15.86:0/3692988501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58981092d0 0x7f58981096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 --2- 172.21.15.86:0/3692988501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5898109c80 0x7f589810dcd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 -- 172.21.15.86:0/3692988501 >> 172.21.15.86:0/3692988501 conn(0x7f5898076210 msgr2=0x7f5898107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:36.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 -- 172.21.15.86:0/3692988501 shutdown_connections 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.185+0000 7f589ee00700 1 -- 172.21.15.86:0/3692988501 wait complete. 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.186+0000 7f589ee00700 1 Processor -- start 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.186+0000 7f589ee00700 1 -- start start 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.186+0000 7f589ee00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58981092d0 0x7f58981a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f589ee00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5898109c80 0x7f58981a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f589ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589810e3b0 0x7f589819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f589ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5898119270 con 0x7f589810e3b0 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f589ee00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58981190f0 con 0x7f5898109c80 2024-04-25T19:00:36.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f589ee00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f58981193f0 con 0x7f58981092d0 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58981092d0 0x7f58981a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58981092d0 0x7f58981a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:57844/0 (socket says 172.21.15.86:57844) 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 -- 172.21.15.86:0/296860630 learned_addr learned my addr 172.21.15.86:0/296860630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f589ce00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589810e3b0 0x7f589819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 -- 172.21.15.86:0/296860630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5898109c80 msgr2=0x7f58981a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5898109c80 0x7f58981a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 -- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589810e3b0 msgr2=0x7f589819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589810e3b0 0x7f589819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.187+0000 7f5897e00700 1 -- 172.21.15.86:0/296860630 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589819ff10 con 0x7f58981092d0 2024-04-25T19:00:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.188+0000 7f5897e00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58981092d0 0x7f58981a5560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f588800e9a0 tx=0x7f588800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:36.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.188+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588800cbd0 con 0x7f58981092d0 2024-04-25T19:00:36.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.188+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588800cd70 con 0x7f58981092d0 2024-04-25T19:00:36.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.188+0000 7f589ee00700 1 -- 172.21.15.86:0/296860630 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58981a0200 con 0x7f58981092d0 2024-04-25T19:00:36.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.188+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888018680 con 0x7f58981092d0 2024-04-25T19:00:36.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.188+0000 7f589ee00700 1 -- 172.21.15.86:0/296860630 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58981ac4e0 con 0x7f58981092d0 2024-04-25T19:00:36.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.189+0000 7f589ee00700 1 -- 172.21.15.86:0/296860630 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5898063190 con 0x7f58981092d0 2024-04-25T19:00:36.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.192+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5888010430 con 0x7f58981092d0 2024-04-25T19:00:36.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.193+0000 7f5894c00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f587c0a3d50 0x7f587c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:36.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.193+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f58880d0fe0 con 0x7f58981092d0 2024-04-25T19:00:36.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.193+0000 7f5897400700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f587c0a3d50 0x7f587c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:36.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.194+0000 7f5894c00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] conn(0x7f587c0ad910 0x7f587c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:36.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.194+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 --> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5888027e50 con 0x7f587c0ad910 2024-04-25T19:00:36.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.194+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f58880d1430 con 0x7f58981092d0 2024-04-25T19:00:36.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.194+0000 7f589ce00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] conn(0x7f587c0ad910 0x7f587c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:36.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.194+0000 7f5897400700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f587c0a3d50 0x7f587c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5884009230 tx=0x7f5884009f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:36.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.195+0000 7f589ce00700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] conn(0x7f587c0ad910 0x7f587c0afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.196+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== osd.8 v2:172.21.15.194:6816/1969375394 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5888027e50 con 0x7f587c0ad910 2024-04-25T19:00:36.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.224+0000 7f589ee00700 1 -- 172.21.15.86:0/296860630 --> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5898063590 con 0x7f587c0ad910 2024-04-25T19:00:36.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.225+0000 7f5894c00700 1 -- 172.21.15.86:0/296860630 <== osd.8 v2:172.21.15.194:6816/1969375394 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5898063590 con 0x7f587c0ad910 2024-04-25T19:00:36.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.228+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] conn(0x7f587c0ad910 msgr2=0x7f587c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:36.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.228+0000 7f588e000700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] conn(0x7f587c0ad910 0x7f587c0afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f587c0a3d50 msgr2=0x7f587c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f588e000700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f587c0a3d50 0x7f587c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5884009230 tx=0x7f5884009f80 comp rx=0 tx=0).stop 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58981092d0 msgr2=0x7f58981a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f588e000700 1 --2- 172.21.15.86:0/296860630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58981092d0 0x7f58981a5560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f588800e9a0 tx=0x7f588800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f589ce00700 1 -- 172.21.15.86:0/296860630 reap_dead start 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 shutdown_connections 2024-04-25T19:00:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.230+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 >> 172.21.15.86:0/296860630 conn(0x7f5898076210 msgr2=0x7f5898077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:36.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.234+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 shutdown_connections 2024-04-25T19:00:36.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:36.234+0000 7f588e000700 1 -- 172.21.15.86:0/296860630 wait complete. 2024-04-25T19:00:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:36.248 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:36.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:36 smithi086 ceph-mon[27783]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-25T19:00:36.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:36 smithi183 ceph-mon[32344]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-25T19:00:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:36 smithi194 ceph-mon[32740]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-25T19:00:37.504 INFO:teuthology.orchestra.run.smithi086.stdout:257698037816 2024-04-25T19:00:37.504 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T19:00:37.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T19:00:37.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T19:00:37.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:37 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-04-25T19:00:37.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T19:00:37.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T19:00:37.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:37 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-04-25T19:00:37.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T19:00:37.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T19:00:37.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:37 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-04-25T19:00:38.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:38 smithi086 ceph-mon[27783]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T19:00:38.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T19:00:38.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T19:00:38.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:38 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-04-25T19:00:38.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:38 smithi086 ceph-mon[27783]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T19:00:38.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:38 smithi183 ceph-mon[32344]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T19:00:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T19:00:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T19:00:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:38 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-04-25T19:00:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:38 smithi183 ceph-mon[32344]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T19:00:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:38 smithi194 ceph-mon[32740]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T19:00:38.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T19:00:38.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T19:00:38.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:38 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-04-25T19:00:38.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:38 smithi194 ceph-mon[32740]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T19:00:39.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:39 smithi086 ceph-mon[27783]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T19:00:39.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:39 smithi086 ceph-mon[27783]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T19:00:39.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:39 smithi086 ceph-mon[27783]: pgmap v384: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-25T19:00:39.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:39 smithi183 ceph-mon[32344]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T19:00:39.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:39 smithi183 ceph-mon[32344]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T19:00:39.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:39 smithi183 ceph-mon[32344]: pgmap v384: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-25T19:00:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:39 smithi194 ceph-mon[32740]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T19:00:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:39 smithi194 ceph-mon[32740]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T19:00:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:39 smithi194 ceph-mon[32740]: pgmap v384: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 8 op/s 2024-04-25T19:00:40.597 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:41.737 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:42.672 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:42 smithi086 ceph-mon[27783]: pgmap v385: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-25T19:00:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:42 smithi194 ceph-mon[32740]: pgmap v385: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-25T19:00:42.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:42 smithi183 ceph-mon[32344]: pgmap v385: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.175+0000 7fa6d1c00700 1 -- 172.21.15.86:0/530022114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc108970 msgr2=0x7fa6cc108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.175+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/530022114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc108970 0x7fa6cc108d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009990 tx=0x7fa6bc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 -- 172.21.15.86:0/530022114 shutdown_connections 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/530022114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6cc103330 0x7fa6cc10f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/530022114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc108970 0x7fa6cc108d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/530022114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 -- 172.21.15.86:0/530022114 >> 172.21.15.86:0/530022114 conn(0x7fa6cc0fe4c0 msgr2=0x7fa6cc1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 -- 172.21.15.86:0/530022114 shutdown_connections 2024-04-25T19:00:43.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.176+0000 7fa6d1c00700 1 -- 172.21.15.86:0/530022114 wait complete. 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.177+0000 7fa6d1c00700 1 Processor -- start 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 -- start start 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc103330 0x7fa6cc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6cc108970 0x7fa6cc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc112bd0 con 0x7fa6cc103330 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc112a50 con 0x7fa6cc102970 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc112d50 con 0x7fa6cc108970 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6cb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc103330 0x7fa6cc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6cbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6d1200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6cc108970 0x7fa6cc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6cbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60924/0 (socket says 172.21.15.86:60924) 2024-04-25T19:00:43.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6cb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc103330 0x7fa6cc075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35388/0 (socket says 172.21.15.86:35388) 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.178+0000 7fa6cbe00700 1 -- 172.21.15.86:0/2999161592 learned_addr learned my addr 172.21.15.86:0/2999161592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6cbe00700 1 -- 172.21.15.86:0/2999161592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6cc108970 msgr2=0x7fa6cc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6cbe00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6cc108970 0x7fa6cc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6cbe00700 1 -- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc103330 msgr2=0x7fa6cc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6cbe00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc103330 0x7fa6cc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6cbe00700 1 -- 172.21.15.86:0/2999161592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc1a75d0 con 0x7fa6cc102970 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6cbe00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005910 tx=0x7fa6bc00b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.179+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc027070 con 0x7fa6cc102970 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.180+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc1a7860 con 0x7fa6cc102970 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.180+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6bc022470 con 0x7fa6cc102970 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.180+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc017650 con 0x7fa6cc102970 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.180+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc1a7d10 con 0x7fa6cc102970 2024-04-25T19:00:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.181+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc069730 con 0x7fa6cc102970 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.183+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa6bc02a070 con 0x7fa6cc102970 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.184+0000 7fa6c8c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6b00a3c40 0x7fa6b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.184+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa6bc0d1360 con 0x7fa6cc102970 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.184+0000 7fa6cb400700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6b00a3c40 0x7fa6b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.185+0000 7fa6cb400700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6b00a3c40 0x7fa6b00a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009680 tx=0x7fa6b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.187+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6bc013070 con 0x7fa6cc102970 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.431+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa6cc02df80 con 0x7fa6cc102970 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.432+0000 7fa6c8c00700 1 -- 172.21.15.86:0/2999161592 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6bc09dd10 con 0x7fa6cc102970 2024-04-25T19:00:43.454 INFO:teuthology.orchestra.run.smithi086.stdout:167503724618 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6b00a3c40 msgr2=0x7fa6b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6b00a3c40 0x7fa6b00a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009680 tx=0x7fa6b8009260 comp rx=0 tx=0).stop 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 msgr2=0x7fa6cc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc074d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005910 tx=0x7fa6bc00b670 comp rx=0 tx=0).stop 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 shutdown_connections 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6b00a3c40 0x7fa6b00a6100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6cc108970 0x7fa6cc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6cc102970 0x7fa6cc074d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 --2- 172.21.15.86:0/2999161592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6cc103330 0x7fa6cc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 >> 172.21.15.86:0/2999161592 conn(0x7fa6cc0fe4c0 msgr2=0x7fa6cc0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 shutdown_connections 2024-04-25T19:00:43.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.435+0000 7fa6d1c00700 1 -- 172.21.15.86:0/2999161592 wait complete. 2024-04-25T19:00:43.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:43 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2999161592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T19:00:43.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:43 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/422750658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T19:00:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:43 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2999161592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T19:00:43.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:43 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/422750658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T19:00:43.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:43 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2999161592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T19:00:43.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:43 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/422750658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T19:00:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:44 smithi194 ceph-mon[32740]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-25T19:00:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:44 smithi086 ceph-mon[27783]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 -- 172.21.15.86:0/4241745057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7958103950 msgr2=0x7f795806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 --2- 172.21.15.86:0/4241745057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7958103950 0x7f795806b110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f794c009990 tx=0x7f794c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 -- 172.21.15.86:0/4241745057 shutdown_connections 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 --2- 172.21.15.86:0/4241745057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f795810f390 0x7f7958111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 --2- 172.21.15.86:0/4241745057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7958103950 0x7f795806b110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 --2- 172.21.15.86:0/4241745057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 -- 172.21.15.86:0/4241745057 >> 172.21.15.86:0/4241745057 conn(0x7f79580788f0 msgr2=0x7f7958078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 -- 172.21.15.86:0/4241745057 shutdown_connections 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.986+0000 7f7960400700 1 -- 172.21.15.86:0/4241745057 wait complete. 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.987+0000 7f7960400700 1 Processor -- start 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 -- start start 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7958103950 0x7f795819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f795810f390 0x7f795819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7958114e20 con 0x7f795806b650 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7958114ca0 con 0x7f7958103950 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f7960400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7958114fa0 con 0x7f795810f390 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f795da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f795da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35412/0 (socket says 172.21.15.86:35412) 2024-04-25T19:00:44.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f795da00700 1 -- 172.21.15.86:0/422750658 learned_addr learned my addr 172.21.15.86:0/422750658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f795d000700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7958103950 0x7f795819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.988+0000 7f795e400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f795810f390 0x7f795819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795d000700 1 -- 172.21.15.86:0/422750658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f795810f390 msgr2=0x7f795819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795d000700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f795810f390 0x7f795819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795d000700 1 -- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 msgr2=0x7f795819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795d000700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795d000700 1 -- 172.21.15.86:0/422750658 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79581a0320 con 0x7f7958103950 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795da00700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f795d000700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7958103950 0x7f795819b850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9f0 tx=0x7f794400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:44.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794400cb70 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794400ee90 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.989+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79581a0610 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.990+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944003d40 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.991+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a7e90 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.992+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7958101f80 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.993+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7944003ee0 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.994+0000 7f7956000700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f79400a3cc0 0x7f79400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.994+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f79440d0550 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.994+0000 7f795da00700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f79400a3cc0 0x7f79400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.995+0000 7f795da00700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f79400a3cc0 0x7f79400a6180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f794c009750 tx=0x7f794c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:42.998+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f794409ce70 con 0x7f7958103950 2024-04-25T19:00:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.238+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f79581a8090 con 0x7f7958103950 2024-04-25T19:00:44.880 INFO:teuthology.orchestra.run.smithi086.stdout:223338299453 2024-04-25T19:00:44.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.238+0000 7f7956000700 1 -- 172.21.15.86:0/422750658 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f794409c770 con 0x7f7958103950 2024-04-25T19:00:44.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f79400a3cc0 msgr2=0x7f79400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:44.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f79400a3cc0 0x7f79400a6180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f794c009750 tx=0x7f794c01a040 comp rx=0 tx=0).stop 2024-04-25T19:00:44.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7958103950 msgr2=0x7f795819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:44.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7958103950 0x7f795819b850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9f0 tx=0x7f794400ed00 comp rx=0 tx=0).stop 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 shutdown_connections 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f79400a3cc0 0x7f79400a6180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f795810f390 0x7f795819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795806b650 0x7f795819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 --2- 172.21.15.86:0/422750658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7958103950 0x7f795819b850 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 >> 172.21.15.86:0/422750658 conn(0x7f79580788f0 msgr2=0x7f7958100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.241+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 shutdown_connections 2024-04-25T19:00:44.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:43.242+0000 7f7960400700 1 -- 172.21.15.86:0/422750658 wait complete. 2024-04-25T19:00:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:44 smithi183 ceph-mon[32344]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.381+0000 7ffb2b400700 1 -- 172.21.15.86:0/3846638376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 msgr2=0x7ffb24115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.381+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3846638376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 0x7ffb24115f70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009fc0 tx=0x7ffb1c00b330 comp rx=0 tx=0).stop 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 -- 172.21.15.86:0/3846638376 shutdown_connections 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3846638376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 0x7ffb24115f70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3846638376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb24109310 0x7ffb241096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3846638376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109cc0 0x7ffb2410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 -- 172.21.15.86:0/3846638376 >> 172.21.15.86:0/3846638376 conn(0x7ffb24076210 msgr2=0x7ffb24078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 -- 172.21.15.86:0/3846638376 shutdown_connections 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.382+0000 7ffb2b400700 1 -- 172.21.15.86:0/3846638376 wait complete. 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.383+0000 7ffb2b400700 1 Processor -- start 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.383+0000 7ffb2b400700 1 -- start start 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb2b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 0x7ffb2419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb2b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb24109cc0 0x7ffb2419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb2b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 0x7ffb241a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb2b400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241192b0 con 0x7ffb24109310 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb2b400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24119130 con 0x7ffb24109cc0 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb2b400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24119430 con 0x7ffb2410e3a0 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 0x7ffb2419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 0x7ffb2419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35424/0 (socket says 172.21.15.86:35424) 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 -- 172.21.15.86:0/3002172487 learned_addr learned my addr 172.21.15.86:0/3002172487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb29400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 0x7ffb241a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb23e00700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb24109cc0 0x7ffb2419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 -- 172.21.15.86:0/3002172487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 msgr2=0x7ffb241a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 0x7ffb241a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 -- 172.21.15.86:0/3002172487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb24109cc0 msgr2=0x7ffb2419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb24109cc0 0x7ffb2419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.384+0000 7ffb28a00700 1 -- 172.21.15.86:0/3002172487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb241a4730 con 0x7ffb24109310 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.385+0000 7ffb28a00700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 0x7ffb2419f720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9c0 tx=0x7ffb1800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.385+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800cbd0 con 0x7ffb24109310 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.385+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb241a4a20 con 0x7ffb24109310 2024-04-25T19:00:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.385+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb1800ee60 con 0x7ffb24109310 2024-04-25T19:00:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.385+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180055c0 con 0x7ffb24109310 2024-04-25T19:00:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.385+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb241ac2a0 con 0x7ffb24109310 2024-04-25T19:00:46.339 INFO:teuthology.orchestra.run.smithi086.stdout:34359738471 2024-04-25T19:00:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.386+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb24069730 con 0x7ffb24109310 2024-04-25T19:00:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.388+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffb18004750 con 0x7ffb24109310 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.389+0000 7ffb21600700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ffb0c0a3c40 0x7ffb0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.390+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ffb180d0770 con 0x7ffb24109310 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.390+0000 7ffb23e00700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ffb0c0a3c40 0x7ffb0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.390+0000 7ffb23e00700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ffb0c0a3c40 0x7ffb0c0a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009750 tx=0x7ffb10009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.394+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb1809d090 con 0x7ffb24109310 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.633+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffb241a0a80 con 0x7ffb24109310 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.633+0000 7ffb21600700 1 -- 172.21.15.86:0/3002172487 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb1809c990 con 0x7ffb24109310 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ffb0c0a3c40 msgr2=0x7ffb0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ffb0c0a3c40 0x7ffb0c0a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009750 tx=0x7ffb10009310 comp rx=0 tx=0).stop 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 msgr2=0x7ffb2419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 0x7ffb2419f720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800e9c0 tx=0x7ffb1800ecd0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 shutdown_connections 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ffb0c0a3c40 0x7ffb0c0a6100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb2410e3a0 0x7ffb241a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24109310 0x7ffb2419f720 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 --2- 172.21.15.86:0/3002172487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb24109cc0 0x7ffb2419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 >> 172.21.15.86:0/3002172487 conn(0x7ffb24076210 msgr2=0x7ffb240784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 shutdown_connections 2024-04-25T19:00:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:45.636+0000 7ffb2b400700 1 -- 172.21.15.86:0/3002172487 wait complete. 2024-04-25T19:00:46.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:46 smithi086 ceph-mon[27783]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-25T19:00:46.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:46 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3002172487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T19:00:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:46 smithi194 ceph-mon[32740]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-25T19:00:46.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:46 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3002172487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T19:00:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:46 smithi183 ceph-mon[32344]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-25T19:00:46.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:46 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3002172487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T19:00:47.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299449 got 223338299453 for osd.7 2024-04-25T19:00:47.611 DEBUG:teuthology.parallel:result is None 2024-04-25T19:00:47.844 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:47.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 -- 172.21.15.86:0/130365475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4075590 msgr2=0x7f3ef410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 --2- 172.21.15.86:0/130365475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4075590 0x7f3ef410ef00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009990 tx=0x7f3ee0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 -- 172.21.15.86:0/130365475 shutdown_connections 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 --2- 172.21.15.86:0/130365475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef410f440 0x7f3ef4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 --2- 172.21.15.86:0/130365475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ef4074c70 0x7f3ef4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 --2- 172.21.15.86:0/130365475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4075590 0x7f3ef410ef00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 -- 172.21.15.86:0/130365475 >> 172.21.15.86:0/130365475 conn(0x7f3ef4079ba0 msgr2=0x7f3ef4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 -- 172.21.15.86:0/130365475 shutdown_connections 2024-04-25T19:00:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.873+0000 7f3efba00700 1 -- 172.21.15.86:0/130365475 wait complete. 2024-04-25T19:00:47.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 Processor -- start 2024-04-25T19:00:47.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 -- start start 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 0x7f3ef419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef4075590 0x7f3ef419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ef410f440 0x7f3ef419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4114e00 con 0x7f3ef4075590 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4114c80 con 0x7f3ef4074c70 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.875+0000 7f3efba00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4114f80 con 0x7f3ef410f440 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 0x7f3ef419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:47.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef4075590 0x7f3ef419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 0x7f3ef419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60988/0 (socket says 172.21.15.86:60988) 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 -- 172.21.15.86:0/375315886 learned_addr learned my addr 172.21.15.86:0/375315886 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9a00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ef410f440 0x7f3ef419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 -- 172.21.15.86:0/375315886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ef410f440 msgr2=0x7f3ef419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ef410f440 0x7f3ef419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 -- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef4075590 msgr2=0x7f3ef419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef4075590 0x7f3ef419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef9000700 1 -- 172.21.15.86:0/375315886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41a02f0 con 0x7f3ef4074c70 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.876+0000 7f3ef3e00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef4075590 0x7f3ef419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.877+0000 7f3ef9000700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 0x7f3ef419b330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800e9a0 tx=0x7f3ee800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.877+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800cbd0 con 0x7f3ef4074c70 2024-04-25T19:00:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.877+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee800cd70 con 0x7f3ef4074c70 2024-04-25T19:00:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.877+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80186f0 con 0x7f3ef4074c70 2024-04-25T19:00:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.877+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41a05e0 con 0x7f3ef4074c70 2024-04-25T19:00:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.877+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41a7e60 con 0x7f3ef4074c70 2024-04-25T19:00:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.879+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef4069730 con 0x7f3ef4074c70 2024-04-25T19:00:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.881+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ee8010a00 con 0x7f3ef4074c70 2024-04-25T19:00:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.882+0000 7f3ef1600700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3edc0a3cc0 0x7f3edc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.882+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f3ee80d06a0 con 0x7f3ef4074c70 2024-04-25T19:00:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.883+0000 7f3ef3e00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3edc0a3cc0 0x7f3edc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.883+0000 7f3ef3e00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3edc0a3cc0 0x7f3edc0a6180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000b2e0 tx=0x7f3ee001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:47.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:47.887+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ee809d020 con 0x7f3ef4074c70 2024-04-25T19:00:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.116+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3ef402df80 con 0x7f3ef4074c70 2024-04-25T19:00:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.116+0000 7f3ef1600700 1 -- 172.21.15.86:0/375315886 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ee809c920 con 0x7f3ef4074c70 2024-04-25T19:00:48.118 INFO:teuthology.orchestra.run.smithi086.stdout:77309411420 2024-04-25T19:00:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.119+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3edc0a3cc0 msgr2=0x7f3edc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.119+0000 7f3efba00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3edc0a3cc0 0x7f3edc0a6180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000b2e0 tx=0x7f3ee001a040 comp rx=0 tx=0).stop 2024-04-25T19:00:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.119+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 msgr2=0x7f3ef419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.119+0000 7f3efba00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 0x7f3ef419b330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800e9a0 tx=0x7f3ee800ed60 comp rx=0 tx=0).stop 2024-04-25T19:00:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 shutdown_connections 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f3edc0a3cc0 0x7f3edc0a6180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ef410f440 0x7f3ef419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4074c70 0x7f3ef419b330 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 --2- 172.21.15.86:0/375315886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef4075590 0x7f3ef419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 >> 172.21.15.86:0/375315886 conn(0x7f3ef4079ba0 msgr2=0x7f3ef40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 shutdown_connections 2024-04-25T19:00:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:48.120+0000 7f3efba00700 1 -- 172.21.15.86:0/375315886 wait complete. 2024-04-25T19:00:48.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724613 got 167503724618 for osd.5 2024-04-25T19:00:48.629 DEBUG:teuthology.parallel:result is None 2024-04-25T19:00:48.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:48 smithi086 ceph-mon[27783]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 935 B/s wr, 11 op/s 2024-04-25T19:00:48.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:48 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/375315886' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T19:00:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:48 smithi194 ceph-mon[32740]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 935 B/s wr, 11 op/s 2024-04-25T19:00:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:48 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/375315886' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T19:00:48.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:48 smithi183 ceph-mon[32344]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 935 B/s wr, 11 op/s 2024-04-25T19:00:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:48 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/375315886' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 -- 172.21.15.86:0/4015093023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1401092d0 msgr2=0x7fc1401096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 --2- 172.21.15.86:0/4015093023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1401092d0 0x7fc1401096b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc134009920 tx=0x7fc134009c30 comp rx=0 tx=0).stop 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 -- 172.21.15.86:0/4015093023 shutdown_connections 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 --2- 172.21.15.86:0/4015093023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc14010e3b0 0x7fc140115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 --2- 172.21.15.86:0/4015093023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1401092d0 0x7fc1401096b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 --2- 172.21.15.86:0/4015093023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc140109c80 0x7fc14010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 -- 172.21.15.86:0/4015093023 >> 172.21.15.86:0/4015093023 conn(0x7fc140076210 msgr2=0x7fc140107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 -- 172.21.15.86:0/4015093023 shutdown_connections 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.370+0000 7fc148800700 1 -- 172.21.15.86:0/4015093023 wait complete. 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.371+0000 7fc148800700 1 Processor -- start 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.371+0000 7fc148800700 1 -- start start 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc148800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 0x7fc1401a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc148800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc140109c80 0x7fc1401a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc148800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14010e3b0 0x7fc14019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc148800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401192b0 con 0x7fc1401092d0 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc148800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc140119130 con 0x7fc140109c80 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc148800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc140119430 con 0x7fc14010e3b0 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc145e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 0x7fc1401a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc145e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 0x7fc1401a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35448/0 (socket says 172.21.15.86:35448) 2024-04-25T19:00:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc145e00700 1 -- 172.21.15.86:0/2571095854 learned_addr learned my addr 172.21.15.86:0/2571095854 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc146800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14010e3b0 0x7fc14019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc145400700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc140109c80 0x7fc1401a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc145e00700 1 -- 172.21.15.86:0/2571095854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14010e3b0 msgr2=0x7fc14019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.372+0000 7fc145e00700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14010e3b0 0x7fc14019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc145e00700 1 -- 172.21.15.86:0/2571095854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc140109c80 msgr2=0x7fc1401a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc145e00700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc140109c80 0x7fc1401a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc145e00700 1 -- 172.21.15.86:0/2571095854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14019fd70 con 0x7fc1401092d0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc145e00700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 0x7fc1401a5450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc13400f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134027070 con 0x7fc1401092d0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1401a0000 con 0x7fc1401092d0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1340055f0 con 0x7fc1401092d0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134004930 con 0x7fc1401092d0 2024-04-25T19:00:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.373+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401ac320 con 0x7fc1401092d0 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.374+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc14010a7b0 con 0x7fc1401092d0 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.376+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc13402a070 con 0x7fc1401092d0 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.378+0000 7fc13aa00700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc1280a3c40 0x7fc1280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.378+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc1340d0db0 con 0x7fc1401092d0 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.378+0000 7fc145400700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc1280a3c40 0x7fc1280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.379+0000 7fc145400700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc1280a3c40 0x7fc1280a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1400770f0 tx=0x7fc130009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.384+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc13405eff0 con 0x7fc1401092d0 2024-04-25T19:00:49.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.616+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc140111620 con 0x7fc1401092d0 2024-04-25T19:00:49.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.617+0000 7fc13aa00700 1 -- 172.21.15.86:0/2571095854 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc1340d5070 con 0x7fc1401092d0 2024-04-25T19:00:49.618 INFO:teuthology.orchestra.run.smithi086.stdout:292057776180 2024-04-25T19:00:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc1280a3c40 msgr2=0x7fc1280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc1280a3c40 0x7fc1280a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1400770f0 tx=0x7fc130009310 comp rx=0 tx=0).stop 2024-04-25T19:00:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 msgr2=0x7fc1401a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 0x7fc1401a5450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc13400f990 comp rx=0 tx=0).stop 2024-04-25T19:00:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 shutdown_connections 2024-04-25T19:00:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc1280a3c40 0x7fc1280a6100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc14010e3b0 0x7fc14019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1401092d0 0x7fc1401a5450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 --2- 172.21.15.86:0/2571095854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc140109c80 0x7fc1401a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.619+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 >> 172.21.15.86:0/2571095854 conn(0x7fc140076210 msgr2=0x7fc1401086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.620+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 shutdown_connections 2024-04-25T19:00:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:49.620+0000 7fc148800700 1 -- 172.21.15.86:0/2571095854 wait complete. 2024-04-25T19:00:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:50.675 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:50 smithi194 ceph-mon[32740]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 755 B/s wr, 10 op/s 2024-04-25T19:00:50.675 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:50 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:00:50.675 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:50 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2571095854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T19:00:50.717 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:50 smithi183 ceph-mon[32344]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 755 B/s wr, 10 op/s 2024-04-25T19:00:50.718 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:50 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:00:50.718 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:50 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2571095854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T19:00:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:50 smithi086 ceph-mon[27783]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 755 B/s wr, 10 op/s 2024-04-25T19:00:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:50 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:00:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:50 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2571095854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T19:00:51.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776173 got 292057776180 for osd.9 2024-04-25T19:00:51.079 DEBUG:teuthology.parallel:result is None 2024-04-25T19:00:51.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:51.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:51.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:51.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:51.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:51.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T19:00:51.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:51 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T19:00:52.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:51 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T19:00:52.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:51 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:00:52.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738471 for osd.0 2024-04-25T19:00:52.342 DEBUG:teuthology.parallel:result is None 2024-04-25T19:00:52.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 -- 172.21.15.86:0/3759211532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 msgr2=0x7ff234102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:52.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 --2- 172.21.15.86:0/3759211532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff234102f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff220009990 tx=0x7ff220009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 -- 172.21.15.86:0/3759211532 shutdown_connections 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 --2- 172.21.15.86:0/3759211532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff234103450 0x7ff23410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 --2- 172.21.15.86:0/3759211532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234108a50 0x7ff234108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 --2- 172.21.15.86:0/3759211532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff234102f10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.546+0000 7ff23b800700 1 -- 172.21.15.86:0/3759211532 >> 172.21.15.86:0/3759211532 conn(0x7ff234079ba0 msgr2=0x7ff234079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.547+0000 7ff23b800700 1 -- 172.21.15.86:0/3759211532 shutdown_connections 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.547+0000 7ff23b800700 1 -- 172.21.15.86:0/3759211532 wait complete. 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 Processor -- start 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 -- start start 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff23419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234103450 0x7ff23419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff234108a50 0x7ff23419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:52.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff234112c50 con 0x7ff234108a50 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff234112ad0 con 0x7ff234103450 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff23b800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff234112dd0 con 0x7ff234102a90 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff238e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff23419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff238e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff23419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39714/0 (socket says 172.21.15.86:39714) 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.548+0000 7ff238e00700 1 -- 172.21.15.86:0/3273808582 learned_addr learned my addr 172.21.15.86:0/3273808582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff233e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234103450 0x7ff23419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff238e00700 1 -- 172.21.15.86:0/3273808582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234103450 msgr2=0x7ff23419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff238e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234103450 0x7ff23419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff238e00700 1 -- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff234108a50 msgr2=0x7ff23419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff238e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff234108a50 0x7ff23419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff238e00700 1 -- 172.21.15.86:0/3273808582 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2341a03f0 con 0x7ff234102a90 2024-04-25T19:00:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff233e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234103450 0x7ff23419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff238e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff23419b3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e9a0 tx=0x7ff22800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22800cb50 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2341a06e0 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22800ccf0 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.549+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228018680 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.550+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2341a7f60 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.551+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff234069730 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.553+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff228010430 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.554+0000 7ff231600700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff21c0a3c40 0x7ff21c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.554+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7ff2280d0680 con 0x7ff234102a90 2024-04-25T19:00:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.555+0000 7ff233e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff21c0a3c40 0x7ff21c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stdout:111669149782 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.556+0000 7ff233e00700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff21c0a3c40 0x7ff21c0a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff23407a4a0 tx=0x7ff22001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.558+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff22809cfa0 con 0x7ff234102a90 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.791+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff23402df80 con 0x7ff234102a90 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.792+0000 7ff231600700 1 -- 172.21.15.86:0/3273808582 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff22809c8a0 con 0x7ff234102a90 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff21c0a3c40 msgr2=0x7ff21c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:52.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff21c0a3c40 0x7ff21c0a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff23407a4a0 tx=0x7ff22001a040 comp rx=0 tx=0).stop 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 msgr2=0x7ff23419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff23419b3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e9a0 tx=0x7ff22800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 shutdown_connections 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7ff21c0a3c40 0x7ff21c0a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff234108a50 0x7ff23419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff234102a90 0x7ff23419b3e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 --2- 172.21.15.86:0/3273808582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff234103450 0x7ff23419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 >> 172.21.15.86:0/3273808582 conn(0x7ff234079ba0 msgr2=0x7ff23410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 shutdown_connections 2024-04-25T19:00:52.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:51.795+0000 7ff23b800700 1 -- 172.21.15.86:0/3273808582 wait complete. 2024-04-25T19:00:53.039 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:52 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3273808582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T19:00:53.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:52 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3273808582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T19:00:53.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:52 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3273808582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T19:00:53.896 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:53 smithi086 ceph-mon[27783]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T19:00:54.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:53 smithi183 ceph-mon[32344]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T19:00:54.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:53 smithi194 ceph-mon[32740]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T19:00:54.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149777 got 111669149782 for osd.3 2024-04-25T19:00:54.977 DEBUG:teuthology.parallel:result is None 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 -- 172.21.15.86:0/1052598803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 msgr2=0x7f169c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 --2- 172.21.15.86:0/1052598803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 0x7f169c10f8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f169400b0a0 tx=0x7f169400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 -- 172.21.15.86:0/1052598803 shutdown_connections 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 --2- 172.21.15.86:0/1052598803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 0x7f169c10f8b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 --2- 172.21.15.86:0/1052598803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f169c108a50 0x7f169c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 --2- 172.21.15.86:0/1052598803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 -- 172.21.15.86:0/1052598803 >> 172.21.15.86:0/1052598803 conn(0x7f169c079ba0 msgr2=0x7f169c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 -- 172.21.15.86:0/1052598803 shutdown_connections 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.977+0000 7f16a2000700 1 -- 172.21.15.86:0/1052598803 wait complete. 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.978+0000 7f16a2000700 1 Processor -- start 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.978+0000 7f16a2000700 1 -- start start 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f16a2000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f16a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 0x7f169c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f16a2000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f169c108a50 0x7f169c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f16a2000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f169c112c50 con 0x7f169c103450 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f16a2000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f169c112ad0 con 0x7f169c102a90 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f16a2000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f169c112dd0 con 0x7f169c108a50 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f169aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 0x7f169c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f169b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f169b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49986/0 (socket says 172.21.15.86:49986) 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f169b400700 1 -- 172.21.15.86:0/3821156017 learned_addr learned my addr 172.21.15.86:0/3821156017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.979+0000 7f169be00700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f169c108a50 0x7f169c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f169b400700 1 -- 172.21.15.86:0/3821156017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f169c108a50 msgr2=0x7f169c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f169b400700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f169c108a50 0x7f169c19da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f169b400700 1 -- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 msgr2=0x7f169c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f169b400700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 0x7f169c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f169b400700 1 -- 172.21.15.86:0/3821156017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c19e1d0 con 0x7f169c102a90 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f169b400700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c1991c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e9a0 tx=0x7f168c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ca10 con 0x7f169c102a90 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c19e4c0 con 0x7f169c102a90 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c00cbb0 con 0x7f169c102a90 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c074c70 con 0x7f169c102a90 2024-04-25T19:00:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.980+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c018420 con 0x7f169c102a90 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.981+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c069730 con 0x7f169c102a90 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.984+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f168c010430 con 0x7f169c102a90 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.985+0000 7f1693e00700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16800a3cc0 0x7f16800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.985+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f168c0d07a0 con 0x7f169c102a90 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.985+0000 7f169aa00700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16800a3cc0 0x7f16800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.986+0000 7f169aa00700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16800a3cc0 0x7f16800a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f169c07a2c0 tx=0x7f1688009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:54.987+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f168c09d0c0 con 0x7f169c102a90 2024-04-25T19:00:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.240+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f169c02df80 con 0x7f169c102a90 2024-04-25T19:00:55.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.240+0000 7f1693e00700 1 -- 172.21.15.86:0/3821156017 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f168c00cd60 con 0x7f169c102a90 2024-04-25T19:00:55.242 INFO:teuthology.orchestra.run.smithi086.stdout:146028888145 2024-04-25T19:00:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.243+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16800a3cc0 msgr2=0x7f16800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.243+0000 7f16a2000700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16800a3cc0 0x7f16800a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f169c07a2c0 tx=0x7f1688009260 comp rx=0 tx=0).stop 2024-04-25T19:00:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.243+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 msgr2=0x7f169c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.243+0000 7f16a2000700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c1991c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e9a0 tx=0x7f168c00ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 shutdown_connections 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f16800a3cc0 0x7f16800a6180 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f169c108a50 0x7f169c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f169c102a90 0x7f169c1991c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 --2- 172.21.15.86:0/3821156017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f169c103450 0x7f169c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 >> 172.21.15.86:0/3821156017 conn(0x7f169c079ba0 msgr2=0x7f169c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 shutdown_connections 2024-04-25T19:00:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.244+0000 7f16a2000700 1 -- 172.21.15.86:0/3821156017 wait complete. 2024-04-25T19:00:55.394 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:55 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3821156017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T19:00:55.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:55 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3821156017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T19:00:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:55 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3821156017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T19:00:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.846+0000 7f5f55a00700 1 -- 172.21.15.86:0/2209512943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f5010f1b0 msgr2=0x7f5f50111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.846+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2209512943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f5010f1b0 0x7f5f50111620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009f90 tx=0x7f5f4800b330 comp rx=0 tx=0).stop 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 -- 172.21.15.86:0/2209512943 shutdown_connections 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2209512943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f5010f1b0 0x7f5f50111620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2209512943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f50074e70 0x7f5f50075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2209512943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50075790 0x7f5f5010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 -- 172.21.15.86:0/2209512943 >> 172.21.15.86:0/2209512943 conn(0x7f5f500fe4c0 msgr2=0x7f5f501008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 -- 172.21.15.86:0/2209512943 shutdown_connections 2024-04-25T19:00:55.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.847+0000 7f5f55a00700 1 -- 172.21.15.86:0/2209512943 wait complete. 2024-04-25T19:00:55.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 Processor -- start 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 -- start start 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 0x7f5f5019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f50075790 0x7f5f5019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5010f1b0 0x7f5f5019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50114d40 con 0x7f5f50074e70 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50114bc0 con 0x7f5f5010f1b0 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.848+0000 7f5f55a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50114ec0 con 0x7f5f50075790 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 0x7f5f5019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 0x7f5f5019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54248/0 (socket says 172.21.15.86:54248) 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 -- 172.21.15.86:0/2833015551 learned_addr learned my addr 172.21.15.86:0/2833015551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f47e00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f50075790 0x7f5f5019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f55000700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5010f1b0 0x7f5f5019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 -- 172.21.15.86:0/2833015551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f50075790 msgr2=0x7f5f5019b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f50075790 0x7f5f5019b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 -- 172.21.15.86:0/2833015551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5010f1b0 msgr2=0x7f5f5019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:55.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5010f1b0 0x7f5f5019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:55.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.849+0000 7f5f4fe00700 1 -- 172.21.15.86:0/2833015551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f501a01e0 con 0x7f5f50074e70 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.850+0000 7f5f4fe00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 0x7f5f5019b1d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e9a0 tx=0x7f5f4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.850+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000c9c0 con 0x7f5f50074e70 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.850+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f4000cb60 con 0x7f5f50074e70 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.850+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f501a04d0 con 0x7f5f50074e70 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.850+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40018680 con 0x7f5f50074e70 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.850+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501a7d50 con 0x7f5f50074e70 2024-04-25T19:00:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.851+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f50069730 con 0x7f5f50074e70 2024-04-25T19:00:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.853+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5f40010430 con 0x7f5f50074e70 2024-04-25T19:00:55.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.854+0000 7f5f4d600700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f340a3c40 0x7f5f340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:55.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.855+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f5f400d0770 con 0x7f5f50074e70 2024-04-25T19:00:55.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.855+0000 7f5f47e00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f340a3c40 0x7f5f340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.856+0000 7f5f47e00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f340a3c40 0x7f5f340a6100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5f50110460 tx=0x7f5f38009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:55.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:55.859+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f4009d090 con 0x7f5f50074e70 2024-04-25T19:00:56.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.095+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5f5002df80 con 0x7f5f50074e70 2024-04-25T19:00:56.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.096+0000 7f5f4d600700 1 -- 172.21.15.86:0/2833015551 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5f4009c990 con 0x7f5f50074e70 2024-04-25T19:00:56.097 INFO:teuthology.orchestra.run.smithi086.stdout:313532612656 2024-04-25T19:00:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.099+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f340a3c40 msgr2=0x7f5f340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f340a3c40 0x7f5f340a6100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5f50110460 tx=0x7f5f38009310 comp rx=0 tx=0).stop 2024-04-25T19:00:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 msgr2=0x7f5f5019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:56.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 0x7f5f5019b1d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e9a0 tx=0x7f5f4000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 shutdown_connections 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f5f340a3c40 0x7f5f340a6100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5010f1b0 0x7f5f5019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f50074e70 0x7f5f5019b1d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 --2- 172.21.15.86:0/2833015551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f50075790 0x7f5f5019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 >> 172.21.15.86:0/2833015551 conn(0x7f5f500fe4c0 msgr2=0x7f5f500ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:56.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 shutdown_connections 2024-04-25T19:00:56.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:56.100+0000 7f5f55a00700 1 -- 172.21.15.86:0/2833015551 wait complete. 2024-04-25T19:00:56.568 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:56 smithi086 ceph-mon[27783]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:56.568 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:56 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2833015551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T19:00:56.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:56 smithi183 ceph-mon[32344]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:56.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:56 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2833015551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T19:00:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:56 smithi194 ceph-mon[32740]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:00:56.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:56 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2833015551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T19:00:57.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612650 got 313532612656 for osd.10 2024-04-25T19:00:57.725 DEBUG:teuthology.parallel:result is None 2024-04-25T19:00:57.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 -- 172.21.15.86:0/325555013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 msgr2=0x7f0e34111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:57.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 --2- 172.21.15.86:0/325555013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e34111830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00b0a0 tx=0x7f0e2c00b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:57.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 -- 172.21.15.86:0/325555013 shutdown_connections 2024-04-25T19:00:57.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 --2- 172.21.15.86:0/325555013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e34111830 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:57.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 --2- 172.21.15.86:0/325555013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e34075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:57.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 --2- 172.21.15.86:0/325555013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e34075590 0x7f0e3410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:57.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.947+0000 7f0e39a00700 1 -- 172.21.15.86:0/325555013 >> 172.21.15.86:0/325555013 conn(0x7f0e34079ba0 msgr2=0x7f0e34079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:57.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.948+0000 7f0e39a00700 1 -- 172.21.15.86:0/325555013 shutdown_connections 2024-04-25T19:00:57.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.948+0000 7f0e39a00700 1 -- 172.21.15.86:0/325555013 wait complete. 2024-04-25T19:00:57.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 Processor -- start 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 -- start start 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e3419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e34075590 0x7f0e3419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e3419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34114de0 con 0x7f0e3410f440 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34114c60 con 0x7f0e34074c70 2024-04-25T19:00:57.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.949+0000 7f0e39a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34114f60 con 0x7f0e34075590 2024-04-25T19:00:57.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e33400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e3419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:57.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e3419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:57.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e34075590 0x7f0e3419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:57.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e33400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e3419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54264/0 (socket says 172.21.15.86:54264) 2024-04-25T19:00:57.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e33400700 1 -- 172.21.15.86:0/1529108485 learned_addr learned my addr 172.21.15.86:0/1529108485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e3419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50028/0 (socket says 172.21.15.86:50028) 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 -- 172.21.15.86:0/1529108485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e34075590 msgr2=0x7f0e3419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e34075590 0x7f0e3419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 -- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 msgr2=0x7f0e3419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e3419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 -- 172.21.15.86:0/1529108485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e341a02b0 con 0x7f0e34074c70 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e33400700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e3419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.950+0000 7f0e32a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e3419b2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e9f0 tx=0x7f0e2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.951+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400c9c0 con 0x7f0e34074c70 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.951+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e341a05a0 con 0x7f0e34074c70 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.951+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e2400cb60 con 0x7f0e34074c70 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.951+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24018680 con 0x7f0e34074c70 2024-04-25T19:00:57.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.951+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341a7e20 con 0x7f0e34074c70 2024-04-25T19:00:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.952+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e34069730 con 0x7f0e34074c70 2024-04-25T19:00:57.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.954+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0e24010430 con 0x7f0e34074c70 2024-04-25T19:00:57.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.955+0000 7f0e2b400700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0e180a3c40 0x7f0e180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:57.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.956+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f0e240d0610 con 0x7f0e34074c70 2024-04-25T19:00:57.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.956+0000 7f0e32000700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0e180a3c40 0x7f0e180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:57.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.956+0000 7f0e32000700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0e180a3c40 0x7f0e180a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e341025e0 tx=0x7f0e20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:57.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:57.960+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e2409cf30 con 0x7f0e34074c70 2024-04-25T19:00:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.192+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0e3402df80 con 0x7f0e34074c70 2024-04-25T19:00:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.192+0000 7f0e2b400700 1 -- 172.21.15.86:0/1529108485 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0e2409c830 con 0x7f0e34074c70 2024-04-25T19:00:58.194 INFO:teuthology.orchestra.run.smithi086.stdout:347892351020 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0e180a3c40 msgr2=0x7f0e180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0e180a3c40 0x7f0e180a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e341025e0 tx=0x7f0e20009260 comp rx=0 tx=0).stop 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 msgr2=0x7f0e3419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e3419b2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e9f0 tx=0x7f0e2400ed00 comp rx=0 tx=0).stop 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 shutdown_connections 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0e180a3c40 0x7f0e180a6100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e3410f440 0x7f0e3419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e34074c70 0x7f0e3419b2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 --2- 172.21.15.86:0/1529108485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e34075590 0x7f0e3419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.195+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 >> 172.21.15.86:0/1529108485 conn(0x7f0e34079ba0 msgr2=0x7f0e340ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.196+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 shutdown_connections 2024-04-25T19:00:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:00:58.196+0000 7f0e39a00700 1 -- 172.21.15.86:0/1529108485 wait complete. 2024-04-25T19:00:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:58 smithi086 ceph-mon[27783]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:58.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:00:58 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1529108485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T19:00:58.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:58 smithi194 ceph-mon[32740]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:58.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:00:58 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1529108485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T19:00:58.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:58 smithi183 ceph-mon[32344]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:00:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:00:58 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1529108485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T19:00:59.161 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888141 got 146028888145 for osd.4 2024-04-25T19:00:59.161 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:00.256 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411420 for osd.2 2024-04-25T19:01:00.257 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:00.311 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:00.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:00 smithi086 ceph-mon[27783]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:00.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351013 got 347892351020 for osd.11 2024-04-25T19:01:00.676 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:00 smithi194 ceph-mon[32740]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:00 smithi183 ceph-mon[32344]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 -- 172.21.15.86:0/960914374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204108a80 msgr2=0x7fa204108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 --2- 172.21.15.86:0/960914374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204108a80 0x7fa204108e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009f90 tx=0x7fa1f400b330 comp rx=0 tx=0).stop 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 -- 172.21.15.86:0/960914374 shutdown_connections 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 --2- 172.21.15.86:0/960914374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204103450 0x7fa20410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 --2- 172.21.15.86:0/960914374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204108a80 0x7fa204108e60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 --2- 172.21.15.86:0/960914374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa204102a90 0x7fa204102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 -- 172.21.15.86:0/960914374 >> 172.21.15.86:0/960914374 conn(0x7fa204079bf0 msgr2=0x7fa20407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 -- 172.21.15.86:0/960914374 shutdown_connections 2024-04-25T19:01:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.151+0000 7fa20a000700 1 -- 172.21.15.86:0/960914374 wait complete. 2024-04-25T19:01:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.152+0000 7fa20a000700 1 Processor -- start 2024-04-25T19:01:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.152+0000 7fa20a000700 1 -- start start 2024-04-25T19:01:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa20a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 0x7fa204197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa20a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204103450 0x7fa204197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa20a000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa204108a80 0x7fa20419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa20a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa204110b00 con 0x7fa204103450 2024-04-25T19:01:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa20a000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa204110980 con 0x7fa204102a90 2024-04-25T19:01:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa20a000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa204110c80 con 0x7fa204108a80 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa203400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 0x7fa204197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa1faa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204103450 0x7fa204197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa203400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 0x7fa204197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:34032/0 (socket says 172.21.15.86:34032) 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa203400700 1 -- 172.21.15.86:0/2920769948 learned_addr learned my addr 172.21.15.86:0/2920769948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.153+0000 7fa203e00700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa204108a80 0x7fa20419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa203400700 1 -- 172.21.15.86:0/2920769948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa204108a80 msgr2=0x7fa20419b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa203400700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa204108a80 0x7fa20419b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa203400700 1 -- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204103450 msgr2=0x7fa204197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa203400700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204103450 0x7fa204197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa203400700 1 -- 172.21.15.86:0/2920769948 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20419bfd0 con 0x7fa204102a90 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa203400700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 0x7fa204197010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009f60 tx=0x7fa1f400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4021070 con 0x7fa204102a90 2024-04-25T19:01:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1f4007ac0 con 0x7fa204102a90 2024-04-25T19:01:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40052a0 con 0x7fa204102a90 2024-04-25T19:01:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.154+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20419c260 con 0x7fa204102a90 2024-04-25T19:01:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.155+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041a3b40 con 0x7fa204102a90 2024-04-25T19:01:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.157+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa204069730 con 0x7fa204102a90 2024-04-25T19:01:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.159+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1f4024070 con 0x7fa204102a90 2024-04-25T19:01:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.160+0000 7fa200c00700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa1e80a81a0 0x7fa1e80aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.161+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa1f400f070 con 0x7fa204102a90 2024-04-25T19:01:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.161+0000 7fa1faa00700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa1e80a81a0 0x7fa1e80aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.161+0000 7fa1faa00700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa1e80a81a0 0x7fa1e80aa660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa20407a0f0 tx=0x7fa1ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.162+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1f409dd20 con 0x7fa204102a90 2024-04-25T19:01:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.402+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa20402df80 con 0x7fa204102a90 2024-04-25T19:01:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.402+0000 7fa200c00700 1 -- 172.21.15.86:0/2920769948 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1f409d620 con 0x7fa204102a90 2024-04-25T19:01:01.404 INFO:teuthology.orchestra.run.smithi086.stdout:55834574947 2024-04-25T19:01:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa1e80a81a0 msgr2=0x7fa1e80aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa1e80a81a0 0x7fa1e80aa660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa20407a0f0 tx=0x7fa1ec009260 comp rx=0 tx=0).stop 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 msgr2=0x7fa204197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 0x7fa204197010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009f60 tx=0x7fa1f400ccb0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 shutdown_connections 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa1e80a81a0 0x7fa1e80aa660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa204108a80 0x7fa20419b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa204102a90 0x7fa204197010 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 --2- 172.21.15.86:0/2920769948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa204103450 0x7fa204197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.405+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 >> 172.21.15.86:0/2920769948 conn(0x7fa204079bf0 msgr2=0x7fa204101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.406+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 shutdown_connections 2024-04-25T19:01:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.406+0000 7fa20a000700 1 -- 172.21.15.86:0/2920769948 wait complete. 2024-04-25T19:01:01.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:01 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2920769948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T19:01:01.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:01 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2920769948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T19:01:01.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:01 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2920769948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T19:01:01.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 -- 172.21.15.86:0/1509457896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade4103b30 msgr2=0x7fade4103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 --2- 172.21.15.86:0/1509457896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade4103b30 0x7fade4103f10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009990 tx=0x7fadd4009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 -- 172.21.15.86:0/1509457896 shutdown_connections 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 --2- 172.21.15.86:0/1509457896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade410cf60 0x7fade410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 --2- 172.21.15.86:0/1509457896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade4103b30 0x7fade4103f10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 --2- 172.21.15.86:0/1509457896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 -- 172.21.15.86:0/1509457896 >> 172.21.15.86:0/1509457896 conn(0x7fade40fd360 msgr2=0x7fade40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 -- 172.21.15.86:0/1509457896 shutdown_connections 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.693+0000 7fade9000700 1 -- 172.21.15.86:0/1509457896 wait complete. 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.694+0000 7fade9000700 1 Processor -- start 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 -- start start 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade4103b30 0x7fade419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade410cf60 0x7fade419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112b80 con 0x7fade410cf60 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112a00 con 0x7fade4103b30 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.695+0000 7fade9000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112d00 con 0x7fade4104450 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade2a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade4103b30 0x7fade419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade2a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46140/0 (socket says 172.21.15.86:46140) 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade2a00700 1 -- 172.21.15.86:0/1245757403 learned_addr learned my addr 172.21.15.86:0/1245757403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3e00700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade410cf60 0x7fade419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 -- 172.21.15.86:0/1245757403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 msgr2=0x7fade419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 -- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade410cf60 msgr2=0x7fade419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade410cf60 0x7fade419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 -- 172.21.15.86:0/1245757403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade41a0260 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade2a00700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.696+0000 7fade3400700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade4103b30 0x7fade419b250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadd40059d0 tx=0x7fadd40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.697+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4027070 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.697+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4022470 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.697+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41a04f0 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.697+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4017650 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.697+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a7dd0 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.698+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade4069730 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.700+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fadd402a070 con 0x7fade4103b30 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.701+0000 7faddbe00700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fadc80a3c40 0x7fadc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:01.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.702+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fadd40d1060 con 0x7fade4103b30 2024-04-25T19:01:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.702+0000 7fade2a00700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fadc80a3c40 0x7fadc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.702+0000 7fade2a00700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fadc80a3c40 0x7fadc80a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fade410e840 tx=0x7fadd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.706+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadd4013070 con 0x7fade4103b30 2024-04-25T19:01:01.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.933+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fade41a7fa0 con 0x7fade4103b30 2024-04-25T19:01:01.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.934+0000 7faddbe00700 1 -- 172.21.15.86:0/1245757403 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fadd409da10 con 0x7fade4103b30 2024-04-25T19:01:01.936 INFO:teuthology.orchestra.run.smithi086.stdout:201863462984 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fadc80a3c40 msgr2=0x7fadc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fadc80a3c40 0x7fadc80a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fade410e840 tx=0x7fadd0009260 comp rx=0 tx=0).stop 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade4103b30 msgr2=0x7fade419b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade4103b30 0x7fade419b250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadd40059d0 tx=0x7fadd40047b0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 shutdown_connections 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fadc80a3c40 0x7fadc80a6100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade410cf60 0x7fade419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade4103b30 0x7fade419b250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 --2- 172.21.15.86:0/1245757403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fade4104450 0x7fade419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 >> 172.21.15.86:0/1245757403 conn(0x7fade40fd360 msgr2=0x7fade40ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 shutdown_connections 2024-04-25T19:01:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:01.937+0000 7fade9000700 1 -- 172.21.15.86:0/1245757403 wait complete. 2024-04-25T19:01:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:02 smithi194 ceph-mon[32740]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:02.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:02 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1245757403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T19:01:02.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:02 smithi086 ceph-mon[27783]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:02.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:02 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1245757403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T19:01:02.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:02 smithi183 ceph-mon[32344]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:02 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1245757403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T19:01:03.031 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462978 got 201863462984 for osd.6 2024-04-25T19:01:03.031 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:04.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574947 for osd.1 2024-04-25T19:01:04.012 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:04.608 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:04 smithi086 ceph-mon[27783]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:04 smithi194 ceph-mon[32740]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:04.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.886+0000 7fa703a00700 1 -- 172.21.15.86:0/663987484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 msgr2=0x7fa6fc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.886+0000 7fa703a00700 1 --2- 172.21.15.86:0/663987484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc102f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80098f0 tx=0x7fa6e8009c00 comp rx=0 tx=0).stop 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 -- 172.21.15.86:0/663987484 shutdown_connections 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 --2- 172.21.15.86:0/663987484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6fc1034b0 0x7fa6fc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 --2- 172.21.15.86:0/663987484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc107940 0x7fa6fc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 --2- 172.21.15.86:0/663987484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc102f70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 -- 172.21.15.86:0/663987484 >> 172.21.15.86:0/663987484 conn(0x7fa6fc079ba0 msgr2=0x7fa6fc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 -- 172.21.15.86:0/663987484 shutdown_connections 2024-04-25T19:01:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.887+0000 7fa703a00700 1 -- 172.21.15.86:0/663987484 wait complete. 2024-04-25T19:01:04.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.888+0000 7fa703a00700 1 Processor -- start 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.888+0000 7fa703a00700 1 -- start start 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa703a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa703a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6fc1034b0 0x7fa6fc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa703a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc107940 0x7fa6fc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa703a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc112c90 con 0x7fa6fc107940 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa703a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc112b10 con 0x7fa6fc1034b0 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa703a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc112e10 con 0x7fa6fc102af0 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc107940 0x7fa6fc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa6fbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6fc1034b0 0x7fa6fc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46162/0 (socket says 172.21.15.86:46162) 2024-04-25T19:01:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 -- 172.21.15.86:0/483691169 learned_addr learned my addr 172.21.15.86:0/483691169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 -- 172.21.15.86:0/483691169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6fc1034b0 msgr2=0x7fa6fc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6fc1034b0 0x7fa6fc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 -- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc107940 msgr2=0x7fa6fc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc107940 0x7fa6fc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.889+0000 7fa701000700 1 -- 172.21.15.86:0/483691169 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1a7750 con 0x7fa6fc102af0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.890+0000 7fa701000700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc074d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e9a0 tx=0x7fa6f000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.890+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000ca10 con 0x7fa6fc102af0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.890+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc1a7a40 con 0x7fa6fc102af0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.890+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f000cbb0 con 0x7fa6fc102af0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.890+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0018680 con 0x7fa6fc102af0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.890+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1a7f50 con 0x7fa6fc102af0 2024-04-25T19:01:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.891+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc069730 con 0x7fa6fc102af0 2024-04-25T19:01:04.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.893+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa6f0010430 con 0x7fa6fc102af0 2024-04-25T19:01:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.895+0000 7fa6f9600700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6e40a3c40 0x7fa6e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.895+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fa6f00d0670 con 0x7fa6fc102af0 2024-04-25T19:01:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.895+0000 7fa6fbe00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6e40a3c40 0x7fa6e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.896+0000 7fa6fbe00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6e40a3c40 0x7fa6e40a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80098c0 tx=0x7fa6e8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:04.897+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6f009cf90 con 0x7fa6fc102af0 2024-04-25T19:01:04.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:04 smithi183 ceph-mon[32344]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:05.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.123+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa6fc02df80 con 0x7fa6fc102af0 2024-04-25T19:01:05.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.123+0000 7fa6f9600700 1 -- 172.21.15.86:0/483691169 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6f0085e20 con 0x7fa6fc102af0 2024-04-25T19:01:05.125 INFO:teuthology.orchestra.run.smithi086.stdout:257698037822 2024-04-25T19:01:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6e40a3c40 msgr2=0x7fa6e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6e40a3c40 0x7fa6e40a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80098c0 tx=0x7fa6e8019040 comp rx=0 tx=0).stop 2024-04-25T19:01:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 msgr2=0x7fa6fc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc074d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e9a0 tx=0x7fa6f000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:01:05.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 shutdown_connections 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fa6e40a3c40 0x7fa6e40a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc107940 0x7fa6fc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6fc102af0 0x7fa6fc074d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 --2- 172.21.15.86:0/483691169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6fc1034b0 0x7fa6fc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.126+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 >> 172.21.15.86:0/483691169 conn(0x7fa6fc079ba0 msgr2=0x7fa6fc100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.127+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 shutdown_connections 2024-04-25T19:01:05.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:05.127+0000 7fa703a00700 1 -- 172.21.15.86:0/483691169 wait complete. 2024-04-25T19:01:05.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:05 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/483691169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T19:01:05.812 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:05 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/483691169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T19:01:05.875 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037816 got 257698037822 for osd.8 2024-04-25T19:01:05.875 DEBUG:teuthology.parallel:result is None 2024-04-25T19:01:05.875 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T19:01:05.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T19:01:05.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:05 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/483691169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T19:01:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:06 smithi194 ceph-mon[32740]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:01:06.748 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:06 smithi086 ceph-mon[27783]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:01:06.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:06 smithi183 ceph-mon[32344]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T19:01:07.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:07.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:07 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:07.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:07 smithi194 ceph-mon[32740]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:01:08.037 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:08.037 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:07 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:08.037 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:07 smithi086 ceph-mon[27783]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:01:08.092 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:08.092 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:07 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:08.092 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:07 smithi183 ceph-mon[32344]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:01:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 -- 172.21.15.86:0/2662984967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff08075590 msgr2=0x7eff0810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 --2- 172.21.15.86:0/2662984967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff08075590 0x7eff0810ef00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efef0009990 tx=0x7efef0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 -- 172.21.15.86:0/2662984967 shutdown_connections 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 --2- 172.21.15.86:0/2662984967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff0810f440 0x7eff08111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 --2- 172.21.15.86:0/2662984967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff08074c70 0x7eff08075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 --2- 172.21.15.86:0/2662984967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff08075590 0x7eff0810ef00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 -- 172.21.15.86:0/2662984967 >> 172.21.15.86:0/2662984967 conn(0x7eff08079bf0 msgr2=0x7eff0807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 -- 172.21.15.86:0/2662984967 shutdown_connections 2024-04-25T19:01:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.554+0000 7eff0de00700 1 -- 172.21.15.86:0/2662984967 wait complete. 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.555+0000 7eff0de00700 1 Processor -- start 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 -- start start 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 0x7eff0819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff08075590 0x7eff0819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff0810f440 0x7eff0819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff08114ed0 con 0x7eff0810f440 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff08114d50 con 0x7eff08074c70 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff0de00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff08115050 con 0x7eff08075590 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff06a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 0x7eff0819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff07400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff0810f440 0x7eff0819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff06a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 0x7eff0819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:34084/0 (socket says 172.21.15.86:34084) 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.556+0000 7eff06a00700 1 -- 172.21.15.86:0/1934624586 learned_addr learned my addr 172.21.15.86:0/1934624586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7efeffe00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff08075590 0x7eff0819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff06a00700 1 -- 172.21.15.86:0/1934624586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff08075590 msgr2=0x7eff0819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff06a00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff08075590 0x7eff0819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff06a00700 1 -- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff0810f440 msgr2=0x7eff0819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff06a00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff0810f440 0x7eff0819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff06a00700 1 -- 172.21.15.86:0/1934624586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff081a03a0 con 0x7eff08074c70 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff06a00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 0x7eff0819b3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efef800d980 tx=0x7efef800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8016070 con 0x7eff08074c70 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff081a0690 con 0x7eff08074c70 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.557+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef8004830 con 0x7eff08074c70 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.558+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8005540 con 0x7eff08074c70 2024-04-25T19:01:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.558+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff081a7f10 con 0x7eff08074c70 2024-04-25T19:01:08.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.559+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff08069730 con 0x7eff08074c70 2024-04-25T19:01:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.561+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efef8004220 con 0x7eff08074c70 2024-04-25T19:01:08.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.562+0000 7eff0ca00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efeec0a3c40 0x7efeec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.562+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7efef80d0a30 con 0x7eff08074c70 2024-04-25T19:01:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.562+0000 7efeffe00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efeec0a3c40 0x7efeec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.563+0000 7efeffe00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efeec0a3c40 0x7efeec0a6100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7eff08111100 tx=0x7efef0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:08.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.569+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efef801f020 con 0x7eff08074c70 2024-04-25T19:01:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.796+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff08062e80 con 0x7efeec0a3c40 2024-04-25T19:01:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.806+0000 7eff0ca00700 1 -- 172.21.15.86:0/1934624586 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259183 (secure 0 0 0) 0x7efef0067480 con 0x7efeec0a3c40 2024-04-25T19:01:08.812 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.812+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efeec0a3c40 msgr2=0x7efeec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.812+0000 7eff0de00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efeec0a3c40 0x7efeec0a6100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7eff08111100 tx=0x7efef0005fd0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.812+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 msgr2=0x7eff0819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.812+0000 7eff0de00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 0x7eff0819b3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efef800d980 tx=0x7efef800dc90 comp rx=0 tx=0).stop 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 shutdown_connections 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7efeec0a3c40 0x7efeec0a6100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff0810f440 0x7eff0819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff08074c70 0x7eff0819b3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 --2- 172.21.15.86:0/1934624586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff08075590 0x7eff0819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 >> 172.21.15.86:0/1934624586 conn(0x7eff08079bf0 msgr2=0x7eff080ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 shutdown_connections 2024-04-25T19:01:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:08.813+0000 7eff0de00700 1 -- 172.21.15.86:0/1934624586 wait complete. 2024-04-25T19:01:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T19:01:08.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:01:08.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:01:08.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:09.033 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:01:09.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:01:09.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:01:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:01:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:01:09.758 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-04-25T19:01:07.375671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659216,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1316,"num_read_kb":783,"num_write":441,"num_write_kb":3086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361768,"kb_used_data":12528,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580056,"statfs":{"total":1151940427776,"available":1151569977344,"internally_reserved":0,"allocated":12828672,"data_stored":7973448,"data_compressed":245349,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005506"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538422+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T19:00:38.538422+0000","last_peered":"2024-04-25T19:00:38.538422+0000","last_clean":"2024-04-25T19:00:38.538422+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T19:00:38.538422+0000","last_undegraded":"2024-04-25T19:00:38.538422+0000","last_fullsized":"2024-04-25T19:00:38.538422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.547521+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T19:00:38.547521+0000","last_peered":"2024-04-25T19:00:38.547521+0000","last_clean":"2024-04-25T19:00:38.547521+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T19:00:38.547521+0000","last_undegraded":"2024-04-25T19:00:38.547521+0000","last_fullsized":"2024-04-25T19:00:38.547521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.546878+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T19:00:38.546878+0000","last_peered":"2024-04-25T19:00:38.546878+0000","last_clean":"2024-04-25T19:00:38.546878+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T19:00:38.546878+0000","last_undegraded":"2024-04-25T19:00:38.546878+0000","last_fullsized":"2024-04-25T19:00:38.546878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.540069+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T19:00:38.540069+0000","last_peered":"2024-04-25T19:00:38.540069+0000","last_clean":"2024-04-25T19:00:38.540069+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T19:00:38.540069+0000","last_undegraded":"2024-04-25T19:00:38.540069+0000","last_fullsized":"2024-04-25T19:00:38.540069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694169+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T19:00:38.694169+0000","last_peered":"2024-04-25T19:00:38.694169+0000","last_clean":"2024-04-25T19:00:38.694169+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T19:00:38.694169+0000","last_undegraded":"2024-04-25T19:00:38.694169+0000","last_fullsized":"2024-04-25T19:00:38.694169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533020+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T19:00:38.533020+0000","last_peered":"2024-04-25T19:00:38.533020+0000","last_clean":"2024-04-25T19:00:38.533020+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T19:00:38.533020+0000","last_undegraded":"2024-04-25T19:00:38.533020+0000","last_fullsized":"2024-04-25T19:00:38.533020+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706164+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T19:00:38.706164+0000","last_peered":"2024-04-25T19:00:38.706164+0000","last_clean":"2024-04-25T19:00:38.706164+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T19:00:38.706164+0000","last_undegraded":"2024-04-25T19:00:38.706164+0000","last_fullsized":"2024-04-25T19:00:38.706164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.724643+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T19:00:38.724643+0000","last_peered":"2024-04-25T19:00:38.724643+0000","last_clean":"2024-04-25T19:00:38.724643+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T19:00:38.724643+0000","last_undegraded":"2024-04-25T19:00:38.724643+0000","last_fullsized":"2024-04-25T19:00:38.724643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.543063+0000","last_change":"2024-04-25T19:00:38.543063+0000","last_active":"2024-04-25T19:00:38.543063+0000","last_peered":"2024-04-25T19:00:38.543063+0000","last_clean":"2024-04-25T19:00:38.543063+0000","last_became_active":"2024-04-25T19:00:38.541854+0000","last_became_peered":"2024-04-25T19:00:38.541854+0000","last_unstale":"2024-04-25T19:00:38.543063+0000","last_undegraded":"2024-04-25T19:00:38.543063+0000","last_fullsized":"2024-04-25T19:00:38.543063+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:10.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533358+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T19:00:38.533358+0000","last_peered":"2024-04-25T19:00:38.533358+0000","last_clean":"2024-04-25T19:00:38.533358+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T19:00:38.533358+0000","last_undegraded":"2024-04-25T19:00:38.533358+0000","last_fullsized":"2024-04-25T19:00:38.533358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706490+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T19:00:38.706490+0000","last_peered":"2024-04-25T19:00:38.706490+0000","last_clean":"2024-04-25T19:00:38.706490+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T19:00:38.706490+0000","last_undegraded":"2024-04-25T19:00:38.706490+0000","last_fullsized":"2024-04-25T19:00:38.706490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.539891+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T19:00:38.539891+0000","last_peered":"2024-04-25T19:00:38.539891+0000","last_clean":"2024-04-25T19:00:38.539891+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T19:00:38.539891+0000","last_undegraded":"2024-04-25T19:00:38.539891+0000","last_fullsized":"2024-04-25T19:00:38.539891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719196+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T19:00:38.719196+0000","last_peered":"2024-04-25T19:00:38.719196+0000","last_clean":"2024-04-25T19:00:38.719196+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T19:00:38.719196+0000","last_undegraded":"2024-04-25T19:00:38.719196+0000","last_fullsized":"2024-04-25T19:00:38.719196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.546859+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T19:00:38.546859+0000","last_peered":"2024-04-25T19:00:38.546859+0000","last_clean":"2024-04-25T19:00:38.546859+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T19:00:38.546859+0000","last_undegraded":"2024-04-25T19:00:38.546859+0000","last_fullsized":"2024-04-25T19:00:38.546859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.546991+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T19:00:38.546991+0000","last_peered":"2024-04-25T19:00:38.546991+0000","last_clean":"2024-04-25T19:00:38.546991+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T19:00:38.546991+0000","last_undegraded":"2024-04-25T19:00:38.546991+0000","last_fullsized":"2024-04-25T19:00:38.546991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797358+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T19:00:38.797358+0000","last_peered":"2024-04-25T19:00:38.797358+0000","last_clean":"2024-04-25T19:00:38.797358+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T19:00:38.797358+0000","last_undegraded":"2024-04-25T19:00:38.797358+0000","last_fullsized":"2024-04-25T19:00:38.797358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538605+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T19:00:38.538605+0000","last_peered":"2024-04-25T19:00:38.538605+0000","last_clean":"2024-04-25T19:00:38.538605+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T19:00:38.538605+0000","last_undegraded":"2024-04-25T19:00:38.538605+0000","last_fullsized":"2024-04-25T19:00:38.538605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533542+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T19:00:38.533542+0000","last_peered":"2024-04-25T19:00:38.533542+0000","last_clean":"2024-04-25T19:00:38.533542+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T19:00:38.533542+0000","last_undegraded":"2024-04-25T19:00:38.533542+0000","last_fullsized":"2024-04-25T19:00:38.533542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636701+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T19:00:38.636701+0000","last_peered":"2024-04-25T19:00:38.636701+0000","last_clean":"2024-04-25T19:00:38.636701+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T19:00:38.636701+0000","last_undegraded":"2024-04-25T19:00:38.636701+0000","last_fullsized":"2024-04-25T19:00:38.636701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:00.750157+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T19:01:00.750157+0000","last_peered":"2024-04-25T19:01:00.750157+0000","last_clean":"2024-04-25T19:01:00.750157+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T19:01:00.750157+0000","last_undegraded":"2024-04-25T19:01:00.750157+0000","last_fullsized":"2024-04-25T19:01:00.750157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725140+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T19:00:38.725140+0000","last_peered":"2024-04-25T19:00:38.725140+0000","last_clean":"2024-04-25T19:00:38.725140+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T19:00:38.725140+0000","last_undegraded":"2024-04-25T19:00:38.725140+0000","last_fullsized":"2024-04-25T19:00:38.725140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725580+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T19:00:38.725580+0000","last_peered":"2024-04-25T19:00:38.725580+0000","last_clean":"2024-04-25T19:00:38.725580+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T19:00:38.725580+0000","last_undegraded":"2024-04-25T19:00:38.725580+0000","last_fullsized":"2024-04-25T19:00:38.725580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.548438+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T19:00:38.548438+0000","last_peered":"2024-04-25T19:00:38.548438+0000","last_clean":"2024-04-25T19:00:38.548438+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T19:00:38.548438+0000","last_undegraded":"2024-04-25T19:00:38.548438+0000","last_fullsized":"2024-04-25T19:00:38.548438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537985+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T19:00:38.537985+0000","last_peered":"2024-04-25T19:00:38.537985+0000","last_clean":"2024-04-25T19:00:38.537985+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T19:00:38.537985+0000","last_undegraded":"2024-04-25T19:00:38.537985+0000","last_fullsized":"2024-04-25T19:00:38.537985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797539+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T19:00:38.797539+0000","last_peered":"2024-04-25T19:00:38.797539+0000","last_clean":"2024-04-25T19:00:38.797539+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T19:00:38.797539+0000","last_undegraded":"2024-04-25T19:00:38.797539+0000","last_fullsized":"2024-04-25T19:00:38.797539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694131+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T19:00:38.694131+0000","last_peered":"2024-04-25T19:00:38.694131+0000","last_clean":"2024-04-25T19:00:38.694131+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T19:00:38.694131+0000","last_undegraded":"2024-04-25T19:00:38.694131+0000","last_fullsized":"2024-04-25T19:00:38.694131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535652+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T19:00:38.535652+0000","last_peered":"2024-04-25T19:00:38.535652+0000","last_clean":"2024-04-25T19:00:38.535652+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T19:00:38.535652+0000","last_undegraded":"2024-04-25T19:00:38.535652+0000","last_fullsized":"2024-04-25T19:00:38.535652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:04.669653+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T19:01:04.669653+0000","last_peered":"2024-04-25T19:01:04.669653+0000","last_clean":"2024-04-25T19:01:04.669653+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T19:01:04.669653+0000","last_undegraded":"2024-04-25T19:01:04.669653+0000","last_fullsized":"2024-04-25T19:01:04.669653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.693802+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T19:00:38.693802+0000","last_peered":"2024-04-25T19:00:38.693802+0000","last_clean":"2024-04-25T19:00:38.693802+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T19:00:38.693802+0000","last_undegraded":"2024-04-25T19:00:38.693802+0000","last_fullsized":"2024-04-25T19:00:38.693802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.532948+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T19:00:38.532948+0000","last_peered":"2024-04-25T19:00:38.532948+0000","last_clean":"2024-04-25T19:00:38.532948+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T19:00:38.532948+0000","last_undegraded":"2024-04-25T19:00:38.532948+0000","last_fullsized":"2024-04-25T19:00:38.532948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.695351+0000","last_change":"2024-04-25T19:00:38.695351+0000","last_active":"2024-04-25T19:00:38.695351+0000","last_peered":"2024-04-25T19:00:38.695351+0000","last_clean":"2024-04-25T19:00:38.695351+0000","last_became_active":"2024-04-25T19:00:38.694923+0000","last_became_peered":"2024-04-25T19:00:38.694923+0000","last_unstale":"2024-04-25T19:00:38.695351+0000","last_undegraded":"2024-04-25T19:00:38.695351+0000","last_fullsized":"2024-04-25T19:00:38.695351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:52.773252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.718678+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T19:00:38.718678+0000","last_peered":"2024-04-25T19:00:38.718678+0000","last_clean":"2024-04-25T19:00:38.718678+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T19:00:38.718678+0000","last_undegraded":"2024-04-25T19:00:38.718678+0000","last_fullsized":"2024-04-25T19:00:38.718678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725817+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T19:00:38.725817+0000","last_peered":"2024-04-25T19:00:38.725817+0000","last_clean":"2024-04-25T19:00:38.725817+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T19:00:38.725817+0000","last_undegraded":"2024-04-25T19:00:38.725817+0000","last_fullsized":"2024-04-25T19:00:38.725817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.534623+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T19:00:38.534623+0000","last_peered":"2024-04-25T19:00:38.534623+0000","last_clean":"2024-04-25T19:00:38.534623+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T19:00:38.534623+0000","last_undegraded":"2024-04-25T19:00:38.534623+0000","last_fullsized":"2024-04-25T19:00:38.534623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.532953+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T19:00:38.532953+0000","last_peered":"2024-04-25T19:00:38.532953+0000","last_clean":"2024-04-25T19:00:38.532953+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T19:00:38.532953+0000","last_undegraded":"2024-04-25T19:00:38.532953+0000","last_fullsized":"2024-04-25T19:00:38.532953+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":263,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:04.668968+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T19:01:04.668968+0000","last_peered":"2024-04-25T19:01:04.668968+0000","last_clean":"2024-04-25T19:01:04.668968+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T19:01:04.668968+0000","last_undegraded":"2024-04-25T19:01:04.668968+0000","last_fullsized":"2024-04-25T19:01:04.668968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797882+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T19:00:38.797882+0000","last_peered":"2024-04-25T19:00:38.797882+0000","last_clean":"2024-04-25T19:00:38.797882+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T19:00:38.797882+0000","last_undegraded":"2024-04-25T19:00:38.797882+0000","last_fullsized":"2024-04-25T19:00:38.797882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706603+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T19:00:38.706603+0000","last_peered":"2024-04-25T19:00:38.706603+0000","last_clean":"2024-04-25T19:00:38.706603+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T19:00:38.706603+0000","last_undegraded":"2024-04-25T19:00:38.706603+0000","last_fullsized":"2024-04-25T19:00:38.706603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.722645+0000","last_change":"2024-04-25T19:00:38.722645+0000","last_active":"2024-04-25T19:00:38.722645+0000","last_peered":"2024-04-25T19:00:38.722645+0000","last_clean":"2024-04-25T19:00:38.722645+0000","last_became_active":"2024-04-25T19:00:38.722223+0000","last_became_peered":"2024-04-25T19:00:38.722223+0000","last_unstale":"2024-04-25T19:00:38.722645+0000","last_undegraded":"2024-04-25T19:00:38.722645+0000","last_fullsized":"2024-04-25T19:00:38.722645+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:51.266531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535764+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T19:00:38.535764+0000","last_peered":"2024-04-25T19:00:38.535764+0000","last_clean":"2024-04-25T19:00:38.535764+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T19:00:38.535764+0000","last_undegraded":"2024-04-25T19:00:38.535764+0000","last_fullsized":"2024-04-25T19:00:38.535764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797746+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T19:00:38.797746+0000","last_peered":"2024-04-25T19:00:38.797746+0000","last_clean":"2024-04-25T19:00:38.797746+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T19:00:38.797746+0000","last_undegraded":"2024-04-25T19:00:38.797746+0000","last_fullsized":"2024-04-25T19:00:38.797746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":303,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:04.669401+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T19:01:04.669401+0000","last_peered":"2024-04-25T19:01:04.669401+0000","last_clean":"2024-04-25T19:01:04.669401+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T19:01:04.669401+0000","last_undegraded":"2024-04-25T19:01:04.669401+0000","last_fullsized":"2024-04-25T19:01:04.669401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":238,"num_read_kb":178,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694138+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T19:00:38.694138+0000","last_peered":"2024-04-25T19:00:38.694138+0000","last_clean":"2024-04-25T19:00:38.694138+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T19:00:38.694138+0000","last_undegraded":"2024-04-25T19:00:38.694138+0000","last_fullsized":"2024-04-25T19:00:38.694138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636586+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T19:00:38.636586+0000","last_peered":"2024-04-25T19:00:38.636586+0000","last_clean":"2024-04-25T19:00:38.636586+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T19:00:38.636586+0000","last_undegraded":"2024-04-25T19:00:38.636586+0000","last_fullsized":"2024-04-25T19:00:38.636586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.718345+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T19:00:38.718345+0000","last_peered":"2024-04-25T19:00:38.718345+0000","last_clean":"2024-04-25T19:00:38.718345+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T19:00:38.718345+0000","last_undegraded":"2024-04-25T19:00:38.718345+0000","last_fullsized":"2024-04-25T19:00:38.718345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535059+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T19:00:38.535059+0000","last_peered":"2024-04-25T19:00:38.535059+0000","last_clean":"2024-04-25T19:00:38.535059+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T19:00:38.535059+0000","last_undegraded":"2024-04-25T19:00:38.535059+0000","last_fullsized":"2024-04-25T19:00:38.535059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706563+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T19:00:38.706563+0000","last_peered":"2024-04-25T19:00:38.706563+0000","last_clean":"2024-04-25T19:00:38.706563+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T19:00:38.706563+0000","last_undegraded":"2024-04-25T19:00:38.706563+0000","last_fullsized":"2024-04-25T19:00:38.706563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:04.785376+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T19:01:04.785376+0000","last_peered":"2024-04-25T19:01:04.785376+0000","last_clean":"2024-04-25T19:01:04.785376+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T19:01:04.785376+0000","last_undegraded":"2024-04-25T19:01:04.785376+0000","last_fullsized":"2024-04-25T19:01:04.785376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706102+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T19:00:38.706102+0000","last_peered":"2024-04-25T19:00:38.706102+0000","last_clean":"2024-04-25T19:00:38.706102+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T19:00:38.706102+0000","last_undegraded":"2024-04-25T19:00:38.706102+0000","last_fullsized":"2024-04-25T19:00:38.706102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.693734+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T19:00:38.693734+0000","last_peered":"2024-04-25T19:00:38.693734+0000","last_clean":"2024-04-25T19:00:38.693734+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T19:00:38.693734+0000","last_undegraded":"2024-04-25T19:00:38.693734+0000","last_fullsized":"2024-04-25T19:00:38.693734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.534887+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T19:00:38.534887+0000","last_peered":"2024-04-25T19:00:38.534887+0000","last_clean":"2024-04-25T19:00:38.534887+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T19:00:38.534887+0000","last_undegraded":"2024-04-25T19:00:38.534887+0000","last_fullsized":"2024-04-25T19:00:38.534887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694142+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T19:00:38.694142+0000","last_peered":"2024-04-25T19:00:38.694142+0000","last_clean":"2024-04-25T19:00:38.694142+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T19:00:38.694142+0000","last_undegraded":"2024-04-25T19:00:38.694142+0000","last_fullsized":"2024-04-25T19:00:38.694142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538165+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T19:00:38.538165+0000","last_peered":"2024-04-25T19:00:38.538165+0000","last_clean":"2024-04-25T19:00:38.538165+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T19:00:38.538165+0000","last_undegraded":"2024-04-25T19:00:38.538165+0000","last_fullsized":"2024-04-25T19:00:38.538165+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533297+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T19:00:38.533297+0000","last_peered":"2024-04-25T19:00:38.533297+0000","last_clean":"2024-04-25T19:00:38.533297+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T19:00:38.533297+0000","last_undegraded":"2024-04-25T19:00:38.533297+0000","last_fullsized":"2024-04-25T19:00:38.533297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533176+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T19:00:38.533176+0000","last_peered":"2024-04-25T19:00:38.533176+0000","last_clean":"2024-04-25T19:00:38.533176+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T19:00:38.533176+0000","last_undegraded":"2024-04-25T19:00:38.533176+0000","last_fullsized":"2024-04-25T19:00:38.533176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:04.758746+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T19:01:04.758746+0000","last_peered":"2024-04-25T19:01:04.758746+0000","last_clean":"2024-04-25T19:01:04.758746+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T19:01:04.758746+0000","last_undegraded":"2024-04-25T19:01:04.758746+0000","last_fullsized":"2024-04-25T19:01:04.758746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533170+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T19:00:38.533170+0000","last_peered":"2024-04-25T19:00:38.533170+0000","last_clean":"2024-04-25T19:00:38.533170+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T19:00:38.533170+0000","last_undegraded":"2024-04-25T19:00:38.533170+0000","last_fullsized":"2024-04-25T19:00:38.533170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533491+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T19:00:38.533491+0000","last_peered":"2024-04-25T19:00:38.533491+0000","last_clean":"2024-04-25T19:00:38.533491+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T19:00:38.533491+0000","last_undegraded":"2024-04-25T19:00:38.533491+0000","last_fullsized":"2024-04-25T19:00:38.533491+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:03.781750+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T19:01:03.781750+0000","last_peered":"2024-04-25T19:01:03.781750+0000","last_clean":"2024-04-25T19:01:03.781750+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T19:01:03.781750+0000","last_undegraded":"2024-04-25T19:01:03.781750+0000","last_fullsized":"2024-04-25T19:01:03.781750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725397+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T19:00:38.725397+0000","last_peered":"2024-04-25T19:00:38.725397+0000","last_clean":"2024-04-25T19:00:38.725397+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T19:00:38.725397+0000","last_undegraded":"2024-04-25T19:00:38.725397+0000","last_fullsized":"2024-04-25T19:00:38.725397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.539671+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T19:00:38.539671+0000","last_peered":"2024-04-25T19:00:38.539671+0000","last_clean":"2024-04-25T19:00:38.539671+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T19:00:38.539671+0000","last_undegraded":"2024-04-25T19:00:38.539671+0000","last_fullsized":"2024-04-25T19:00:38.539671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706211+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T19:00:38.706211+0000","last_peered":"2024-04-25T19:00:38.706211+0000","last_clean":"2024-04-25T19:00:38.706211+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T19:00:38.706211+0000","last_undegraded":"2024-04-25T19:00:38.706211+0000","last_fullsized":"2024-04-25T19:00:38.706211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.718968+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T19:00:38.718968+0000","last_peered":"2024-04-25T19:00:38.718968+0000","last_clean":"2024-04-25T19:00:38.718968+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T19:00:38.718968+0000","last_undegraded":"2024-04-25T19:00:38.718968+0000","last_fullsized":"2024-04-25T19:00:38.718968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706475+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T19:00:38.706475+0000","last_peered":"2024-04-25T19:00:38.706475+0000","last_clean":"2024-04-25T19:00:38.706475+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T19:00:38.706475+0000","last_undegraded":"2024-04-25T19:00:38.706475+0000","last_fullsized":"2024-04-25T19:00:38.706475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537718+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T19:00:38.537718+0000","last_peered":"2024-04-25T19:00:38.537718+0000","last_clean":"2024-04-25T19:00:38.537718+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T19:00:38.537718+0000","last_undegraded":"2024-04-25T19:00:38.537718+0000","last_fullsized":"2024-04-25T19:00:38.537718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719609+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T19:00:38.719609+0000","last_peered":"2024-04-25T19:00:38.719609+0000","last_clean":"2024-04-25T19:00:38.719609+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T19:00:38.719609+0000","last_undegraded":"2024-04-25T19:00:38.719609+0000","last_fullsized":"2024-04-25T19:00:38.719609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533675+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T19:00:38.533675+0000","last_peered":"2024-04-25T19:00:38.533675+0000","last_clean":"2024-04-25T19:00:38.533675+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T19:00:38.533675+0000","last_undegraded":"2024-04-25T19:00:38.533675+0000","last_fullsized":"2024-04-25T19:00:38.533675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725315+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T19:00:38.725315+0000","last_peered":"2024-04-25T19:00:38.725315+0000","last_clean":"2024-04-25T19:00:38.725315+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T19:00:38.725315+0000","last_undegraded":"2024-04-25T19:00:38.725315+0000","last_fullsized":"2024-04-25T19:00:38.725315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.540179+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T19:00:38.540179+0000","last_peered":"2024-04-25T19:00:38.540179+0000","last_clean":"2024-04-25T19:00:38.540179+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T19:00:38.540179+0000","last_undegraded":"2024-04-25T19:00:38.540179+0000","last_fullsized":"2024-04-25T19:00:38.540179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.549759+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T19:00:38.549759+0000","last_peered":"2024-04-25T19:00:38.549759+0000","last_clean":"2024-04-25T19:00:38.549759+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T19:00:38.549759+0000","last_undegraded":"2024-04-25T19:00:38.549759+0000","last_fullsized":"2024-04-25T19:00:38.549759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797448+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T19:00:38.797448+0000","last_peered":"2024-04-25T19:00:38.797448+0000","last_clean":"2024-04-25T19:00:38.797448+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T19:00:38.797448+0000","last_undegraded":"2024-04-25T19:00:38.797448+0000","last_fullsized":"2024-04-25T19:00:38.797448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.693856+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T19:00:38.693856+0000","last_peered":"2024-04-25T19:00:38.693856+0000","last_clean":"2024-04-25T19:00:38.693856+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T19:00:38.693856+0000","last_undegraded":"2024-04-25T19:00:38.693856+0000","last_fullsized":"2024-04-25T19:00:38.693856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.724830+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T19:00:38.724830+0000","last_peered":"2024-04-25T19:00:38.724830+0000","last_clean":"2024-04-25T19:00:38.724830+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T19:00:38.724830+0000","last_undegraded":"2024-04-25T19:00:38.724830+0000","last_fullsized":"2024-04-25T19:00:38.724830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537789+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T19:00:38.537789+0000","last_peered":"2024-04-25T19:00:38.537789+0000","last_clean":"2024-04-25T19:00:38.537789+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T19:00:38.537789+0000","last_undegraded":"2024-04-25T19:00:38.537789+0000","last_fullsized":"2024-04-25T19:00:38.537789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636604+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T19:00:38.636604+0000","last_peered":"2024-04-25T19:00:38.636604+0000","last_clean":"2024-04-25T19:00:38.636604+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T19:00:38.636604+0000","last_undegraded":"2024-04-25T19:00:38.636604+0000","last_fullsized":"2024-04-25T19:00:38.636604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725310+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T19:00:38.725310+0000","last_peered":"2024-04-25T19:00:38.725310+0000","last_clean":"2024-04-25T19:00:38.725310+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T19:00:38.725310+0000","last_undegraded":"2024-04-25T19:00:38.725310+0000","last_fullsized":"2024-04-25T19:00:38.725310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694228+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T19:00:38.694228+0000","last_peered":"2024-04-25T19:00:38.694228+0000","last_clean":"2024-04-25T19:00:38.694228+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T19:00:38.694228+0000","last_undegraded":"2024-04-25T19:00:38.694228+0000","last_fullsized":"2024-04-25T19:00:38.694228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706360+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T19:00:38.706360+0000","last_peered":"2024-04-25T19:00:38.706360+0000","last_clean":"2024-04-25T19:00:38.706360+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T19:00:38.706360+0000","last_undegraded":"2024-04-25T19:00:38.706360+0000","last_fullsized":"2024-04-25T19:00:38.706360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:02.666725+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T19:01:02.666725+0000","last_peered":"2024-04-25T19:01:02.666725+0000","last_clean":"2024-04-25T19:01:02.666725+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T19:01:02.666725+0000","last_undegraded":"2024-04-25T19:01:02.666725+0000","last_fullsized":"2024-04-25T19:01:02.666725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706731+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T19:00:38.706731+0000","last_peered":"2024-04-25T19:00:38.706731+0000","last_clean":"2024-04-25T19:00:38.706731+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T19:00:38.706731+0000","last_undegraded":"2024-04-25T19:00:38.706731+0000","last_fullsized":"2024-04-25T19:00:38.706731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535583+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T19:00:38.535583+0000","last_peered":"2024-04-25T19:00:38.535583+0000","last_clean":"2024-04-25T19:00:38.535583+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T19:00:38.535583+0000","last_undegraded":"2024-04-25T19:00:38.535583+0000","last_fullsized":"2024-04-25T19:00:38.535583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706601+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T19:00:38.706601+0000","last_peered":"2024-04-25T19:00:38.706601+0000","last_clean":"2024-04-25T19:00:38.706601+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T19:00:38.706601+0000","last_undegraded":"2024-04-25T19:00:38.706601+0000","last_fullsized":"2024-04-25T19:00:38.706601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.720146+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T19:00:38.720146+0000","last_peered":"2024-04-25T19:00:38.720146+0000","last_clean":"2024-04-25T19:00:38.720146+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T19:00:38.720146+0000","last_undegraded":"2024-04-25T19:00:38.720146+0000","last_fullsized":"2024-04-25T19:00:38.720146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706785+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T19:00:38.706785+0000","last_peered":"2024-04-25T19:00:38.706785+0000","last_clean":"2024-04-25T19:00:38.706785+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T19:00:38.706785+0000","last_undegraded":"2024-04-25T19:00:38.706785+0000","last_fullsized":"2024-04-25T19:00:38.706785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537896+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T19:00:38.537896+0000","last_peered":"2024-04-25T19:00:38.537896+0000","last_clean":"2024-04-25T19:00:38.537896+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T19:00:38.537896+0000","last_undegraded":"2024-04-25T19:00:38.537896+0000","last_fullsized":"2024-04-25T19:00:38.537896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636647+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T19:00:38.636647+0000","last_peered":"2024-04-25T19:00:38.636647+0000","last_clean":"2024-04-25T19:00:38.636647+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T19:00:38.636647+0000","last_undegraded":"2024-04-25T19:00:38.636647+0000","last_fullsized":"2024-04-25T19:00:38.636647+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636527+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T19:00:38.636527+0000","last_peered":"2024-04-25T19:00:38.636527+0000","last_clean":"2024-04-25T19:00:38.636527+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T19:00:38.636527+0000","last_undegraded":"2024-04-25T19:00:38.636527+0000","last_fullsized":"2024-04-25T19:00:38.636527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.550116+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T19:00:38.550116+0000","last_peered":"2024-04-25T19:00:38.550116+0000","last_clean":"2024-04-25T19:00:38.550116+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T19:00:38.550116+0000","last_undegraded":"2024-04-25T19:00:38.550116+0000","last_fullsized":"2024-04-25T19:00:38.550116+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719855+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T19:00:38.719855+0000","last_peered":"2024-04-25T19:00:38.719855+0000","last_clean":"2024-04-25T19:00:38.719855+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T19:00:38.719855+0000","last_undegraded":"2024-04-25T19:00:38.719855+0000","last_fullsized":"2024-04-25T19:00:38.719855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538136+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T19:00:38.538136+0000","last_peered":"2024-04-25T19:00:38.538136+0000","last_clean":"2024-04-25T19:00:38.538136+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T19:00:38.538136+0000","last_undegraded":"2024-04-25T19:00:38.538136+0000","last_fullsized":"2024-04-25T19:00:38.538136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636838+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T19:00:38.636838+0000","last_peered":"2024-04-25T19:00:38.636838+0000","last_clean":"2024-04-25T19:00:38.636838+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T19:00:38.636838+0000","last_undegraded":"2024-04-25T19:00:38.636838+0000","last_fullsized":"2024-04-25T19:00:38.636838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.724985+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T19:00:38.724985+0000","last_peered":"2024-04-25T19:00:38.724985+0000","last_clean":"2024-04-25T19:00:38.724985+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T19:00:38.724985+0000","last_undegraded":"2024-04-25T19:00:38.724985+0000","last_fullsized":"2024-04-25T19:00:38.724985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.534625+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T19:00:38.534625+0000","last_peered":"2024-04-25T19:00:38.534625+0000","last_clean":"2024-04-25T19:00:38.534625+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T19:00:38.534625+0000","last_undegraded":"2024-04-25T19:00:38.534625+0000","last_fullsized":"2024-04-25T19:00:38.534625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725193+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T19:00:38.725193+0000","last_peered":"2024-04-25T19:00:38.725193+0000","last_clean":"2024-04-25T19:00:38.725193+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T19:00:38.725193+0000","last_undegraded":"2024-04-25T19:00:38.725193+0000","last_fullsized":"2024-04-25T19:00:38.725193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797580+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T19:00:38.797580+0000","last_peered":"2024-04-25T19:00:38.797580+0000","last_clean":"2024-04-25T19:00:38.797580+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T19:00:38.797580+0000","last_undegraded":"2024-04-25T19:00:38.797580+0000","last_fullsized":"2024-04-25T19:00:38.797580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719192+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T19:00:38.719192+0000","last_peered":"2024-04-25T19:00:38.719192+0000","last_clean":"2024-04-25T19:00:38.719192+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T19:00:38.719192+0000","last_undegraded":"2024-04-25T19:00:38.719192+0000","last_fullsized":"2024-04-25T19:00:38.719192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.536963+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T19:00:38.536963+0000","last_peered":"2024-04-25T19:00:38.536963+0000","last_clean":"2024-04-25T19:00:38.536963+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T19:00:38.536963+0000","last_undegraded":"2024-04-25T19:00:38.536963+0000","last_fullsized":"2024-04-25T19:00:38.536963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279982,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68614,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1180,"num_read_kb":664,"num_write":258,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351022,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":500954,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612659,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1152999,"data_compressed":47157,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776183,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":500772,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037822,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":501038,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1094873,"data_compressed":47157,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462985,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":501021,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":562530,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":501828,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":562750,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":500314,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500882,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724623,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1093487,"data_compressed":47157,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:01:09.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T19:01:09.876 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:09 smithi086 ceph-mon[27783]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:01:09.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:09 smithi086 ceph-mon[27783]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:09 smithi194 ceph-mon[32740]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:01:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:09 smithi194 ceph-mon[32740]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:10.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:09 smithi183 ceph-mon[32344]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:01:10.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:09 smithi183 ceph-mon[32344]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:10.245 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.541+0000 7fedfee00700 1 -- 172.21.15.86:0/3105940837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075590 msgr2=0x7fedf810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.541+0000 7fedfee00700 1 --2- 172.21.15.86:0/3105940837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075590 0x7fedf810ef00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fede4009990 tx=0x7fede4009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 -- 172.21.15.86:0/3105940837 shutdown_connections 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 --2- 172.21.15.86:0/3105940837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf810f440 0x7fedf8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 --2- 172.21.15.86:0/3105940837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 --2- 172.21.15.86:0/3105940837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075590 0x7fedf810ef00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 -- 172.21.15.86:0/3105940837 >> 172.21.15.86:0/3105940837 conn(0x7fedf8079ba0 msgr2=0x7fedf8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 -- 172.21.15.86:0/3105940837 shutdown_connections 2024-04-25T19:01:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.542+0000 7fedfee00700 1 -- 172.21.15.86:0/3105940837 wait complete. 2024-04-25T19:01:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 Processor -- start 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 -- start start 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf8075590 0x7fedf819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf810f440 0x7fedf819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8114e20 con 0x7fedf8074c70 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8114ca0 con 0x7fedf810f440 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.543+0000 7fedfee00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8114fa0 con 0x7fedf8075590 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38388/0 (socket says 172.21.15.86:38388) 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 -- 172.21.15.86:0/1129611511 learned_addr learned my addr 172.21.15.86:0/1129611511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7400700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf8075590 0x7fedf819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedfce00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf810f440 0x7fedf819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 -- 172.21.15.86:0/1129611511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf8075590 msgr2=0x7fedf819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf8075590 0x7fedf819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 -- 172.21.15.86:0/1129611511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf810f440 msgr2=0x7fedf819fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf810f440 0x7fedf819fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 -- 172.21.15.86:0/1129611511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf81a02f0 con 0x7fedf8074c70 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.544+0000 7fedf7e00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf819b330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fede800e9a0 tx=0x7fede800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.545+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede800ca10 con 0x7fedf8074c70 2024-04-25T19:01:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.545+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf81a05e0 con 0x7fedf8074c70 2024-04-25T19:01:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.545+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede800cbb0 con 0x7fedf8074c70 2024-04-25T19:01:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.545+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8018680 con 0x7fedf8074c70 2024-04-25T19:01:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.545+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf81a7e60 con 0x7fedf8074c70 2024-04-25T19:01:12.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.546+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf8069730 con 0x7fedf8074c70 2024-04-25T19:01:12.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.548+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fede8010430 con 0x7fedf8074c70 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.549+0000 7fedf4c00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7feddc0a3cc0 0x7feddc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.550+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fede80d05b0 con 0x7fedf8074c70 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.550+0000 7fedf7400700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7feddc0a3cc0 0x7feddc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.550+0000 7fedf7400700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7feddc0a3cc0 0x7feddc0a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fede400b2e0 tx=0x7fede4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.554+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fede809ced0 con 0x7fedf8074c70 2024-04-25T19:01:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:12 smithi194 ceph-mon[32740]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:12.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.778+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedf8062e80 con 0x7feddc0a3cc0 2024-04-25T19:01:12.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:12 smithi086 ceph-mon[27783]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.809+0000 7fedf4c00700 1 -- 172.21.15.86:0/1129611511 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259183 (secure 0 0 0) 0x7fede4067480 con 0x7feddc0a3cc0 2024-04-25T19:01:12.816 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7feddc0a3cc0 msgr2=0x7feddc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7feddc0a3cc0 0x7feddc0a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fede400b2e0 tx=0x7fede4005960 comp rx=0 tx=0).stop 2024-04-25T19:01:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 msgr2=0x7fedf819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf819b330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fede800e9a0 tx=0x7fede800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 shutdown_connections 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7feddc0a3cc0 0x7feddc0a6180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf810f440 0x7fedf819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf8074c70 0x7fedf819b330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 --2- 172.21.15.86:0/1129611511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf8075590 0x7fedf819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 >> 172.21.15.86:0/1129611511 conn(0x7fedf8079ba0 msgr2=0x7fedf80ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.816+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 shutdown_connections 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:12.817+0000 7fedfee00700 1 -- 172.21.15.86:0/1129611511 wait complete. 2024-04-25T19:01:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T19:01:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:12 smithi183 ceph-mon[32344]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:13.669 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-04-25T19:01:11.377166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659216,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1361,"num_read_kb":808,"num_write":441,"num_write_kb":3086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361768,"kb_used_data":12528,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580056,"statfs":{"total":1151940427776,"available":1151569977344,"internally_reserved":0,"allocated":12828672,"data_stored":7973448,"data_compressed":245349,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005642"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538422+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T19:00:38.538422+0000","last_peered":"2024-04-25T19:00:38.538422+0000","last_clean":"2024-04-25T19:00:38.538422+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T19:00:38.538422+0000","last_undegraded":"2024-04-25T19:00:38.538422+0000","last_fullsized":"2024-04-25T19:00:38.538422+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.547521+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T19:00:38.547521+0000","last_peered":"2024-04-25T19:00:38.547521+0000","last_clean":"2024-04-25T19:00:38.547521+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T19:00:38.547521+0000","last_undegraded":"2024-04-25T19:00:38.547521+0000","last_fullsized":"2024-04-25T19:00:38.547521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.546878+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T19:00:38.546878+0000","last_peered":"2024-04-25T19:00:38.546878+0000","last_clean":"2024-04-25T19:00:38.546878+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T19:00:38.546878+0000","last_undegraded":"2024-04-25T19:00:38.546878+0000","last_fullsized":"2024-04-25T19:00:38.546878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.540069+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T19:00:38.540069+0000","last_peered":"2024-04-25T19:00:38.540069+0000","last_clean":"2024-04-25T19:00:38.540069+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T19:00:38.540069+0000","last_undegraded":"2024-04-25T19:00:38.540069+0000","last_fullsized":"2024-04-25T19:00:38.540069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694169+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T19:00:38.694169+0000","last_peered":"2024-04-25T19:00:38.694169+0000","last_clean":"2024-04-25T19:00:38.694169+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T19:00:38.694169+0000","last_undegraded":"2024-04-25T19:00:38.694169+0000","last_fullsized":"2024-04-25T19:00:38.694169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533020+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T19:00:38.533020+0000","last_peered":"2024-04-25T19:00:38.533020+0000","last_clean":"2024-04-25T19:00:38.533020+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T19:00:38.533020+0000","last_undegraded":"2024-04-25T19:00:38.533020+0000","last_fullsized":"2024-04-25T19:00:38.533020+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706164+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T19:00:38.706164+0000","last_peered":"2024-04-25T19:00:38.706164+0000","last_clean":"2024-04-25T19:00:38.706164+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T19:00:38.706164+0000","last_undegraded":"2024-04-25T19:00:38.706164+0000","last_fullsized":"2024-04-25T19:00:38.706164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.724643+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T19:00:38.724643+0000","last_peered":"2024-04-25T19:00:38.724643+0000","last_clean":"2024-04-25T19:00:38.724643+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T19:00:38.724643+0000","last_undegraded":"2024-04-25T19:00:38.724643+0000","last_fullsized":"2024-04-25T19:00:38.724643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.543063+0000","last_change":"2024-04-25T19:00:38.543063+0000","last_active":"2024-04-25T19:00:38.543063+0000","last_peered":"2024-04-25T19:00:38.543063+0000","last_clean":"2024-04-25T19:00:38.543063+0000","last_became_active":"2024-04-25T19:00:38.541854+0000","last_became_peered":"2024-04-25T19:00:38.541854+0000","last_unstale":"2024-04-25T19:00:38.543063+0000","last_undegraded":"2024-04-25T19:00:38.543063+0000","last_fullsized":"2024-04-25T19:00:38.543063+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:10.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533358+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T19:00:38.533358+0000","last_peered":"2024-04-25T19:00:38.533358+0000","last_clean":"2024-04-25T19:00:38.533358+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T19:00:38.533358+0000","last_undegraded":"2024-04-25T19:00:38.533358+0000","last_fullsized":"2024-04-25T19:00:38.533358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706490+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T19:00:38.706490+0000","last_peered":"2024-04-25T19:00:38.706490+0000","last_clean":"2024-04-25T19:00:38.706490+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T19:00:38.706490+0000","last_undegraded":"2024-04-25T19:00:38.706490+0000","last_fullsized":"2024-04-25T19:00:38.706490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.539891+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T19:00:38.539891+0000","last_peered":"2024-04-25T19:00:38.539891+0000","last_clean":"2024-04-25T19:00:38.539891+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T19:00:38.539891+0000","last_undegraded":"2024-04-25T19:00:38.539891+0000","last_fullsized":"2024-04-25T19:00:38.539891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719196+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T19:00:38.719196+0000","last_peered":"2024-04-25T19:00:38.719196+0000","last_clean":"2024-04-25T19:00:38.719196+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T19:00:38.719196+0000","last_undegraded":"2024-04-25T19:00:38.719196+0000","last_fullsized":"2024-04-25T19:00:38.719196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.546915+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T19:00:38.546915+0000","last_peered":"2024-04-25T19:00:38.546915+0000","last_clean":"2024-04-25T19:00:38.546915+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T19:00:38.546915+0000","last_undegraded":"2024-04-25T19:00:38.546915+0000","last_fullsized":"2024-04-25T19:00:38.546915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.547051+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T19:00:38.547051+0000","last_peered":"2024-04-25T19:00:38.547051+0000","last_clean":"2024-04-25T19:00:38.547051+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T19:00:38.547051+0000","last_undegraded":"2024-04-25T19:00:38.547051+0000","last_fullsized":"2024-04-25T19:00:38.547051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797358+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T19:00:38.797358+0000","last_peered":"2024-04-25T19:00:38.797358+0000","last_clean":"2024-04-25T19:00:38.797358+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T19:00:38.797358+0000","last_undegraded":"2024-04-25T19:00:38.797358+0000","last_fullsized":"2024-04-25T19:00:38.797358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538605+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T19:00:38.538605+0000","last_peered":"2024-04-25T19:00:38.538605+0000","last_clean":"2024-04-25T19:00:38.538605+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T19:00:38.538605+0000","last_undegraded":"2024-04-25T19:00:38.538605+0000","last_fullsized":"2024-04-25T19:00:38.538605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533610+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T19:00:38.533610+0000","last_peered":"2024-04-25T19:00:38.533610+0000","last_clean":"2024-04-25T19:00:38.533610+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T19:00:38.533610+0000","last_undegraded":"2024-04-25T19:00:38.533610+0000","last_fullsized":"2024-04-25T19:00:38.533610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636701+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T19:00:38.636701+0000","last_peered":"2024-04-25T19:00:38.636701+0000","last_clean":"2024-04-25T19:00:38.636701+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T19:00:38.636701+0000","last_undegraded":"2024-04-25T19:00:38.636701+0000","last_fullsized":"2024-04-25T19:00:38.636701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:05.760160+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T19:01:05.760160+0000","last_peered":"2024-04-25T19:01:05.760160+0000","last_clean":"2024-04-25T19:01:05.760160+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T19:01:05.760160+0000","last_undegraded":"2024-04-25T19:01:05.760160+0000","last_fullsized":"2024-04-25T19:01:05.760160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725140+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T19:00:38.725140+0000","last_peered":"2024-04-25T19:00:38.725140+0000","last_clean":"2024-04-25T19:00:38.725140+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T19:00:38.725140+0000","last_undegraded":"2024-04-25T19:00:38.725140+0000","last_fullsized":"2024-04-25T19:00:38.725140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725580+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T19:00:38.725580+0000","last_peered":"2024-04-25T19:00:38.725580+0000","last_clean":"2024-04-25T19:00:38.725580+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T19:00:38.725580+0000","last_undegraded":"2024-04-25T19:00:38.725580+0000","last_fullsized":"2024-04-25T19:00:38.725580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.548438+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T19:00:38.548438+0000","last_peered":"2024-04-25T19:00:38.548438+0000","last_clean":"2024-04-25T19:00:38.548438+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T19:00:38.548438+0000","last_undegraded":"2024-04-25T19:00:38.548438+0000","last_fullsized":"2024-04-25T19:00:38.548438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537985+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T19:00:38.537985+0000","last_peered":"2024-04-25T19:00:38.537985+0000","last_clean":"2024-04-25T19:00:38.537985+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T19:00:38.537985+0000","last_undegraded":"2024-04-25T19:00:38.537985+0000","last_fullsized":"2024-04-25T19:00:38.537985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797539+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T19:00:38.797539+0000","last_peered":"2024-04-25T19:00:38.797539+0000","last_clean":"2024-04-25T19:00:38.797539+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T19:00:38.797539+0000","last_undegraded":"2024-04-25T19:00:38.797539+0000","last_fullsized":"2024-04-25T19:00:38.797539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694131+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T19:00:38.694131+0000","last_peered":"2024-04-25T19:00:38.694131+0000","last_clean":"2024-04-25T19:00:38.694131+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T19:00:38.694131+0000","last_undegraded":"2024-04-25T19:00:38.694131+0000","last_fullsized":"2024-04-25T19:00:38.694131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535652+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T19:00:38.535652+0000","last_peered":"2024-04-25T19:00:38.535652+0000","last_clean":"2024-04-25T19:00:38.535652+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T19:00:38.535652+0000","last_undegraded":"2024-04-25T19:00:38.535652+0000","last_fullsized":"2024-04-25T19:00:38.535652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:09.676403+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T19:01:09.676403+0000","last_peered":"2024-04-25T19:01:09.676403+0000","last_clean":"2024-04-25T19:01:09.676403+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T19:01:09.676403+0000","last_undegraded":"2024-04-25T19:01:09.676403+0000","last_fullsized":"2024-04-25T19:01:09.676403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.693802+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T19:00:38.693802+0000","last_peered":"2024-04-25T19:00:38.693802+0000","last_clean":"2024-04-25T19:00:38.693802+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T19:00:38.693802+0000","last_undegraded":"2024-04-25T19:00:38.693802+0000","last_fullsized":"2024-04-25T19:00:38.693802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.532948+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T19:00:38.532948+0000","last_peered":"2024-04-25T19:00:38.532948+0000","last_clean":"2024-04-25T19:00:38.532948+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T19:00:38.532948+0000","last_undegraded":"2024-04-25T19:00:38.532948+0000","last_fullsized":"2024-04-25T19:00:38.532948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.695351+0000","last_change":"2024-04-25T19:00:38.695351+0000","last_active":"2024-04-25T19:00:38.695351+0000","last_peered":"2024-04-25T19:00:38.695351+0000","last_clean":"2024-04-25T19:00:38.695351+0000","last_became_active":"2024-04-25T19:00:38.694923+0000","last_became_peered":"2024-04-25T19:00:38.694923+0000","last_unstale":"2024-04-25T19:00:38.695351+0000","last_undegraded":"2024-04-25T19:00:38.695351+0000","last_fullsized":"2024-04-25T19:00:38.695351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:52.773252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.718843+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T19:00:38.718843+0000","last_peered":"2024-04-25T19:00:38.718843+0000","last_clean":"2024-04-25T19:00:38.718843+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T19:00:38.718843+0000","last_undegraded":"2024-04-25T19:00:38.718843+0000","last_fullsized":"2024-04-25T19:00:38.718843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725817+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T19:00:38.725817+0000","last_peered":"2024-04-25T19:00:38.725817+0000","last_clean":"2024-04-25T19:00:38.725817+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T19:00:38.725817+0000","last_undegraded":"2024-04-25T19:00:38.725817+0000","last_fullsized":"2024-04-25T19:00:38.725817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.534684+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T19:00:38.534684+0000","last_peered":"2024-04-25T19:00:38.534684+0000","last_clean":"2024-04-25T19:00:38.534684+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T19:00:38.534684+0000","last_undegraded":"2024-04-25T19:00:38.534684+0000","last_fullsized":"2024-04-25T19:00:38.534684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.532953+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T19:00:38.532953+0000","last_peered":"2024-04-25T19:00:38.532953+0000","last_clean":"2024-04-25T19:00:38.532953+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T19:00:38.532953+0000","last_undegraded":"2024-04-25T19:00:38.532953+0000","last_fullsized":"2024-04-25T19:00:38.532953+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":274,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:09.676054+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T19:01:09.676054+0000","last_peered":"2024-04-25T19:01:09.676054+0000","last_clean":"2024-04-25T19:01:09.676054+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T19:01:09.676054+0000","last_undegraded":"2024-04-25T19:01:09.676054+0000","last_fullsized":"2024-04-25T19:01:09.676054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797882+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T19:00:38.797882+0000","last_peered":"2024-04-25T19:00:38.797882+0000","last_clean":"2024-04-25T19:00:38.797882+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T19:00:38.797882+0000","last_undegraded":"2024-04-25T19:00:38.797882+0000","last_fullsized":"2024-04-25T19:00:38.797882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706603+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T19:00:38.706603+0000","last_peered":"2024-04-25T19:00:38.706603+0000","last_clean":"2024-04-25T19:00:38.706603+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T19:00:38.706603+0000","last_undegraded":"2024-04-25T19:00:38.706603+0000","last_fullsized":"2024-04-25T19:00:38.706603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.722645+0000","last_change":"2024-04-25T19:00:38.722645+0000","last_active":"2024-04-25T19:00:38.722645+0000","last_peered":"2024-04-25T19:00:38.722645+0000","last_clean":"2024-04-25T19:00:38.722645+0000","last_became_active":"2024-04-25T19:00:38.722223+0000","last_became_peered":"2024-04-25T19:00:38.722223+0000","last_unstale":"2024-04-25T19:00:38.722645+0000","last_undegraded":"2024-04-25T19:00:38.722645+0000","last_fullsized":"2024-04-25T19:00:38.722645+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:51.266531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535824+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T19:00:38.535824+0000","last_peered":"2024-04-25T19:00:38.535824+0000","last_clean":"2024-04-25T19:00:38.535824+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T19:00:38.535824+0000","last_undegraded":"2024-04-25T19:00:38.535824+0000","last_fullsized":"2024-04-25T19:00:38.535824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797746+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T19:00:38.797746+0000","last_peered":"2024-04-25T19:00:38.797746+0000","last_clean":"2024-04-25T19:00:38.797746+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T19:00:38.797746+0000","last_undegraded":"2024-04-25T19:00:38.797746+0000","last_fullsized":"2024-04-25T19:00:38.797746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":313,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:09.675841+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T19:01:09.675841+0000","last_peered":"2024-04-25T19:01:09.675841+0000","last_clean":"2024-04-25T19:01:09.675841+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T19:01:09.675841+0000","last_undegraded":"2024-04-25T19:01:09.675841+0000","last_fullsized":"2024-04-25T19:01:09.675841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":183,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694138+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T19:00:38.694138+0000","last_peered":"2024-04-25T19:00:38.694138+0000","last_clean":"2024-04-25T19:00:38.694138+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T19:00:38.694138+0000","last_undegraded":"2024-04-25T19:00:38.694138+0000","last_fullsized":"2024-04-25T19:00:38.694138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636627+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T19:00:38.636627+0000","last_peered":"2024-04-25T19:00:38.636627+0000","last_clean":"2024-04-25T19:00:38.636627+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T19:00:38.636627+0000","last_undegraded":"2024-04-25T19:00:38.636627+0000","last_fullsized":"2024-04-25T19:00:38.636627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.718514+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T19:00:38.718514+0000","last_peered":"2024-04-25T19:00:38.718514+0000","last_clean":"2024-04-25T19:00:38.718514+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T19:00:38.718514+0000","last_undegraded":"2024-04-25T19:00:38.718514+0000","last_fullsized":"2024-04-25T19:00:38.718514+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535059+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T19:00:38.535059+0000","last_peered":"2024-04-25T19:00:38.535059+0000","last_clean":"2024-04-25T19:00:38.535059+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T19:00:38.535059+0000","last_undegraded":"2024-04-25T19:00:38.535059+0000","last_fullsized":"2024-04-25T19:00:38.535059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706563+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T19:00:38.706563+0000","last_peered":"2024-04-25T19:00:38.706563+0000","last_clean":"2024-04-25T19:00:38.706563+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T19:00:38.706563+0000","last_undegraded":"2024-04-25T19:00:38.706563+0000","last_fullsized":"2024-04-25T19:00:38.706563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:08.794350+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T19:01:08.794350+0000","last_peered":"2024-04-25T19:01:08.794350+0000","last_clean":"2024-04-25T19:01:08.794350+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T19:01:08.794350+0000","last_undegraded":"2024-04-25T19:01:08.794350+0000","last_fullsized":"2024-04-25T19:01:08.794350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706102+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T19:00:38.706102+0000","last_peered":"2024-04-25T19:00:38.706102+0000","last_clean":"2024-04-25T19:00:38.706102+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T19:00:38.706102+0000","last_undegraded":"2024-04-25T19:00:38.706102+0000","last_fullsized":"2024-04-25T19:00:38.706102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.693824+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T19:00:38.693824+0000","last_peered":"2024-04-25T19:00:38.693824+0000","last_clean":"2024-04-25T19:00:38.693824+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T19:00:38.693824+0000","last_undegraded":"2024-04-25T19:00:38.693824+0000","last_fullsized":"2024-04-25T19:00:38.693824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.534948+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T19:00:38.534948+0000","last_peered":"2024-04-25T19:00:38.534948+0000","last_clean":"2024-04-25T19:00:38.534948+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T19:00:38.534948+0000","last_undegraded":"2024-04-25T19:00:38.534948+0000","last_fullsized":"2024-04-25T19:00:38.534948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694142+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T19:00:38.694142+0000","last_peered":"2024-04-25T19:00:38.694142+0000","last_clean":"2024-04-25T19:00:38.694142+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T19:00:38.694142+0000","last_undegraded":"2024-04-25T19:00:38.694142+0000","last_fullsized":"2024-04-25T19:00:38.694142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538165+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T19:00:38.538165+0000","last_peered":"2024-04-25T19:00:38.538165+0000","last_clean":"2024-04-25T19:00:38.538165+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T19:00:38.538165+0000","last_undegraded":"2024-04-25T19:00:38.538165+0000","last_fullsized":"2024-04-25T19:00:38.538165+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533297+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T19:00:38.533297+0000","last_peered":"2024-04-25T19:00:38.533297+0000","last_clean":"2024-04-25T19:00:38.533297+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T19:00:38.533297+0000","last_undegraded":"2024-04-25T19:00:38.533297+0000","last_fullsized":"2024-04-25T19:00:38.533297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533176+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T19:00:38.533176+0000","last_peered":"2024-04-25T19:00:38.533176+0000","last_clean":"2024-04-25T19:00:38.533176+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T19:00:38.533176+0000","last_undegraded":"2024-04-25T19:00:38.533176+0000","last_fullsized":"2024-04-25T19:00:38.533176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:04.758746+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T19:01:04.758746+0000","last_peered":"2024-04-25T19:01:04.758746+0000","last_clean":"2024-04-25T19:01:04.758746+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T19:01:04.758746+0000","last_undegraded":"2024-04-25T19:01:04.758746+0000","last_fullsized":"2024-04-25T19:01:04.758746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533170+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T19:00:38.533170+0000","last_peered":"2024-04-25T19:00:38.533170+0000","last_clean":"2024-04-25T19:00:38.533170+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T19:00:38.533170+0000","last_undegraded":"2024-04-25T19:00:38.533170+0000","last_fullsized":"2024-04-25T19:00:38.533170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533491+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T19:00:38.533491+0000","last_peered":"2024-04-25T19:00:38.533491+0000","last_clean":"2024-04-25T19:00:38.533491+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T19:00:38.533491+0000","last_undegraded":"2024-04-25T19:00:38.533491+0000","last_fullsized":"2024-04-25T19:00:38.533491+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:08.793505+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T19:01:08.793505+0000","last_peered":"2024-04-25T19:01:08.793505+0000","last_clean":"2024-04-25T19:01:08.793505+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T19:01:08.793505+0000","last_undegraded":"2024-04-25T19:01:08.793505+0000","last_fullsized":"2024-04-25T19:01:08.793505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725397+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T19:00:38.725397+0000","last_peered":"2024-04-25T19:00:38.725397+0000","last_clean":"2024-04-25T19:00:38.725397+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T19:00:38.725397+0000","last_undegraded":"2024-04-25T19:00:38.725397+0000","last_fullsized":"2024-04-25T19:00:38.725397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.539671+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T19:00:38.539671+0000","last_peered":"2024-04-25T19:00:38.539671+0000","last_clean":"2024-04-25T19:00:38.539671+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T19:00:38.539671+0000","last_undegraded":"2024-04-25T19:00:38.539671+0000","last_fullsized":"2024-04-25T19:00:38.539671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706211+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T19:00:38.706211+0000","last_peered":"2024-04-25T19:00:38.706211+0000","last_clean":"2024-04-25T19:00:38.706211+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T19:00:38.706211+0000","last_undegraded":"2024-04-25T19:00:38.706211+0000","last_fullsized":"2024-04-25T19:00:38.706211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.718968+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T19:00:38.718968+0000","last_peered":"2024-04-25T19:00:38.718968+0000","last_clean":"2024-04-25T19:00:38.718968+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T19:00:38.718968+0000","last_undegraded":"2024-04-25T19:00:38.718968+0000","last_fullsized":"2024-04-25T19:00:38.718968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706475+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T19:00:38.706475+0000","last_peered":"2024-04-25T19:00:38.706475+0000","last_clean":"2024-04-25T19:00:38.706475+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T19:00:38.706475+0000","last_undegraded":"2024-04-25T19:00:38.706475+0000","last_fullsized":"2024-04-25T19:00:38.706475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537718+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T19:00:38.537718+0000","last_peered":"2024-04-25T19:00:38.537718+0000","last_clean":"2024-04-25T19:00:38.537718+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T19:00:38.537718+0000","last_undegraded":"2024-04-25T19:00:38.537718+0000","last_fullsized":"2024-04-25T19:00:38.537718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719609+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T19:00:38.719609+0000","last_peered":"2024-04-25T19:00:38.719609+0000","last_clean":"2024-04-25T19:00:38.719609+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T19:00:38.719609+0000","last_undegraded":"2024-04-25T19:00:38.719609+0000","last_fullsized":"2024-04-25T19:00:38.719609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.533675+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T19:00:38.533675+0000","last_peered":"2024-04-25T19:00:38.533675+0000","last_clean":"2024-04-25T19:00:38.533675+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T19:00:38.533675+0000","last_undegraded":"2024-04-25T19:00:38.533675+0000","last_fullsized":"2024-04-25T19:00:38.533675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725315+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T19:00:38.725315+0000","last_peered":"2024-04-25T19:00:38.725315+0000","last_clean":"2024-04-25T19:00:38.725315+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T19:00:38.725315+0000","last_undegraded":"2024-04-25T19:00:38.725315+0000","last_fullsized":"2024-04-25T19:00:38.725315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.540179+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T19:00:38.540179+0000","last_peered":"2024-04-25T19:00:38.540179+0000","last_clean":"2024-04-25T19:00:38.540179+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T19:00:38.540179+0000","last_undegraded":"2024-04-25T19:00:38.540179+0000","last_fullsized":"2024-04-25T19:00:38.540179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.549759+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T19:00:38.549759+0000","last_peered":"2024-04-25T19:00:38.549759+0000","last_clean":"2024-04-25T19:00:38.549759+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T19:00:38.549759+0000","last_undegraded":"2024-04-25T19:00:38.549759+0000","last_fullsized":"2024-04-25T19:00:38.549759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797448+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T19:00:38.797448+0000","last_peered":"2024-04-25T19:00:38.797448+0000","last_clean":"2024-04-25T19:00:38.797448+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T19:00:38.797448+0000","last_undegraded":"2024-04-25T19:00:38.797448+0000","last_fullsized":"2024-04-25T19:00:38.797448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.693856+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T19:00:38.693856+0000","last_peered":"2024-04-25T19:00:38.693856+0000","last_clean":"2024-04-25T19:00:38.693856+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T19:00:38.693856+0000","last_undegraded":"2024-04-25T19:00:38.693856+0000","last_fullsized":"2024-04-25T19:00:38.693856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.724830+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T19:00:38.724830+0000","last_peered":"2024-04-25T19:00:38.724830+0000","last_clean":"2024-04-25T19:00:38.724830+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T19:00:38.724830+0000","last_undegraded":"2024-04-25T19:00:38.724830+0000","last_fullsized":"2024-04-25T19:00:38.724830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537789+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T19:00:38.537789+0000","last_peered":"2024-04-25T19:00:38.537789+0000","last_clean":"2024-04-25T19:00:38.537789+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T19:00:38.537789+0000","last_undegraded":"2024-04-25T19:00:38.537789+0000","last_fullsized":"2024-04-25T19:00:38.537789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636604+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T19:00:38.636604+0000","last_peered":"2024-04-25T19:00:38.636604+0000","last_clean":"2024-04-25T19:00:38.636604+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T19:00:38.636604+0000","last_undegraded":"2024-04-25T19:00:38.636604+0000","last_fullsized":"2024-04-25T19:00:38.636604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725310+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T19:00:38.725310+0000","last_peered":"2024-04-25T19:00:38.725310+0000","last_clean":"2024-04-25T19:00:38.725310+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T19:00:38.725310+0000","last_undegraded":"2024-04-25T19:00:38.725310+0000","last_fullsized":"2024-04-25T19:00:38.725310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.694228+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T19:00:38.694228+0000","last_peered":"2024-04-25T19:00:38.694228+0000","last_clean":"2024-04-25T19:00:38.694228+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T19:00:38.694228+0000","last_undegraded":"2024-04-25T19:00:38.694228+0000","last_fullsized":"2024-04-25T19:00:38.694228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706360+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T19:00:38.706360+0000","last_peered":"2024-04-25T19:00:38.706360+0000","last_clean":"2024-04-25T19:00:38.706360+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T19:00:38.706360+0000","last_undegraded":"2024-04-25T19:00:38.706360+0000","last_fullsized":"2024-04-25T19:00:38.706360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:01:06.672415+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T19:01:06.672415+0000","last_peered":"2024-04-25T19:01:06.672415+0000","last_clean":"2024-04-25T19:01:06.672415+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T19:01:06.672415+0000","last_undegraded":"2024-04-25T19:01:06.672415+0000","last_fullsized":"2024-04-25T19:01:06.672415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706731+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T19:00:38.706731+0000","last_peered":"2024-04-25T19:00:38.706731+0000","last_clean":"2024-04-25T19:00:38.706731+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T19:00:38.706731+0000","last_undegraded":"2024-04-25T19:00:38.706731+0000","last_fullsized":"2024-04-25T19:00:38.706731+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.535583+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T19:00:38.535583+0000","last_peered":"2024-04-25T19:00:38.535583+0000","last_clean":"2024-04-25T19:00:38.535583+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T19:00:38.535583+0000","last_undegraded":"2024-04-25T19:00:38.535583+0000","last_fullsized":"2024-04-25T19:00:38.535583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706601+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T19:00:38.706601+0000","last_peered":"2024-04-25T19:00:38.706601+0000","last_clean":"2024-04-25T19:00:38.706601+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T19:00:38.706601+0000","last_undegraded":"2024-04-25T19:00:38.706601+0000","last_fullsized":"2024-04-25T19:00:38.706601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.720146+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T19:00:38.720146+0000","last_peered":"2024-04-25T19:00:38.720146+0000","last_clean":"2024-04-25T19:00:38.720146+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T19:00:38.720146+0000","last_undegraded":"2024-04-25T19:00:38.720146+0000","last_fullsized":"2024-04-25T19:00:38.720146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.706785+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T19:00:38.706785+0000","last_peered":"2024-04-25T19:00:38.706785+0000","last_clean":"2024-04-25T19:00:38.706785+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T19:00:38.706785+0000","last_undegraded":"2024-04-25T19:00:38.706785+0000","last_fullsized":"2024-04-25T19:00:38.706785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.537896+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T19:00:38.537896+0000","last_peered":"2024-04-25T19:00:38.537896+0000","last_clean":"2024-04-25T19:00:38.537896+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T19:00:38.537896+0000","last_undegraded":"2024-04-25T19:00:38.537896+0000","last_fullsized":"2024-04-25T19:00:38.537896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636647+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T19:00:38.636647+0000","last_peered":"2024-04-25T19:00:38.636647+0000","last_clean":"2024-04-25T19:00:38.636647+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T19:00:38.636647+0000","last_undegraded":"2024-04-25T19:00:38.636647+0000","last_fullsized":"2024-04-25T19:00:38.636647+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636527+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T19:00:38.636527+0000","last_peered":"2024-04-25T19:00:38.636527+0000","last_clean":"2024-04-25T19:00:38.636527+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T19:00:38.636527+0000","last_undegraded":"2024-04-25T19:00:38.636527+0000","last_fullsized":"2024-04-25T19:00:38.636527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.550116+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T19:00:38.550116+0000","last_peered":"2024-04-25T19:00:38.550116+0000","last_clean":"2024-04-25T19:00:38.550116+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T19:00:38.550116+0000","last_undegraded":"2024-04-25T19:00:38.550116+0000","last_fullsized":"2024-04-25T19:00:38.550116+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719855+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T19:00:38.719855+0000","last_peered":"2024-04-25T19:00:38.719855+0000","last_clean":"2024-04-25T19:00:38.719855+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T19:00:38.719855+0000","last_undegraded":"2024-04-25T19:00:38.719855+0000","last_fullsized":"2024-04-25T19:00:38.719855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.538136+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T19:00:38.538136+0000","last_peered":"2024-04-25T19:00:38.538136+0000","last_clean":"2024-04-25T19:00:38.538136+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T19:00:38.538136+0000","last_undegraded":"2024-04-25T19:00:38.538136+0000","last_fullsized":"2024-04-25T19:00:38.538136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.636838+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T19:00:38.636838+0000","last_peered":"2024-04-25T19:00:38.636838+0000","last_clean":"2024-04-25T19:00:38.636838+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T19:00:38.636838+0000","last_undegraded":"2024-04-25T19:00:38.636838+0000","last_fullsized":"2024-04-25T19:00:38.636838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.724985+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T19:00:38.724985+0000","last_peered":"2024-04-25T19:00:38.724985+0000","last_clean":"2024-04-25T19:00:38.724985+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T19:00:38.724985+0000","last_undegraded":"2024-04-25T19:00:38.724985+0000","last_fullsized":"2024-04-25T19:00:38.724985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.534625+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T19:00:38.534625+0000","last_peered":"2024-04-25T19:00:38.534625+0000","last_clean":"2024-04-25T19:00:38.534625+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T19:00:38.534625+0000","last_undegraded":"2024-04-25T19:00:38.534625+0000","last_fullsized":"2024-04-25T19:00:38.534625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.725193+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T19:00:38.725193+0000","last_peered":"2024-04-25T19:00:38.725193+0000","last_clean":"2024-04-25T19:00:38.725193+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T19:00:38.725193+0000","last_undegraded":"2024-04-25T19:00:38.725193+0000","last_fullsized":"2024-04-25T19:00:38.725193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.797654+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T19:00:38.797654+0000","last_peered":"2024-04-25T19:00:38.797654+0000","last_clean":"2024-04-25T19:00:38.797654+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T19:00:38.797654+0000","last_undegraded":"2024-04-25T19:00:38.797654+0000","last_fullsized":"2024-04-25T19:00:38.797654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.719192+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T19:00:38.719192+0000","last_peered":"2024-04-25T19:00:38.719192+0000","last_clean":"2024-04-25T19:00:38.719192+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T19:00:38.719192+0000","last_undegraded":"2024-04-25T19:00:38.719192+0000","last_fullsized":"2024-04-25T19:00:38.719192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T19:00:38.536963+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T19:00:38.536963+0000","last_peered":"2024-04-25T19:00:38.536963+0000","last_clean":"2024-04-25T19:00:38.536963+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T19:00:38.536963+0000","last_undegraded":"2024-04-25T19:00:38.536963+0000","last_fullsized":"2024-04-25T19:00:38.536963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279982,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68614,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1225,"num_read_kb":689,"num_write":258,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351022,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":500954,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612660,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1152999,"data_compressed":47157,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":500772,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":501038,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1094873,"data_compressed":47157,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":501021,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":562530,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":501828,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":562750,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":500314,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500882,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1093487,"data_compressed":47157,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33969,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:01:13.676 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T19:01:13.676 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T19:01:13.677 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T19:01:13.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T19:01:14.156 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:14.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:14 smithi086 ceph-mon[27783]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:01:14.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:14 smithi086 ceph-mon[27783]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:14 smithi194 ceph-mon[32740]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:01:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:14 smithi194 ceph-mon[32740]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:14 smithi183 ceph-mon[32344]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:01:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:14 smithi183 ceph-mon[32344]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 -- 172.21.15.86:0/3362187626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 msgr2=0x7f0708106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 --2- 172.21.15.86:0/3362187626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 0x7f0708106780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009990 tx=0x7f06f4009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 -- 172.21.15.86:0/3362187626 shutdown_connections 2024-04-25T19:01:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 --2- 172.21.15.86:0/3362187626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708106da0 0x7f0708107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 --2- 172.21.15.86:0/3362187626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708101990 0x7f0708101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 --2- 172.21.15.86:0/3362187626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 0x7f0708106780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 -- 172.21.15.86:0/3362187626 >> 172.21.15.86:0/3362187626 conn(0x7f07080fd3f0 msgr2=0x7f07080ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 -- 172.21.15.86:0/3362187626 shutdown_connections 2024-04-25T19:01:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.357+0000 7f070d600700 1 -- 172.21.15.86:0/3362187626 wait complete. 2024-04-25T19:01:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 Processor -- start 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 -- start start 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 0x7f070819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 0x7f070819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708106da0 0x7f07081a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07081191f0 con 0x7f0708106da0 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0708119070 con 0x7f07081022b0 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.359+0000 7f070d600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0708119370 con 0x7f0708101990 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 0x7f070819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 0x7f070819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:57516/0 (socket says 172.21.15.86:57516) 2024-04-25T19:01:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 -- 172.21.15.86:0/4059285137 learned_addr learned my addr 172.21.15.86:0/4059285137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f070cc00700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708106da0 0x7f07081a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707400700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 0x7f070819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 -- 172.21.15.86:0/4059285137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 msgr2=0x7f070819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 0x7f070819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 -- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708106da0 msgr2=0x7f07081a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708106da0 0x7f07081a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 -- 172.21.15.86:0/4059285137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07081a4750 con 0x7f0708101990 2024-04-25T19:01:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.360+0000 7f0707e00700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 0x7f070819f740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06f800e9a0 tx=0x7f06f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.361+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f800c9c0 con 0x7f0708101990 2024-04-25T19:01:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.361+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07081a4a40 con 0x7f0708101990 2024-04-25T19:01:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.361+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f800cb60 con 0x7f0708101990 2024-04-25T19:01:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.361+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8018680 con 0x7f0708101990 2024-04-25T19:01:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.361+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07081ac2c0 con 0x7f0708101990 2024-04-25T19:01:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.362+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0708069730 con 0x7f0708101990 2024-04-25T19:01:16.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.364+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f06f8010430 con 0x7f0708101990 2024-04-25T19:01:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.365+0000 7f0704c00700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06ec0a3c40 0x7f06ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.366+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f06f80d0ab0 con 0x7f0708101990 2024-04-25T19:01:16.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.366+0000 7f0707400700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06ec0a3c40 0x7f06ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:16.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.366+0000 7f0707400700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06ec0a3c40 0x7f06ec0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0708100b30 tx=0x7f06f400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:16.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.368+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06f809d3d0 con 0x7f0708101990 2024-04-25T19:01:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.639+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f07081a0aa0 con 0x7f0708101990 2024-04-25T19:01:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.639+0000 7f0704c00700 1 -- 172.21.15.86:0/4059285137 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f06f809ccd0 con 0x7f0708101990 2024-04-25T19:01:16.642 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:16.642 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06ec0a3c40 msgr2=0x7f06ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06ec0a3c40 0x7f06ec0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0708100b30 tx=0x7f06f400b260 comp rx=0 tx=0).stop 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 msgr2=0x7f070819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 0x7f070819f740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06f800e9a0 tx=0x7f06f800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 shutdown_connections 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06ec0a3c40 0x7f06ec0a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708106da0 0x7f07081a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0708101990 0x7f070819f740 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 --2- 172.21.15.86:0/4059285137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07081022b0 0x7f070819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:16.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 >> 172.21.15.86:0/4059285137 conn(0x7f07080fd3f0 msgr2=0x7f0708110560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:16.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.642+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 shutdown_connections 2024-04-25T19:01:16.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:16.643+0000 7f070d600700 1 -- 172.21.15.86:0/4059285137 wait complete. 2024-04-25T19:01:16.697 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:16 smithi086 ceph-mon[27783]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:16 smithi194 ceph-mon[32740]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:16 smithi183 ceph-mon[32344]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:17.498 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T19:01:17.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:01:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:17 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/4059285137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T19:01:17.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:17 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/4059285137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T19:01:17.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:17 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/4059285137' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T19:01:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:18 smithi194 ceph-mon[32740]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:01:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:18 smithi086 ceph-mon[27783]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:01:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:18 smithi183 ceph-mon[32344]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T19:01:20.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 -- 172.21.15.86:0/2160456137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 msgr2=0x7fd8d0109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:20.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 --2- 172.21.15.86:0/2160456137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 0x7fd8d0109400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009990 tx=0x7fd8bc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 -- 172.21.15.86:0/2160456137 shutdown_connections 2024-04-25T19:01:20.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 --2- 172.21.15.86:0/2160456137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d0115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 --2- 172.21.15.86:0/2160456137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d010f030 0x7fd8d010f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 --2- 172.21.15.86:0/2160456137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 0x7fd8d0109400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 -- 172.21.15.86:0/2160456137 >> 172.21.15.86:0/2160456137 conn(0x7fd8d00787d0 msgr2=0x7fd8d0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:20.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.050+0000 7fd8cf200700 1 -- 172.21.15.86:0/2160456137 shutdown_connections 2024-04-25T19:01:20.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.051+0000 7fd8cf200700 1 -- 172.21.15.86:0/2160456137 wait complete. 2024-04-25T19:01:20.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.051+0000 7fd8cf200700 1 Processor -- start 2024-04-25T19:01:20.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 -- start start 2024-04-25T19:01:20.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 0x7fd8d019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d010f030 0x7fd8d01a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01190f0 con 0x7fd8d010f030 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0118f70 con 0x7fd8d0108f80 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cf200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0119270 con 0x7fd8d0109940 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cd400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8ce800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d010f030 0x7fd8d01a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cd400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d019fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46900/0 (socket says 172.21.15.86:46900) 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.052+0000 7fd8cd400700 1 -- 172.21.15.86:0/422806445 learned_addr learned my addr 172.21.15.86:0/422806445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cde00700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 0x7fd8d019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cd400700 1 -- 172.21.15.86:0/422806445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 msgr2=0x7fd8d019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cd400700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 0x7fd8d019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cd400700 1 -- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d010f030 msgr2=0x7fd8d01a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:20.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cd400700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d010f030 0x7fd8d01a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cd400700 1 -- 172.21.15.86:0/422806445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d01a4580 con 0x7fd8d0109940 2024-04-25T19:01:20.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8cd400700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d019fb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc000c00 tx=0x7fd8bc00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:20.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc027070 con 0x7fd8d0109940 2024-04-25T19:01:20.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.054+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d01a4810 con 0x7fd8d0109940 2024-04-25T19:01:20.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.053+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8bc022470 con 0x7fd8d0109940 2024-04-25T19:01:20.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.054+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc017650 con 0x7fd8d0109940 2024-04-25T19:01:20.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.054+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d01ac0f0 con 0x7fd8d0109940 2024-04-25T19:01:20.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.055+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d0069730 con 0x7fd8d0109940 2024-04-25T19:01:20.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.057+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd8bc02a070 con 0x7fd8d0109940 2024-04-25T19:01:20.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.058+0000 7fd8c6a00700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd8b40a3c40 0x7fd8b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:20.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.058+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fd8bc0d15a0 con 0x7fd8d0109940 2024-04-25T19:01:20.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.059+0000 7fd8cde00700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd8b40a3c40 0x7fd8b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:20.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.059+0000 7fd8cde00700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd8b40a3c40 0x7fd8b40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009680 tx=0x7fd8c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:20.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.061+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8bc013070 con 0x7fd8d0109940 2024-04-25T19:01:20.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.335+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8d002df80 con 0x7fd8d0109940 2024-04-25T19:01:20.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.337+0000 7fd8c6a00700 1 -- 172.21.15.86:0/422806445 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11937 (secure 0 0 0) 0x7fd8bc09df50 con 0x7fd8d0109940 2024-04-25T19:01:20.339 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:20.339 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T18:59:15.320078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24478,24527,24493,24494,24509]},"id":1}]} 2024-04-25T19:01:20.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd8b40a3c40 msgr2=0x7fd8b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:20.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd8b40a3c40 0x7fd8b40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009680 tx=0x7fd8c0009260 comp rx=0 tx=0).stop 2024-04-25T19:01:20.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 msgr2=0x7fd8d019fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d019fb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc000c00 tx=0x7fd8bc00bfd0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 shutdown_connections 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd8b40a3c40 0x7fd8b40a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d010f030 0x7fd8d01a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0108f80 0x7fd8d019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 --2- 172.21.15.86:0/422806445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8d0109940 0x7fd8d019fb00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:20.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 >> 172.21.15.86:0/422806445 conn(0x7fd8d00787d0 msgr2=0x7fd8d0079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 shutdown_connections 2024-04-25T19:01:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:20.339+0000 7fd8cf200700 1 -- 172.21.15.86:0/422806445 wait complete. 2024-04-25T19:01:20.343 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T19:01:20.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:20 smithi086 ceph-mon[27783]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:20.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:20 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/422806445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:20 smithi194 ceph-mon[32740]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:20 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/422806445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:20 smithi183 ceph-mon[32344]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:20 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/422806445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:21.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:01:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:22 smithi194 ceph-mon[32740]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:22.802 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:22 smithi086 ceph-mon[27783]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:22 smithi183 ceph-mon[32344]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 -- 172.21.15.86:0/3832171548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40104ec0 msgr2=0x7f4e401052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 --2- 172.21.15.86:0/3832171548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40104ec0 0x7f4e401052a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e300099c0 tx=0x7f4e30009cd0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 -- 172.21.15.86:0/3832171548 shutdown_connections 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 --2- 172.21.15.86:0/3832171548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e40109fa0 0x7f4e40111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 --2- 172.21.15.86:0/3832171548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40104ec0 0x7f4e401052a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 --2- 172.21.15.86:0/3832171548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40105870 0x7f4e401098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 -- 172.21.15.86:0/3832171548 >> 172.21.15.86:0/3832171548 conn(0x7f4e40100750 msgr2=0x7f4e40102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 -- 172.21.15.86:0/3832171548 shutdown_connections 2024-04-25T19:01:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.912+0000 7f4e46000700 1 -- 172.21.15.86:0/3832171548 wait complete. 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.913+0000 7f4e46000700 1 Processor -- start 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.913+0000 7f4e46000700 1 -- start start 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e46000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 0x7f4e4019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e46000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 0x7f4e4019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e46000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e40109fa0 0x7f4e4019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e46000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40114e80 con 0x7f4e40105870 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e46000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40114d00 con 0x7f4e40104ec0 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e46000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40115000 con 0x7f4e40109fa0 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e3ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 0x7f4e4019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e3f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 0x7f4e4019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e3ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 0x7f4e4019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34622/0 (socket says 172.21.15.86:34622) 2024-04-25T19:01:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e3f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 0x7f4e4019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:41856/0 (socket says 172.21.15.86:41856) 2024-04-25T19:01:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e3ea00700 1 -- 172.21.15.86:0/1930801289 learned_addr learned my addr 172.21.15.86:0/1930801289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.914+0000 7f4e3fe00700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e40109fa0 0x7f4e4019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3f400700 1 -- 172.21.15.86:0/1930801289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e40109fa0 msgr2=0x7f4e4019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3f400700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e40109fa0 0x7f4e4019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3f400700 1 -- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 msgr2=0x7f4e4019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3f400700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 0x7f4e4019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3f400700 1 -- 172.21.15.86:0/1930801289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e401a03f0 con 0x7f4e40104ec0 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3ea00700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 0x7f4e4019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e3f400700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 0x7f4e4019b430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e300098e0 tx=0x7f4e300055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.915+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30026070 con 0x7f4e40104ec0 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.916+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e401a0680 con 0x7f4e40104ec0 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.916+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3000f460 con 0x7f4e40104ec0 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.916+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30009e60 con 0x7f4e40104ec0 2024-04-25T19:01:23.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.916+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401a7f60 con 0x7f4e40104ec0 2024-04-25T19:01:23.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.917+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e40069730 con 0x7f4e40104ec0 2024-04-25T19:01:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.919+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e30029070 con 0x7f4e40104ec0 2024-04-25T19:01:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.920+0000 7f4e37e00700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4e240a3cc0 0x7f4e240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.920+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4e300d0ed0 con 0x7f4e40104ec0 2024-04-25T19:01:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.920+0000 7f4e3ea00700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4e240a3cc0 0x7f4e240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.921+0000 7f4e3ea00700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4e240a3cc0 0x7f4e240a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e4010b600 tx=0x7f4e2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:23.922+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e30030020 con 0x7f4e40104ec0 2024-04-25T19:01:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.196+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e4002df80 con 0x7f4e40104ec0 2024-04-25T19:01:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.197+0000 7f4e37e00700 1 -- 172.21.15.86:0/1930801289 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11937 (secure 0 0 0) 0x7f4e30013070 con 0x7f4e40104ec0 2024-04-25T19:01:24.199 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:24.199 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T18:59:15.320078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24478,24527,24493,24494,24509]},"id":1}]} 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4e240a3cc0 msgr2=0x7f4e240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4e240a3cc0 0x7f4e240a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e4010b600 tx=0x7f4e2c009260 comp rx=0 tx=0).stop 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 msgr2=0x7f4e4019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 0x7f4e4019b430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e300098e0 tx=0x7f4e300055f0 comp rx=0 tx=0).stop 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 shutdown_connections 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4e240a3cc0 0x7f4e240a6180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e40109fa0 0x7f4e4019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e40104ec0 0x7f4e4019b430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 --2- 172.21.15.86:0/1930801289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40105870 0x7f4e4019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 >> 172.21.15.86:0/1930801289 conn(0x7f4e40100750 msgr2=0x7f4e40102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.200+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 shutdown_connections 2024-04-25T19:01:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:24.201+0000 7f4e46000700 1 -- 172.21.15.86:0/1930801289 wait complete. 2024-04-25T19:01:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T19:01:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:24 smithi194 ceph-mon[32740]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:24 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1930801289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:24 smithi086 ceph-mon[27783]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:24 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1930801289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:24 smithi183 ceph-mon[32344]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:24 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1930801289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:25.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T19:01:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:26.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:26 smithi086 ceph-mon[27783]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:26 smithi194 ceph-mon[32740]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:26 smithi183 ceph-mon[32344]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 -- 172.21.15.86:0/2545510316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4104dd0 msgr2=0x7f19a41051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2545510316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4104dd0 0x7f19a41051b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19940099c0 tx=0x7f1994009cd0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 -- 172.21.15.86:0/2545510316 shutdown_connections 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2545510316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4109eb0 0x7f19a4111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2545510316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4104dd0 0x7f19a41051b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2545510316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 0x7f19a41097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.724+0000 7f19a8e00700 1 -- 172.21.15.86:0/2545510316 >> 172.21.15.86:0/2545510316 conn(0x7f19a4100660 msgr2=0x7f19a4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.725+0000 7f19a8e00700 1 -- 172.21.15.86:0/2545510316 shutdown_connections 2024-04-25T19:01:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.725+0000 7f19a8e00700 1 -- 172.21.15.86:0/2545510316 wait complete. 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 Processor -- start 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 -- start start 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4104dd0 0x7f19a419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 0x7f19a419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 0x7f19a419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4114d50 con 0x7f19a4109eb0 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4114bd0 con 0x7f19a4104dd0 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.726+0000 7f19a8e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4114ed0 con 0x7f19a4105780 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 0x7f19a419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a2a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 0x7f19a419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 0x7f19a419fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34648/0 (socket says 172.21.15.86:34648) 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 -- 172.21.15.86:0/2443580869 learned_addr learned my addr 172.21.15.86:0/2443580869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3400700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4104dd0 0x7f19a419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 -- 172.21.15.86:0/2443580869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 msgr2=0x7f19a419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 0x7f19a419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 -- 172.21.15.86:0/2443580869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4104dd0 msgr2=0x7f19a419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4104dd0 0x7f19a419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 -- 172.21.15.86:0/2443580869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a41a0200 con 0x7f19a4109eb0 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a2a00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 0x7f19a419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3400700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4104dd0 0x7f19a419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.727+0000 7f19a3e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 0x7f19a419fac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f199c00ef30 tx=0x7f199c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.728+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c0092e0 con 0x7f19a4109eb0 2024-04-25T19:01:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.728+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c004bb0 con 0x7f19a4109eb0 2024-04-25T19:01:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.728+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c004020 con 0x7f19a4109eb0 2024-04-25T19:01:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.728+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a41a0490 con 0x7f19a4109eb0 2024-04-25T19:01:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.728+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a41a7d70 con 0x7f19a4109eb0 2024-04-25T19:01:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.730+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a4069730 con 0x7f19a4109eb0 2024-04-25T19:01:27.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.731+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f199c01b030 con 0x7f19a4109eb0 2024-04-25T19:01:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.733+0000 7f199be00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f19880a3cc0 0x7f19880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.733+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f199c0cfe70 con 0x7f19a4109eb0 2024-04-25T19:01:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.733+0000 7f19a3400700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f19880a3cc0 0x7f19880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.734+0000 7f19a3400700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f19880a3cc0 0x7f19880a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1994003760 tx=0x7f1994005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.737+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f199c022020 con 0x7f19a4109eb0 2024-04-25T19:01:27.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.967+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19a402df80 con 0x7f19a4109eb0 2024-04-25T19:01:27.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.969+0000 7f199be00700 1 -- 172.21.15.86:0/2443580869 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22867 (secure 0 0 0) 0x7f199c010040 con 0x7f19a4109eb0 2024-04-25T19:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T19:00:38.512884+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:01:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f19880a3cc0 msgr2=0x7f19880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f19880a3cc0 0x7f19880a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1994003760 tx=0x7f1994005fb0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 msgr2=0x7f19a419fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 0x7f19a419fac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f199c00ef30 tx=0x7f199c00c3a0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 shutdown_connections 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f19880a3cc0 0x7f19880a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.972+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19a4109eb0 0x7f19a419fac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.973+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19a4104dd0 0x7f19a419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.973+0000 7f19a8e00700 1 --2- 172.21.15.86:0/2443580869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a4105780 0x7f19a419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.973+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 >> 172.21.15.86:0/2443580869 conn(0x7f19a4100660 msgr2=0x7f19a4109040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.973+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 shutdown_connections 2024-04-25T19:01:27.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:27.973+0000 7f19a8e00700 1 -- 172.21.15.86:0/2443580869 wait complete. 2024-04-25T19:01:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:28 smithi194 ceph-mon[32740]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:28 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2443580869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:01:28.792 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:28 smithi086 ceph-mon[27783]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:28.792 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:28 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2443580869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:01:28.866 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:01:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:28 smithi183 ceph-mon[32344]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:28 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2443580869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:01:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:30 smithi086 ceph-mon[27783]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:30 smithi183 ceph-mon[32344]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:30 smithi194 ceph-mon[32740]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.523+0000 7f1bd8a00700 1 -- 172.21.15.86:0/34223348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 msgr2=0x7f1bd4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.523+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/34223348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 0x7f1bd4102f70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009990 tx=0x7f1bc0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.523+0000 7f1bd8a00700 1 -- 172.21.15.86:0/34223348 shutdown_connections 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.524+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/34223348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 0x7f1bd410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.524+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/34223348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.524+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/34223348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 0x7f1bd4102f70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.524+0000 7f1bd8a00700 1 -- 172.21.15.86:0/34223348 >> 172.21.15.86:0/34223348 conn(0x7f1bd4079ba0 msgr2=0x7f1bd4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.524+0000 7f1bd8a00700 1 -- 172.21.15.86:0/34223348 shutdown_connections 2024-04-25T19:01:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.524+0000 7f1bd8a00700 1 -- 172.21.15.86:0/34223348 wait complete. 2024-04-25T19:01:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.525+0000 7f1bd8a00700 1 Processor -- start 2024-04-25T19:01:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.525+0000 7f1bd8a00700 1 -- start start 2024-04-25T19:01:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd8a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 0x7f1bd419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 0x7f1bd419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd8a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4112c30 con 0x7f1bd41034b0 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd8a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4112ab0 con 0x7f1bd4107940 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd8a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4112db0 con 0x7f1bd4102af0 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 0x7f1bd419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55384/0 (socket says 172.21.15.86:55384) 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 0x7f1bd419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44924/0 (socket says 172.21.15.86:44924) 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 -- 172.21.15.86:0/769874635 learned_addr learned my addr 172.21.15.86:0/769874635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd1e00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 0x7f1bd419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 -- 172.21.15.86:0/769874635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 msgr2=0x7f1bd419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 0x7f1bd419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 -- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 msgr2=0x7f1bd419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 0x7f1bd419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.526+0000 7f1bd2800700 1 -- 172.21.15.86:0/769874635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd41a02b0 con 0x7f1bd4107940 2024-04-25T19:01:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.527+0000 7f1bd2800700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd419fb70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00efc0 tx=0x7f1bcc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.527+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0092e0 con 0x7f1bd4107940 2024-04-25T19:01:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.527+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd41a0540 con 0x7f1bd4107940 2024-04-25T19:01:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.527+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc004bb0 con 0x7f1bd4107940 2024-04-25T19:01:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.527+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc004020 con 0x7f1bd4107940 2024-04-25T19:01:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.527+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd41a7e20 con 0x7f1bd4107940 2024-04-25T19:01:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.528+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd4069730 con 0x7f1bd4107940 2024-04-25T19:01:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.530+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1bcc01b030 con 0x7f1bd4107940 2024-04-25T19:01:31.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.532+0000 7f1bcaa00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1bb80a3c40 0x7f1bb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:31.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.532+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f1bcc0cfd70 con 0x7f1bd4107940 2024-04-25T19:01:31.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.532+0000 7f1bd1e00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1bb80a3c40 0x7f1bb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.532+0000 7f1bd1e00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1bb80a3c40 0x7f1bb80a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009be0 tx=0x7f1bc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.536+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bcc022020 con 0x7f1bd4107940 2024-04-25T19:01:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.812+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bd41a7ff0 con 0x7f1bd4107940 2024-04-25T19:01:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.813+0000 7f1bcaa00700 1 -- 172.21.15.86:0/769874635 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11937 (secure 0 0 0) 0x7f1bcc010040 con 0x7f1bd4107940 2024-04-25T19:01:31.815 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:01:31.815 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T18:59:15.320078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24478,24527,24493,24494,24509]},"id":1}]} 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1bb80a3c40 msgr2=0x7f1bb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1bb80a3c40 0x7f1bb80a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009be0 tx=0x7f1bc4009260 comp rx=0 tx=0).stop 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 msgr2=0x7f1bd419fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd419fb70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00efc0 tx=0x7f1bcc00c3a0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 shutdown_connections 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1bb80a3c40 0x7f1bb80a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bd4107940 0x7f1bd419fb70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.817+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd4102af0 0x7f1bd419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.818+0000 7f1bd8a00700 1 --2- 172.21.15.86:0/769874635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bd41034b0 0x7f1bd419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.818+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 >> 172.21.15.86:0/769874635 conn(0x7f1bd4079ba0 msgr2=0x7f1bd4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.818+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 shutdown_connections 2024-04-25T19:01:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:31.818+0000 7f1bd8a00700 1 -- 172.21.15.86:0/769874635 wait complete. 2024-04-25T19:01:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T19:01:32.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:58:51.446663+0000', 'modified': '2024-04-25T18:59:15.320078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24509, 'mds_2': 24494, 'mds_3': 24493, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/45783241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 45783241}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 45783241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2135214296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2135214296}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2135214296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/3883478130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3883478130}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3883478130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/425079996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 425079996}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 425079996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/4174239962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 4174239962}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 4174239962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.194:6837/3328875881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3328875881}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3328875881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.194:6839/2085744336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 2085744336}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 2085744336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.183:6837/3273542947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3273542947}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3273542947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.86:6839/3676043902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3676043902}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3676043902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.183:6839/703487979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 703487979}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 703487979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24478, 24527, 24493, 24494, 24509]} 2024-04-25T19:01:32.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T19:01:32.488 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-25T19:01:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:32 smithi086 ceph-mon[27783]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:32 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/769874635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:32 smithi183 ceph-mon[32344]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:32 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/769874635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:32.929 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:32 smithi194 ceph-mon[32740]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:32.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:32 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/769874635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:34.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:34 smithi086 ceph-mon[27783]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:34.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:34 smithi183 ceph-mon[32344]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:34.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:34 smithi194 ceph-mon[32740]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:35.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 -- 172.21.15.86:0/3267665315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 msgr2=0x7f2c84102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 --2- 172.21.15.86:0/3267665315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 0x7f2c84102ef0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009990 tx=0x7f2c6c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 -- 172.21.15.86:0/3267665315 shutdown_connections 2024-04-25T19:01:35.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 --2- 172.21.15.86:0/3267665315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 0x7f2c8410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 --2- 172.21.15.86:0/3267665315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c84107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 --2- 172.21.15.86:0/3267665315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 0x7f2c84102ef0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.281+0000 7f2c88c00700 1 -- 172.21.15.86:0/3267665315 >> 172.21.15.86:0/3267665315 conn(0x7f2c84079bf0 msgr2=0x7f2c8407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.282+0000 7f2c88c00700 1 -- 172.21.15.86:0/3267665315 shutdown_connections 2024-04-25T19:01:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.282+0000 7f2c88c00700 1 -- 172.21.15.86:0/3267665315 wait complete. 2024-04-25T19:01:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.282+0000 7f2c88c00700 1 Processor -- start 2024-04-25T19:01:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 -- start start 2024-04-25T19:01:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 0x7f2c84199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 0x7f2c841996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c8419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112c10 con 0x7f2c84107930 2024-04-25T19:01:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112a90 con 0x7f2c84102a90 2024-04-25T19:01:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c88c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112d90 con 0x7f2c84103430 2024-04-25T19:01:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c82800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c8419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c82800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c8419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44952/0 (socket says 172.21.15.86:44952) 2024-04-25T19:01:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c82800700 1 -- 172.21.15.86:0/1435822943 learned_addr learned my addr 172.21.15.86:0/1435822943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c7be00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 0x7f2c841996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.283+0000 7f2c81e00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 0x7f2c84199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c82800700 1 -- 172.21.15.86:0/1435822943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 msgr2=0x7f2c841996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c82800700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 0x7f2c841996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c82800700 1 -- 172.21.15.86:0/1435822943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 msgr2=0x7f2c84199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c82800700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 0x7f2c84199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c82800700 1 -- 172.21.15.86:0/1435822943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8419e190 con 0x7f2c84107930 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c7be00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 0x7f2c841996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c82800700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c8419da50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c370 tx=0x7f2c7c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c7aa00700 1 -- 172.21.15.86:0/1435822943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c0092e0 con 0x7f2c84107930 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8419e420 con 0x7f2c84107930 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.284+0000 7f2c7aa00700 1 -- 172.21.15.86:0/1435822943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c003a00 con 0x7f2c84107930 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.285+0000 7f2c7aa00700 1 -- 172.21.15.86:0/1435822943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c004690 con 0x7f2c84107930 2024-04-25T19:01:35.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.285+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a5d00 con 0x7f2c84107930 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.288+0000 7f2c7aa00700 1 -- 172.21.15.86:0/1435822943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2c7c01a020 con 0x7f2c84107930 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.289+0000 7f2c7aa00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2c680a3c40 0x7f2c680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.289+0000 7f2c934c1b80 1 Processor -- start 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c7aa00700 1 -- 172.21.15.86:0/1435822943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f2c7c0cfd10 con 0x7f2c84107930 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 -- start start 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c81e00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2c680a3c40 0x7f2c680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e81380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e818c0 0x558303e856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e85c00 0x558303e86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558303dcee40 con 0x558303e80fa0 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558303d756e0 con 0x558303e85c00 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558303dcae00 con 0x558303e818c0 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e81380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e81380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44966/0 (socket says 172.21.15.86:44966) 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.290+0000 7f2c81e00700 1 -- 172.21.15.86:0/3224589085 learned_addr learned my addr 172.21.15.86:0/3224589085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c82800700 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e85c00 0x558303e86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c7be00700 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e818c0 0x558303e856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2c680a3c40 0x7f2c680a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009680 tx=0x7f2c74011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 -- 172.21.15.86:0/3224589085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e818c0 msgr2=0x558303e856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e818c0 0x558303e856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 -- 172.21.15.86:0/3224589085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e85c00 msgr2=0x558303e86060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e85c00 0x558303e86060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 -- 172.21.15.86:0/3224589085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558303d6b630 con 0x558303e80fa0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e81380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c74011e80 tx=0x7f2c7401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a470c0cd3b43581f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.291+0000 7f2c73400700 1 -- 172.21.15.86:0/3224589085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74017040 con 0x558303e80fa0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c73400700 1 -- 172.21.15.86:0/3224589085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c74019050 con 0x558303e80fa0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c73400700 1 -- 172.21.15.86:0/3224589085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400e610 con 0x558303e80fa0 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3224589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 msgr2=0x558303e81380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e81380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c74011e80 tx=0x7f2c7401c000 comp rx=0 tx=0).stop 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3224589085 shutdown_connections 2024-04-25T19:01:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e85c00 0x558303e86060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e81380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3224589085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e818c0 0x558303e856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.292+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3224589085 >> 172.21.15.86:0/3224589085 conn(0x558303daaf00 msgr2=0x558303e778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.293+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3224589085 shutdown_connections 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.293+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3224589085 wait complete. 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.293+0000 7f2c934c1b80 1 -- start start 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.293+0000 7f2c934c1b80 1 Processor -- start 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e80fa0 0x558303e8d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 0x558303e8a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558303e8dd30 con 0x558303e818c0 2024-04-25T19:01:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558303d88810 con 0x558303e80fa0 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558303e8deb0 con 0x558303e85c00 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c934c1b80 10 client.0 _ll_register_callbacks cb 0x558303d676d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 0x558303e8a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 0x558303e8a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44978/0 (socket says 172.21.15.86:44978) 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c7be00700 1 -- 172.21.15.86:0/3115458731 learned_addr learned my addr 172.21.15.86:0/3115458731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e80fa0 0x558303e8d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.294+0000 7f2c82800700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c7be00700 1 -- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 msgr2=0x558303e8a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c7be00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c7be00700 1 -- 172.21.15.86:0/3115458731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e80fa0 msgr2=0x558303e8d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c7be00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e80fa0 0x558303e8d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c7be00700 1 -- 172.21.15.86:0/3115458731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558303d1c880 con 0x558303e818c0 2024-04-25T19:01:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e80fa0 0x558303e8d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c7be00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 0x558303e8a3b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c015000 tx=0x7f2c6c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43aa6bcfc4aa5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c67e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:01:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c67e00700 1 -- 172.21.15.86:0/3115458731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c003880 con 0x558303e818c0 2024-04-25T19:01:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c67e00700 1 -- 172.21.15.86:0/3115458731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f2c6c003a20 con 0x558303e818c0 2024-04-25T19:01:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.295+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c74019050 con 0x558303e818c0 2024-04-25T19:01:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c67e00700 1 -- 172.21.15.86:0/3115458731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c6c0056d0 con 0x558303e818c0 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 10 client.14745 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558303d65aa0 con 0x558303e818c0 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c67e00700 1 -- 172.21.15.86:0/3115458731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c74019050 con 0x558303e818c0 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c67e00700 1 -- 172.21.15.86:0/3115458731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2c6c00fbb0 con 0x558303e818c0 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 10 client.14745 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 10 client.14745 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 20 client.14745 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303e9aa90 0x558303e9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 4 client.14745 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.296+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 --> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558303ddc630 con 0x558303e9aa90 2024-04-25T19:01:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.297+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303e9aa90 0x558303e9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.297+0000 7f2c67e00700 10 client.14745 ms_handle_connect on v2:172.21.15.194:6836/3328875881 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.297+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303e9aa90 0x558303e9bf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=813015cbf30fbf61 server_cookie=6c235423d6467593 in_seq=0 out_seq=0 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.298+0000 7f2c67e00700 1 -- 172.21.15.86:0/3115458731 <== mds.4 v2:172.21.15.194:6836/3328875881 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558303ddc630 con 0x558303e9aa90 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.298+0000 7f2c67e00700 10 client.14745 handle_command_reply: tid=0 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.298+0000 7f2c934c1b80 1 client.14745 shutdown 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 msgr2=0x558303e8a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 0x558303e8a3b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c015000 tx=0x7f2c6c005bc0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 shutdown_connections 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303e9aa90 0x558303e9bf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c67e00700 0 client.14745 ms_handle_reset on v2:172.21.15.194:6836/3328875881 2024-04-25T19:01:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e80fa0 0x558303e8d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3115458731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e818c0 0x558303e8a3b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 >> 172.21.15.86:0/3115458731 conn(0x558303daaf00 msgr2=0x558303e76e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:35.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 shutdown_connections 2024-04-25T19:01:35.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3115458731 wait complete. 2024-04-25T19:01:35.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.299+0000 7f2c934c1b80 20 client.14745 trim_cache size 0 max 16384 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.321+0000 7f2c934c1b80 1 Processor -- start 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.321+0000 7f2c934c1b80 1 -- start start 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.321+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e88030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.321+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303e775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558303e9a240 con 0x558303e80fa0 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558303e9a910 con 0x558303e818c0 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558303e9afe0 con 0x558303e85c00 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e88030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44992/0 (socket says 172.21.15.86:44992) 2024-04-25T19:01:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 -- 172.21.15.86:0/3578848365 learned_addr learned my addr 172.21.15.86:0/3578848365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c7be00700 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303e775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c82800700 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 -- 172.21.15.86:0/3578848365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 msgr2=0x558303e8a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 -- 172.21.15.86:0/3578848365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 msgr2=0x558303e775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303e775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c81e00700 1 -- 172.21.15.86:0/3578848365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558303e81ef0 con 0x558303e80fa0 2024-04-25T19:01:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.322+0000 7f2c7be00700 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303e775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.323+0000 7f2c81e00700 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e88030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2c7401da00 tx=0x7f2c7401dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db71f06cbcfab818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.323+0000 7f2c72a00700 1 -- 172.21.15.86:0/3578848365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74017040 con 0x558303e80fa0 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.323+0000 7f2c72a00700 1 -- 172.21.15.86:0/3578848365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7402fdc0 con 0x558303e80fa0 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.323+0000 7f2c72a00700 1 -- 172.21.15.86:0/3578848365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400f3c0 con 0x558303e80fa0 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3578848365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 msgr2=0x558303e88030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e88030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2c7401da00 tx=0x7f2c7401dbb0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3578848365 shutdown_connections 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e8a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303e88030 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/3578848365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303e775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3578848365 >> 172.21.15.86:0/3578848365 conn(0x558303daaf00 msgr2=0x558303e867d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3578848365 shutdown_connections 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 -- 172.21.15.86:0/3578848365 wait complete. 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.324+0000 7f2c934c1b80 1 -- start start 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 Processor -- start 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303ea5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303ea58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e76f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558303e9bd80 con 0x558303e80fa0 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558303e9b6b0 con 0x558303e818c0 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558303e9c450 con 0x558303e85c00 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c934c1b80 10 client.0 _ll_register_callbacks cb 0x558303d676d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303ea5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c81e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303ea5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45008/0 (socket says 172.21.15.86:45008) 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c81e00700 1 -- 172.21.15.86:0/1173814750 learned_addr learned my addr 172.21.15.86:0/1173814750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.325+0000 7f2c7be00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303ea58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c82800700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e76f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c81e00700 1 -- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 msgr2=0x558303e76f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c81e00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e76f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c81e00700 1 -- 172.21.15.86:0/1173814750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 msgr2=0x558303ea58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c81e00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303ea58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c81e00700 1 -- 172.21.15.86:0/1173814750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558303e77480 con 0x558303e80fa0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c7be00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303ea58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c81e00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303ea5370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2c740099d0 tx=0x7f2c74005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827d9a10fe278d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c67e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c67e00700 1 -- 172.21.15.86:0/1173814750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74017040 con 0x558303e80fa0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c67e00700 1 -- 172.21.15.86:0/1173814750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f2c7400f3c0 con 0x558303e80fa0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.326+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7402fdc0 con 0x558303e80fa0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c67e00700 1 -- 172.21.15.86:0/1173814750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c7400f5c0 con 0x558303e80fa0 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 10 client.14751 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:35.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558303f02400 con 0x558303e80fa0 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c67e00700 1 -- 172.21.15.86:0/1173814750 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7402fdc0 con 0x558303e80fa0 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c67e00700 1 -- 172.21.15.86:0/1173814750 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2c7402f3a0 con 0x558303e80fa0 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 10 client.14751 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 10 client.14751 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 20 client.14751 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303f068d0 0x558303f055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 4 client.14751 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.327+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 --> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] -- command(tid 0: {"prefix": "status"}) -- 0x558303dd66d0 con 0x558303f068d0 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.328+0000 7f2c7be00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303f068d0 0x558303f055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.328+0000 7f2c7be00700 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303f068d0 0x558303f055e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc9fde0b54d8e07f server_cookie=8fb75e446040ac43 in_seq=0 out_seq=0 2024-04-25T19:01:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.328+0000 7f2c67e00700 10 client.14751 ms_handle_connect on v2:172.21.15.194:6836/3328875881 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c67e00700 1 -- 172.21.15.86:0/1173814750 <== mds.4 v2:172.21.15.194:6836/3328875881 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558303dd66d0 con 0x558303f068d0 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c67e00700 10 client.14751 handle_command_reply: tid=0 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 client.14751 shutdown 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 msgr2=0x558303ea5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303ea5370 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2c740099d0 tx=0x7f2c74005220 comp rx=0 tx=0).stop 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 shutdown_connections 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x558303f068d0 0x558303f055e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558303e85c00 0x558303e76f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558303e80fa0 0x558303ea5370 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 --2- 172.21.15.86:0/1173814750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558303e818c0 0x558303ea58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 >> 172.21.15.86:0/1173814750 conn(0x558303daaf00 msgr2=0x558303e8a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c67e00700 0 client.14751 ms_handle_reset on v2:172.21.15.194:6836/3328875881 2024-04-25T19:01:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 shutdown_connections 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 1 -- 172.21.15.86:0/1173814750 wait complete. 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.329+0000 7f2c934c1b80 20 client.14751 trim_cache size 0 max 16384 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24478, 2024-04-25T19:01:35.338 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 144.84949555, 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 203.43388364699999 2024-04-25T19:01:35.339 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2c680a3c40 msgr2=0x7f2c680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2c680a3c40 0x7f2c680a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009680 tx=0x7f2c74011440 comp rx=0 tx=0).stop 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 msgr2=0x7f2c8419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c8419da50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c370 tx=0x7f2c7c00c680 comp rx=0 tx=0).stop 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 shutdown_connections 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2c680a3c40 0x7f2c680a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84107930 0x7f2c8419da50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c84102a90 0x7f2c84199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 --2- 172.21.15.86:0/1435822943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c84103430 0x7f2c841996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:35.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 >> 172.21.15.86:0/1435822943 conn(0x7f2c84079bf0 msgr2=0x7f2c84100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:35.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 shutdown_connections 2024-04-25T19:01:35.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:35.330+0000 7f2c88c00700 1 -- 172.21.15.86:0/1435822943 wait complete. 2024-04-25T19:01:36.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T19:01:36.643 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:36.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:36 smithi086 ceph-mon[27783]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:36 smithi183 ceph-mon[32344]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:36 smithi194 ceph-mon[32740]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:38.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:38 smithi086 ceph-mon[27783]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 -- 172.21.15.86:0/1897640255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4301092d0 msgr2=0x7fc4301096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 --2- 172.21.15.86:0/1897640255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4301092d0 0x7fc4301096b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4240099c0 tx=0x7fc424009cd0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 -- 172.21.15.86:0/1897640255 shutdown_connections 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 --2- 172.21.15.86:0/1897640255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc43010e3b0 0x7fc430115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 --2- 172.21.15.86:0/1897640255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4301092d0 0x7fc4301096b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 --2- 172.21.15.86:0/1897640255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc430109c80 0x7fc43010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 -- 172.21.15.86:0/1897640255 >> 172.21.15.86:0/1897640255 conn(0x7fc430076210 msgr2=0x7fc430107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 -- 172.21.15.86:0/1897640255 shutdown_connections 2024-04-25T19:01:38.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.909+0000 7fc438600700 1 -- 172.21.15.86:0/1897640255 wait complete. 2024-04-25T19:01:38.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.910+0000 7fc438600700 1 Processor -- start 2024-04-25T19:01:38.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 -- start start 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4301092d0 0x7fc4301a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 0x7fc4301a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43010e3b0 0x7fc43019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301192b0 con 0x7fc430109c80 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc430119130 con 0x7fc4301092d0 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc438600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc430119430 con 0x7fc43010e3b0 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc435200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 0x7fc4301a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc435200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 0x7fc4301a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45014/0 (socket says 172.21.15.86:45014) 2024-04-25T19:01:38.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.911+0000 7fc435200700 1 -- 172.21.15.86:0/2099984478 learned_addr learned my addr 172.21.15.86:0/2099984478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:38.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435c00700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4301092d0 0x7fc4301a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc436600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43010e3b0 0x7fc43019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435200700 1 -- 172.21.15.86:0/2099984478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43010e3b0 msgr2=0x7fc43019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435200700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43010e3b0 0x7fc43019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435200700 1 -- 172.21.15.86:0/2099984478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4301092d0 msgr2=0x7fc4301a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435200700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4301092d0 0x7fc4301a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435200700 1 -- 172.21.15.86:0/2099984478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43019fd70 con 0x7fc430109c80 2024-04-25T19:01:38.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc436600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43010e3b0 0x7fc43019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:38.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.912+0000 7fc435200700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 0x7fc4301a5940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00d950 tx=0x7fc41c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:38.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.913+0000 7fc42e000700 1 -- 172.21.15.86:0/2099984478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c017070 con 0x7fc430109c80 2024-04-25T19:01:38.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.913+0000 7fc42e000700 1 -- 172.21.15.86:0/2099984478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c005020 con 0x7fc430109c80 2024-04-25T19:01:38.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.913+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4301a0060 con 0x7fc430109c80 2024-04-25T19:01:38.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:38 smithi183 ceph-mon[32344]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:38.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.913+0000 7fc42e000700 1 -- 172.21.15.86:0/2099984478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c009ab0 con 0x7fc430109c80 2024-04-25T19:01:38.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.913+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4301ac2d0 con 0x7fc430109c80 2024-04-25T19:01:38.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.916+0000 7fc42e000700 1 -- 172.21.15.86:0/2099984478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc41c009c50 con 0x7fc430109c80 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc42e000700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4180a3c40 0x7fc4180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc442eb3b80 1 Processor -- start 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc42e000700 1 -- 172.21.15.86:0/2099984478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc41c0cfd70 con 0x7fc430109c80 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc442eb3b80 1 -- start start 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc435c00700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4180a3c40 0x7fc4180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e02087b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e020880a0 0x559e020819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.918+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e02082330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e01fd3e40 con 0x559e02081ef0 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e01f7a6e0 con 0x559e02087780 2024-04-25T19:01:38.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559e01fcfe00 con 0x559e020880a0 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e02082330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e02082330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45018/0 (socket says 172.21.15.86:45018) 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 -- 172.21.15.86:0/4252338764 learned_addr learned my addr 172.21.15.86:0/4252338764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc435200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e020880a0 0x559e020819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc435c00700 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e02087b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 -- 172.21.15.86:0/4252338764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e020880a0 msgr2=0x559e020819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e020880a0 0x559e020819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 -- 172.21.15.86:0/4252338764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 msgr2=0x559e02087b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e02087b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc436600700 1 -- 172.21.15.86:0/4252338764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e01f70630 con 0x559e02081ef0 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.919+0000 7fc435c00700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4180a3c40 0x7fc4180a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc424009990 tx=0x7fc42400b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:38.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.920+0000 7fc436600700 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e02082330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc42800e160 tx=0x7fc42800e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75feb0697021342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:38.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.920+0000 7fc422a00700 1 -- 172.21.15.86:0/4252338764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42800ed00 con 0x559e02081ef0 2024-04-25T19:01:38.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.920+0000 7fc422a00700 1 -- 172.21.15.86:0/4252338764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc428004bb0 con 0x559e02081ef0 2024-04-25T19:01:38.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.920+0000 7fc422a00700 1 -- 172.21.15.86:0/4252338764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428004430 con 0x559e02081ef0 2024-04-25T19:01:38.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 -- 172.21.15.86:0/4252338764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 msgr2=0x559e02082330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e02082330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc42800e160 tx=0x7fc42800e470 comp rx=0 tx=0).stop 2024-04-25T19:01:38.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 -- 172.21.15.86:0/4252338764 shutdown_connections 2024-04-25T19:01:38.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e02082330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e02087b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/4252338764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e020880a0 0x559e020819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 -- 172.21.15.86:0/4252338764 >> 172.21.15.86:0/4252338764 conn(0x559e01faff00 msgr2=0x559e0207c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 -- 172.21.15.86:0/4252338764 shutdown_connections 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.921+0000 7fc442eb3b80 1 -- 172.21.15.86:0/4252338764 wait complete. 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.922+0000 7fc442eb3b80 1 -- start start 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.922+0000 7fc442eb3b80 1 Processor -- start 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.922+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e0208dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e02087780 0x559e0208af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e0208b450 0x559e0208b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e0208e920 con 0x559e02081ef0 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e01f8d810 con 0x559e0208b450 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559e0208eaa0 con 0x559e02087780 2024-04-25T19:01:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc442eb3b80 10 client.0 _ll_register_callbacks cb 0x559e01f6c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e0208dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e0208dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45026/0 (socket says 172.21.15.86:45026) 2024-04-25T19:01:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435c00700 1 -- 172.21.15.86:0/3838381134 learned_addr learned my addr 172.21.15.86:0/3838381134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc436600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e0208b450 0x559e0208b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435200700 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e02087780 0x559e0208af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435200700 1 -- 172.21.15.86:0/3838381134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e0208b450 msgr2=0x559e0208b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435200700 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e0208b450 0x559e0208b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435200700 1 -- 172.21.15.86:0/3838381134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 msgr2=0x559e0208dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435200700 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e0208dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.923+0000 7fc435200700 1 -- 172.21.15.86:0/3838381134 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e01f21880 con 0x559e02087780 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc435200700 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e02087780 0x559e0208af10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc41c09bf20 tx=0x7fc41c09c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37257bb81f293e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc417400700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc417400700 1 -- 172.21.15.86:0/3838381134 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0b4020 con 0x559e02087780 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc417400700 1 -- 172.21.15.86:0/3838381134 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc41c005480 con 0x559e02087780 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc428004bb0 con 0x559e02087780 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc417400700 1 -- 172.21.15.86:0/3838381134 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc41c020040 con 0x559e02087780 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc442eb3b80 10 client.24886 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e01f6aaa0 con 0x559e02087780 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.924+0000 7fc417400700 1 -- 172.21.15.86:0/3838381134 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc428004bb0 con 0x559e02087780 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc417400700 1 -- 172.21.15.86:0/3838381134 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc41c0b5220 con 0x559e02087780 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc442eb3b80 10 client.24886 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc442eb3b80 10 client.24886 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc442eb3b80 20 client.24886 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e0209b960 0x559e0209cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc442eb3b80 4 client.24886 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 --> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e01fe1630 con 0x559e0209b960 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.925+0000 7fc435c00700 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e0209b960 0x559e0209cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.926+0000 7fc435c00700 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e0209b960 0x559e0209cdd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d356c55efe1813bb server_cookie=92ac7a502874ad1e in_seq=0 out_seq=0 2024-04-25T19:01:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.926+0000 7fc417400700 10 client.24886 ms_handle_connect on v2:172.21.15.194:6838/2085744336 2024-04-25T19:01:38.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc417400700 1 -- 172.21.15.86:0/3838381134 <== mds.3 v2:172.21.15.194:6838/2085744336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e01fe1630 con 0x559e0209b960 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc417400700 10 client.24886 handle_command_reply: tid=0 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 client.24886 shutdown 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e02087780 msgr2=0x559e0208af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e02087780 0x559e0208af10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc41c09bf20 tx=0x7fc41c09c6e0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 shutdown_connections 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e0209b960 0x559e0209cdd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e0208b450 0x559e0208b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e0208dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3838381134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e02087780 0x559e0208af10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 >> 172.21.15.86:0/3838381134 conn(0x559e01faff00 msgr2=0x559e0207be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc417400700 0 client.24886 ms_handle_reset on v2:172.21.15.194:6838/2085744336 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 shutdown_connections 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.927+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3838381134 wait complete. 2024-04-25T19:01:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.928+0000 7fc442eb3b80 20 client.24886 trim_cache size 0 max 16384 2024-04-25T19:01:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.949+0000 7fc442eb3b80 1 Processor -- start 2024-04-25T19:01:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.949+0000 7fc442eb3b80 1 -- start start 2024-04-25T19:01:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02081ef0 0x559e020819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 0x559e02097d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e0207c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e0209b110 con 0x559e02087780 2024-04-25T19:01:38.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e0209b7e0 con 0x559e02081ef0 2024-04-25T19:01:38.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559e0209beb0 con 0x559e0208b450 2024-04-25T19:01:38.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc435200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 0x559e02097d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc435200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 0x559e02097d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45040/0 (socket says 172.21.15.86:45040) 2024-04-25T19:01:38.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc435200700 1 -- 172.21.15.86:0/2425211643 learned_addr learned my addr 172.21.15.86:0/2425211643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:38.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc435c00700 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02081ef0 0x559e020819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc436600700 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e0207c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc435200700 1 -- 172.21.15.86:0/2425211643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 msgr2=0x559e0207c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.950+0000 7fc435200700 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e0207c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc435200700 1 -- 172.21.15.86:0/2425211643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02081ef0 msgr2=0x559e020819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc435200700 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02081ef0 0x559e020819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc435200700 1 -- 172.21.15.86:0/2425211643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0208d800 con 0x559e02087780 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc436600700 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e0207c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc435200700 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 0x559e02097d70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b5680 tx=0x7fc41c09a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c0bb7b53c030d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc422000700 1 -- 172.21.15.86:0/2425211643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0b4070 con 0x559e02087780 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc422000700 1 -- 172.21.15.86:0/2425211643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0a49c0 con 0x559e02087780 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.951+0000 7fc422000700 1 -- 172.21.15.86:0/2425211643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c005480 con 0x559e02087780 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 -- 172.21.15.86:0/2425211643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 msgr2=0x559e02097d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 0x559e02097d70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b5680 tx=0x7fc41c09a4a0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 -- 172.21.15.86:0/2425211643 shutdown_connections 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e0207c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02081ef0 0x559e020819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/2425211643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02087780 0x559e02097d70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 -- 172.21.15.86:0/2425211643 >> 172.21.15.86:0/2425211643 conn(0x559e01faff00 msgr2=0x559e02078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 -- 172.21.15.86:0/2425211643 shutdown_connections 2024-04-25T19:01:38.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 -- 172.21.15.86:0/2425211643 wait complete. 2024-04-25T19:01:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.952+0000 7fc442eb3b80 1 -- start start 2024-04-25T19:01:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 Processor -- start 2024-04-25T19:01:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e020a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e020a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e020a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e0209cc50 con 0x559e02081ef0 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e0209c580 con 0x559e02087780 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559e0209d320 con 0x559e0208b450 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.953+0000 7fc442eb3b80 10 client.0 _ll_register_callbacks cb 0x559e01f6c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e020a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc435c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e020a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc435200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e020a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e020a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:57860/0 (socket says 172.21.15.86:57860) 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc435200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e020a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55460/0 (socket says 172.21.15.86:55460) 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 -- 172.21.15.86:0/3398685523 learned_addr learned my addr 172.21.15.86:0/3398685523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:38.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 -- 172.21.15.86:0/3398685523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 msgr2=0x559e020a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e020a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 -- 172.21.15.86:0/3398685523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 msgr2=0x559e020a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e020a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 -- 172.21.15.86:0/3398685523 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e020a1f80 con 0x559e0208b450 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc435200700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e020a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc435c00700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e020a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc436600700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e020a1740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc428007f50 tx=0x7fc42800ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe6b0bce52188a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.954+0000 7fc417400700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc417400700 1 -- 172.21.15.86:0/3398685523 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428024030 con 0x559e0208b450 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc417400700 1 -- 172.21.15.86:0/3398685523 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc428019040 con 0x559e0208b450 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc41c0a49c0 con 0x559e0208b450 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc417400700 1 -- 172.21.15.86:0/3398685523 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc428003c90 con 0x559e0208b450 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc442eb3b80 10 client.24892 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e02103aa0 con 0x559e0208b450 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc417400700 1 -- 172.21.15.86:0/3398685523 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c0a49c0 con 0x559e0208b450 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.955+0000 7fc417400700 1 -- 172.21.15.86:0/3398685523 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc428014480 con 0x559e0208b450 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc442eb3b80 10 client.24892 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc442eb3b80 10 client.24892 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc442eb3b80 20 client.24892 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e02108560 0x559e02107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc442eb3b80 4 client.24892 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 --> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] -- command(tid 0: {"prefix": "status"}) -- 0x559e01fdb6d0 con 0x559e02108560 2024-04-25T19:01:38.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.956+0000 7fc435c00700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e02108560 0x559e02107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc435c00700 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e02108560 0x559e02107270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac5d19c7771d4829 server_cookie=6d27e82ef95ae2cc in_seq=0 out_seq=0 2024-04-25T19:01:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc417400700 10 client.24892 ms_handle_connect on v2:172.21.15.194:6838/2085744336 2024-04-25T19:01:38.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc417400700 1 -- 172.21.15.86:0/3398685523 <== mds.3 v2:172.21.15.194:6838/2085744336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559e01fdb6d0 con 0x559e02108560 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc417400700 10 client.24892 handle_command_reply: tid=0 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 client.24892 shutdown 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 msgr2=0x559e020a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e020a1740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc428007f50 tx=0x7fc42800ef30 comp rx=0 tx=0).stop 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 shutdown_connections 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e0208b450 0x559e020a1740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e02081ef0 0x559e020a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x559e02108560 0x559e02107270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 --2- 172.21.15.86:0/3398685523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e02087780 0x559e020a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 >> 172.21.15.86:0/3398685523 conn(0x559e01faff00 msgr2=0x559e02088490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc417400700 0 client.24892 ms_handle_reset on v2:172.21.15.194:6838/2085744336 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 shutdown_connections 2024-04-25T19:01:38.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 1 -- 172.21.15.86:0/3398685523 wait complete. 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.957+0000 7fc442eb3b80 20 client.24892 trim_cache size 0 max 16384 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24493, 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 149.49179264899999, 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:01:38.963 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:01:38.964 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:01:38.964 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 200.99307582899999 2024-04-25T19:01:38.964 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:01:38.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4180a3c40 msgr2=0x7fc4180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4180a3c40 0x7fc4180a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc424009990 tx=0x7fc42400b790 comp rx=0 tx=0).stop 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 msgr2=0x7fc4301a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 0x7fc4301a5940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00d950 tx=0x7fc41c00dc60 comp rx=0 tx=0).stop 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 shutdown_connections 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4180a3c40 0x7fc4180a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc43010e3b0 0x7fc43019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4301092d0 0x7fc4301a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 --2- 172.21.15.86:0/2099984478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc430109c80 0x7fc4301a5940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 >> 172.21.15.86:0/2099984478 conn(0x7fc430076210 msgr2=0x7fc430076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.958+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 shutdown_connections 2024-04-25T19:01:38.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:38.959+0000 7fc438600700 1 -- 172.21.15.86:0/2099984478 wait complete. 2024-04-25T19:01:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:38 smithi194 ceph-mon[32740]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:39.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T19:01:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:40.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:40 smithi086 ceph-mon[27783]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:40.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:40 smithi183 ceph-mon[32344]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:40 smithi194 ceph-mon[32740]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 -- 172.21.15.86:0/92383330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 msgr2=0x7f06c0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 --2- 172.21.15.86:0/92383330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c0108e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009990 tx=0x7f06b0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 -- 172.21.15.86:0/92383330 shutdown_connections 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 --2- 172.21.15.86:0/92383330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006b0f0 0x7f06c010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 --2- 172.21.15.86:0/92383330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c0108e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 --2- 172.21.15.86:0/92383330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c006a750 0x7f06c006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 -- 172.21.15.86:0/92383330 >> 172.21.15.86:0/92383330 conn(0x7f06c00788f0 msgr2=0x7f06c0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 -- 172.21.15.86:0/92383330 shutdown_connections 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.444+0000 7f06c6e00700 1 -- 172.21.15.86:0/92383330 wait complete. 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.445+0000 7f06c6e00700 1 Processor -- start 2024-04-25T19:01:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.445+0000 7f06c6e00700 1 -- start start 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c6e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 0x7f06c019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c6e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c006b0f0 0x7f06c019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c6e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c6e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0112c00 con 0x7f06c0108a30 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c6e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0112a80 con 0x7f06c006b0f0 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c6e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0112d80 con 0x7f06c006a750 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 0x7f06c019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06c4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 0x7f06c019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:40444/0 (socket says 172.21.15.86:40444) 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06bfe00700 1 -- 172.21.15.86:0/111826935 learned_addr learned my addr 172.21.15.86:0/111826935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.446+0000 7f06bf400700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c006b0f0 0x7f06c019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bfe00700 1 -- 172.21.15.86:0/111826935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c006b0f0 msgr2=0x7f06c019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bfe00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c006b0f0 0x7f06c019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bfe00700 1 -- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 msgr2=0x7f06c019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bfe00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bfe00700 1 -- 172.21.15.86:0/111826935 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c01a0390 con 0x7f06c006a750 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06c4e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bfe00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 0x7f06c019b380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005910 tx=0x7f06b000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bcc00700 1 -- 172.21.15.86:0/111826935 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0027070 con 0x7f06c006a750 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c01a0620 con 0x7f06c006a750 2024-04-25T19:01:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bcc00700 1 -- 172.21.15.86:0/111826935 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b0022470 con 0x7f06c006a750 2024-04-25T19:01:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06bcc00700 1 -- 172.21.15.86:0/111826935 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0017650 con 0x7f06c006a750 2024-04-25T19:01:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.447+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01a7f00 con 0x7f06c006a750 2024-04-25T19:01:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.451+0000 7f06bcc00700 1 -- 172.21.15.86:0/111826935 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f06b002a070 con 0x7f06c006a750 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 Processor -- start 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06bcc00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06a40a3cc0 0x7f06a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 -- start start 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06bcc00700 1 -- 172.21.15.86:0/111826935 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f06b00d10b0 con 0x7f06c006a750 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06c4e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06a40a3cc0 0x7f06a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32731b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327320a0 0x55ca3272b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca3272c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca3267de40 con 0x55ca3272bef0 2024-04-25T19:01:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca326246e0 con 0x55ca327320a0 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.452+0000 7f06d185cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca32679e00 con 0x55ca32731780 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32731b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32731b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:40452/0 (socket says 172.21.15.86:40452) 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 -- 172.21.15.86:0/1427433354 learned_addr learned my addr 172.21.15.86:0/1427433354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bf400700 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327320a0 0x55ca3272b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06c4e00700 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca3272c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 -- 172.21.15.86:0/1427433354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327320a0 msgr2=0x55ca3272b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327320a0 0x55ca3272b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 -- 172.21.15.86:0/1427433354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 msgr2=0x55ca3272c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca3272c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 -- 172.21.15.86:0/1427433354 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3261a630 con 0x55ca32731780 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06bfe00700 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32731b60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06b00a5a70 tx=0x7f06b0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c58bc4b7a0141eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.453+0000 7f06c4e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06a40a3cc0 0x7f06a40a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06b8003e80 tx=0x7f06b8009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06b5600700 1 -- 172.21.15.86:0/1427433354 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00a1770 con 0x55ca32731780 2024-04-25T19:01:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06b5600700 1 -- 172.21.15.86:0/1427433354 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00a5fa0 con 0x55ca32731780 2024-04-25T19:01:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06b5600700 1 -- 172.21.15.86:0/1427433354 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00a5b10 con 0x55ca32731780 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 -- 172.21.15.86:0/1427433354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 msgr2=0x55ca32731b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32731b60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06b00a5a70 tx=0x7f06b0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 -- 172.21.15.86:0/1427433354 shutdown_connections 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca3272c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32731b60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1427433354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327320a0 0x55ca3272b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.454+0000 7f06d185cb80 1 -- 172.21.15.86:0/1427433354 >> 172.21.15.86:0/1427433354 conn(0x55ca32659f00 msgr2=0x55ca327268a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.455+0000 7f06d185cb80 1 -- 172.21.15.86:0/1427433354 shutdown_connections 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.455+0000 7f06d185cb80 1 -- 172.21.15.86:0/1427433354 wait complete. 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.455+0000 7f06d185cb80 1 -- start start 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.455+0000 7f06d185cb80 1 Processor -- start 2024-04-25T19:01:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 0x55ca32734f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca32735940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca327386d0 con 0x55ca32731780 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca32637810 con 0x55ca3272bef0 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca32738850 con 0x55ca327354c0 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06d185cb80 10 client.0 _ll_register_callbacks cb 0x55ca326166d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 0x55ca32734f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 0x55ca32734f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59804/0 (socket says 172.21.15.86:59804) 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 -- 172.21.15.86:0/1680690319 learned_addr learned my addr 172.21.15.86:0/1680690319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06c4e00700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca32735940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 -- 172.21.15.86:0/1680690319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 msgr2=0x55ca32735940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca32735940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 -- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca3272bef0 msgr2=0x55ca32737c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bf400700 1 -- 172.21.15.86:0/1680690319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca325cb880 con 0x55ca32731780 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.456+0000 7f06bfe00700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06bf400700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 0x55ca32734f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0127f0 tx=0x7f06ac012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3d944cc2b15763 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06aa000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06aa000700 1 -- 172.21.15.86:0/1680690319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00f4e0 con 0x55ca32731780 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b00a5fa0 con 0x55ca32731780 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06aa000700 1 -- 172.21.15.86:0/1680690319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f06ac00f680 con 0x55ca32731780 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06aa000700 1 -- 172.21.15.86:0/1680690319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06ac00f880 con 0x55ca32731780 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06d185cb80 10 client.14781 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.457+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca32614aa0 con 0x55ca32731780 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06aa000700 1 -- 172.21.15.86:0/1680690319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00a5fa0 con 0x55ca32731780 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06aa000700 1 -- 172.21.15.86:0/1680690319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f06ac024020 con 0x55ca32731780 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06d185cb80 10 client.14781 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06d185cb80 10 client.14781 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06d185cb80 20 client.14781 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca32745780 0x55ca32746bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06d185cb80 4 client.14781 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.458+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 --> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca3268b630 con 0x55ca32745780 2024-04-25T19:01:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.459+0000 7f06bfe00700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca32745780 0x55ca32746bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.459+0000 7f06bfe00700 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca32745780 0x55ca32746bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51b730685cc5da server_cookie=a37d36b355321da1 in_seq=0 out_seq=0 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.459+0000 7f06aa000700 10 client.14781 ms_handle_connect on v2:172.21.15.183:6836/3273542947 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.460+0000 7f06aa000700 1 -- 172.21.15.86:0/1680690319 <== mds.2 v2:172.21.15.183:6836/3273542947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca3268b630 con 0x55ca32745780 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.460+0000 7f06aa000700 10 client.14781 handle_command_reply: tid=0 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.460+0000 7f06d185cb80 1 client.14781 shutdown 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.460+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 msgr2=0x55ca32734f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.460+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 0x55ca32734f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0127f0 tx=0x7f06ac012b00 comp rx=0 tx=0).stop 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 shutdown_connections 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca32745780 0x55ca32746bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca32735940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 --2- 172.21.15.86:0/1680690319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32731780 0x55ca32734f80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 >> 172.21.15.86:0/1680690319 conn(0x55ca32659f00 msgr2=0x55ca32725e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06aa000700 0 client.14781 ms_handle_reset on v2:172.21.15.183:6836/3273542947 2024-04-25T19:01:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 shutdown_connections 2024-04-25T19:01:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 1 -- 172.21.15.86:0/1680690319 wait complete. 2024-04-25T19:01:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.461+0000 7f06d185cb80 20 client.14781 trim_cache size 0 max 16384 2024-04-25T19:01:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.483+0000 7f06d185cb80 1 Processor -- start 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 -- start start 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca32731780 0x55ca3272c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca327265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca32744f30 con 0x55ca3272bef0 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca32745600 con 0x55ca32731780 2024-04-25T19:01:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06d185cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca32745cd0 con 0x55ca327354c0 2024-04-25T19:01:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59818/0 (socket says 172.21.15.86:59818) 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06bfe00700 1 -- 172.21.15.86:0/3573387756 learned_addr learned my addr 172.21.15.86:0/3573387756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06bf400700 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca32731780 0x55ca3272c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.484+0000 7f06c4e00700 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca327265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bfe00700 1 -- 172.21.15.86:0/3573387756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 msgr2=0x55ca327265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bfe00700 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca327265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bfe00700 1 -- 172.21.15.86:0/3573387756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca32731780 msgr2=0x55ca3272c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bfe00700 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca32731780 0x55ca3272c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bfe00700 1 -- 172.21.15.86:0/3573387756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca32745420 con 0x55ca3272bef0 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bf400700 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca32731780 0x55ca3272c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06bfe00700 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06b0099820 tx=0x7f06b0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e6fe8a011f764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06b4c00700 1 -- 172.21.15.86:0/3573387756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b009d620 con 0x55ca3272bef0 2024-04-25T19:01:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06b4c00700 1 -- 172.21.15.86:0/3573387756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b009d7c0 con 0x55ca3272bef0 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.485+0000 7f06b4c00700 1 -- 172.21.15.86:0/3573387756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0099110 con 0x55ca3272bef0 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 -- 172.21.15.86:0/3573387756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 msgr2=0x55ca32737c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06b0099820 tx=0x7f06b0009910 comp rx=0 tx=0).stop 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 -- 172.21.15.86:0/3573387756 shutdown_connections 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca327354c0 0x55ca327265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32737c80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 --2- 172.21.15.86:0/3573387756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca32731780 0x55ca3272c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 -- 172.21.15.86:0/3573387756 >> 172.21.15.86:0/3573387756 conn(0x55ca32659f00 msgr2=0x55ca32727ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 -- 172.21.15.86:0/3573387756 shutdown_connections 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.486+0000 7f06d185cb80 1 -- 172.21.15.86:0/3573387756 wait complete. 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 -- start start 2024-04-25T19:01:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 Processor -- start 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32750330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32725660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327354c0 0x55ca32725ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca32746a70 con 0x55ca3272bef0 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca327463a0 con 0x55ca327354c0 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca32747140 con 0x55ca32731780 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.487+0000 7f06d185cb80 10 client.0 _ll_register_callbacks cb 0x55ca326166d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32750330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32725660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32750330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59822/0 (socket says 172.21.15.86:59822) 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bfe00700 1 -- 172.21.15.86:0/2938378058 learned_addr learned my addr 172.21.15.86:0/2938378058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06c4e00700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327354c0 0x55ca32725ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 -- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327354c0 msgr2=0x55ca32725ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327354c0 0x55ca32725ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 -- 172.21.15.86:0/2938378058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 msgr2=0x55ca32750330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32750330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 -- 172.21.15.86:0/2938378058 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca32726320 con 0x55ca32731780 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bfe00700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32750330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06bf400700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32725660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f06ac026d10 tx=0x7f06ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb885a54dbcc0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.488+0000 7f06aa000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:01:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06aa000700 1 -- 172.21.15.86:0/2938378058 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00fe50 con 0x55ca32731780 2024-04-25T19:01:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b009d7c0 con 0x55ca32731780 2024-04-25T19:01:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06aa000700 1 -- 172.21.15.86:0/2938378058 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f06ac018070 con 0x55ca32731780 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06aa000700 1 -- 172.21.15.86:0/2938378058 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06ac026670 con 0x55ca32731780 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06d185cb80 10 client.24910 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca327b50c0 con 0x55ca32731780 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06aa000700 1 -- 172.21.15.86:0/2938378058 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b009d7c0 con 0x55ca32731780 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06aa000700 1 -- 172.21.15.86:0/2938378058 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f06ac024070 con 0x55ca32731780 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.489+0000 7f06d185cb80 10 client.24910 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06d185cb80 10 client.24910 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06d185cb80 20 client.24910 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06d185cb80 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca327b9c70 0x55ca327b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06d185cb80 4 client.24910 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 --> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] -- command(tid 0: {"prefix": "status"}) -- 0x55ca326856d0 con 0x55ca327b9c70 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06bfe00700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca327b9c70 0x55ca327b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06bfe00700 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca327b9c70 0x55ca327b8980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a530985d9e21874 server_cookie=2e3fd7f189f5b30c in_seq=0 out_seq=0 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.490+0000 7f06aa000700 10 client.24910 ms_handle_connect on v2:172.21.15.183:6836/3273542947 2024-04-25T19:01:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06aa000700 1 -- 172.21.15.86:0/2938378058 <== mds.2 v2:172.21.15.183:6836/3273542947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ca326856d0 con 0x55ca327b9c70 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06aa000700 10 client.24910 handle_command_reply: tid=0 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 client.24910 shutdown 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 msgr2=0x55ca32725660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32725660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f06ac026d10 tx=0x7f06ac009780 comp rx=0 tx=0).stop 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 shutdown_connections 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55ca327b9c70 0x55ca327b8980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca327354c0 0x55ca32725ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3272bef0 0x55ca32750330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 --2- 172.21.15.86:0/2938378058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca32731780 0x55ca32725660 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 >> 172.21.15.86:0/2938378058 conn(0x55ca32659f00 msgr2=0x55ca32737c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06aa000700 0 client.24910 ms_handle_reset on v2:172.21.15.183:6836/3273542947 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 shutdown_connections 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 1 -- 172.21.15.86:0/2938378058 wait complete. 2024-04-25T19:01:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.491+0000 7f06d185cb80 20 client.24910 trim_cache size 0 max 16384 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24494, 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 154.04344984400001, 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:01:42.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 207.64375019299999 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06a40a3cc0 msgr2=0x7f06a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06a40a3cc0 0x7f06a40a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06b8003e80 tx=0x7f06b8009920 comp rx=0 tx=0).stop 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 msgr2=0x7f06c019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 0x7f06c019b380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005910 tx=0x7f06b000b670 comp rx=0 tx=0).stop 2024-04-25T19:01:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 shutdown_connections 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f06a40a3cc0 0x7f06a40a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06c0108a30 0x7f06c019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c006a750 0x7f06c019b380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 --2- 172.21.15.86:0/111826935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c006b0f0 0x7f06c019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 >> 172.21.15.86:0/111826935 conn(0x7f06c00788f0 msgr2=0x7f06c010ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 shutdown_connections 2024-04-25T19:01:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:42.492+0000 7f06c6e00700 1 -- 172.21.15.86:0/111826935 wait complete. 2024-04-25T19:01:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:42 smithi086 ceph-mon[27783]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:42.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:42 smithi183 ceph-mon[32344]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:42 smithi194 ceph-mon[32740]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:43.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.g status 2024-04-25T19:01:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:44.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:44 smithi086 ceph-mon[27783]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:44 smithi183 ceph-mon[32344]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:44 smithi194 ceph-mon[32740]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 -- 172.21.15.86:0/1785097089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf81092d0 msgr2=0x7fdbf81096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/1785097089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf81092d0 0x7fdbf81096b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80099c0 tx=0x7fdbe8009cd0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 -- 172.21.15.86:0/1785097089 shutdown_connections 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/1785097089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf810e3b0 0x7fdbf8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/1785097089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf81092d0 0x7fdbf81096b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/1785097089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbf8109c80 0x7fdbf810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 -- 172.21.15.86:0/1785097089 >> 172.21.15.86:0/1785097089 conn(0x7fdbf8076210 msgr2=0x7fdbf8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 -- 172.21.15.86:0/1785097089 shutdown_connections 2024-04-25T19:01:46.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.023+0000 7fdbfcc00700 1 -- 172.21.15.86:0/1785097089 wait complete. 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.024+0000 7fdbfcc00700 1 Processor -- start 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.024+0000 7fdbfcc00700 1 -- start start 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbfcc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 0x7fdbf819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbfcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf8109c80 0x7fdbf819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbfcc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbf810e3b0 0x7fdbf81a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbfcc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81192c0 con 0x7fdbf8109c80 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbfcc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8119140 con 0x7fdbf810e3b0 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbfcc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8119440 con 0x7fdbf81092d0 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 0x7fdbf819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 0x7fdbf819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:40506/0 (socket says 172.21.15.86:40506) 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf8109c80 0x7fdbf819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 -- 172.21.15.86:0/44524769 learned_addr learned my addr 172.21.15.86:0/44524769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf6800700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbf810e3b0 0x7fdbf81a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 -- 172.21.15.86:0/44524769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbf810e3b0 msgr2=0x7fdbf81a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbf810e3b0 0x7fdbf81a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 -- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf8109c80 msgr2=0x7fdbf819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.025+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf8109c80 0x7fdbf819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbf5e00700 1 -- 172.21.15.86:0/44524769 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf81a4750 con 0x7fdbf81092d0 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbf5400700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf8109c80 0x7fdbf819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 0x7fdbf819f790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005910 tx=0x7fdbe8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbeea00700 1 -- 172.21.15.86:0/44524769 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8026070 con 0x7fdbf81092d0 2024-04-25T19:01:46.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbeea00700 1 -- 172.21.15.86:0/44524769 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe8003c70 con 0x7fdbf81092d0 2024-04-25T19:01:46.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81a49e0 con 0x7fdbf81092d0 2024-04-25T19:01:46.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbeea00700 1 -- 172.21.15.86:0/44524769 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800fa40 con 0x7fdbf81092d0 2024-04-25T19:01:46.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.026+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81ac2c0 con 0x7fdbf81092d0 2024-04-25T19:01:46.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.029+0000 7fdbeea00700 1 -- 172.21.15.86:0/44524769 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdbe8029070 con 0x7fdbf81092d0 2024-04-25T19:01:46.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 Processor -- start 2024-04-25T19:01:46.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdbeea00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdbdc0a3c40 0x7fdbdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 -- start start 2024-04-25T19:01:46.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdbeea00700 1 -- 172.21.15.86:0/44524769 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fdbe80d0c20 con 0x7fdbf81092d0 2024-04-25T19:01:46.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdbf6800700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdbdc0a3c40 0x7fdbdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 0x55c8e16d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d80a0 0x55c8e16d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8e1623e40 con 0x55c8e16d1ef0 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8e15ca6e0 con 0x55c8e16d80a0 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.031+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c8e161fe00 con 0x55c8e16d7780 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 0x55c8e16d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 0x55c8e16d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:40510/0 (socket says 172.21.15.86:40510) 2024-04-25T19:01:46.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 -- 172.21.15.86:0/523781510 learned_addr learned my addr 172.21.15.86:0/523781510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf6800700 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5400700 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d80a0 0x55c8e16d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf6800700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdbdc0a3c40 0x7fdbdc0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000f590 tx=0x7fdbf0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 -- 172.21.15.86:0/523781510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d80a0 msgr2=0x55c8e16d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d80a0 0x55c8e16d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 -- 172.21.15.86:0/523781510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d1ef0 msgr2=0x55c8e16d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 -- 172.21.15.86:0/523781510 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e15c0630 con 0x55c8e16d7780 2024-04-25T19:01:46.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5400700 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d80a0 0x55c8e16d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:46.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.032+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 0x55c8e16d7b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80a5580 tx=0x7fdbe80036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befd91ca76e99522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.033+0000 7fdbe3400700 1 -- 172.21.15.86:0/523781510 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe809c830 con 0x55c8e16d7780 2024-04-25T19:01:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.033+0000 7fdbe3400700 1 -- 172.21.15.86:0/523781510 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe8003e20 con 0x55c8e16d7780 2024-04-25T19:01:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.033+0000 7fdbe3400700 1 -- 172.21.15.86:0/523781510 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80a5880 con 0x55c8e16d7780 2024-04-25T19:01:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.033+0000 7fdc0765eb80 1 -- 172.21.15.86:0/523781510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 msgr2=0x55c8e16d7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.033+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 0x55c8e16d7b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80a5580 tx=0x7fdbe80036f0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 -- 172.21.15.86:0/523781510 shutdown_connections 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d7780 0x55c8e16d7b60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/523781510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d80a0 0x55c8e16d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 -- 172.21.15.86:0/523781510 >> 172.21.15.86:0/523781510 conn(0x55c8e15fff00 msgr2=0x55c8e16cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 -- 172.21.15.86:0/523781510 shutdown_connections 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 -- 172.21.15.86:0/523781510 wait complete. 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 -- start start 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.034+0000 7fdc0765eb80 1 Processor -- start 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16db930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16de6d0 con 0x55c8e16d7780 2024-04-25T19:01:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8e15dd810 con 0x55c8e16d1ef0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16de850 con 0x55c8e16db4b0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdc0765eb80 10 client.0 _ll_register_callbacks cb 0x55c8e15bc6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdbf5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16ddc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:56724/0 (socket says 172.21.15.86:56724) 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdbf5e00700 1 -- 172.21.15.86:0/3866505290 learned_addr learned my addr 172.21.15.86:0/3866505290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.035+0000 7fdbf6800700 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbf5e00700 1 -- 172.21.15.86:0/3866505290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 msgr2=0x55c8e16db930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16db930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbf5e00700 1 -- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 msgr2=0x55c8e16daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbf5e00700 1 -- 172.21.15.86:0/3866505290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e1571880 con 0x55c8e16d1ef0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16ddc70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009fd0 tx=0x7fdbe809ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceec7487cffeac0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:46.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbd7e00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbd7e00700 1 -- 172.21.15.86:0/3866505290 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe809c830 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdbd7e00700 1 -- 172.21.15.86:0/3866505290 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fdbe809b1d0 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.036+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c8e1631630 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdbd7e00700 1 -- 172.21.15.86:0/3866505290 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbe80a1280 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 10 client.24923 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c8e15baaa0 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdbd7e00700 1 -- 172.21.15.86:0/3866505290 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c8e1631630 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdbd7e00700 1 -- 172.21.15.86:0/3866505290 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdbe809c440 con 0x55c8e16d1ef0 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 10 client.24923 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 10 client.24923 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 20 client.24923 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e16eb410 0x55c8e16ec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 4 client.24923 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.037+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 --> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e162b6d0 con 0x55c8e16eb410 2024-04-25T19:01:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.038+0000 7fdbf5400700 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e16eb410 0x55c8e16ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.038+0000 7fdbd7e00700 10 client.24923 ms_handle_connect on v2:172.21.15.86:6838/3676043902 2024-04-25T19:01:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.038+0000 7fdbf5400700 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e16eb410 0x55c8e16ec880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c50b6c37cdc444d server_cookie=93a7a9c17a59036 in_seq=0 out_seq=0 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdbd7e00700 1 -- 172.21.15.86:0/3866505290 <== mds.1 v2:172.21.15.86:6838/3676043902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8e162b6d0 con 0x55c8e16eb410 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdbd7e00700 10 client.24923 handle_command_reply: tid=0 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 client.24923 shutdown 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 msgr2=0x55c8e16ddc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16ddc70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009fd0 tx=0x7fdbe809ced0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 shutdown_connections 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e16eb410 0x55c8e16ec880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16db930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16ddc70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdbd7e00700 0 client.24923 ms_handle_reset on v2:172.21.15.86:6838/3676043902 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/3866505290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.039+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 >> 172.21.15.86:0/3866505290 conn(0x55c8e15fff00 msgr2=0x55c8e16cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.040+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 shutdown_connections 2024-04-25T19:01:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.040+0000 7fdc0765eb80 1 -- 172.21.15.86:0/3866505290 wait complete. 2024-04-25T19:01:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.040+0000 7fdc0765eb80 20 client.24923 trim_cache size 0 max 16384 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 Processor -- start 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 -- start start 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d7780 0x55c8e16de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16db4b0 0x55c8e16cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16eabc0 con 0x55c8e16db4b0 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16eb290 con 0x55c8e16d7780 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.062+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16eb960 con 0x55c8e16d1ef0 2024-04-25T19:01:46.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16db4b0 0x55c8e16cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16db4b0 0x55c8e16cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59844/0 (socket says 172.21.15.86:59844) 2024-04-25T19:01:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf6800700 1 -- 172.21.15.86:0/1421138653 learned_addr learned my addr 172.21.15.86:0/1421138653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d7780 0x55c8e16de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 -- 172.21.15.86:0/1421138653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d1ef0 msgr2=0x55c8e16d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 -- 172.21.15.86:0/1421138653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16db4b0 msgr2=0x55c8e16cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16db4b0 0x55c8e16cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 -- 172.21.15.86:0/1421138653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e16ed7e0 con 0x55c8e16d7780 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.063+0000 7fdbf5400700 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d7780 0x55c8e16de250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55c8e16ed0b0 tx=0x7fdbe4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d933d98c539d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdbe2a00700 1 -- 172.21.15.86:0/1421138653 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4012bb0 con 0x55c8e16d7780 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdbe2a00700 1 -- 172.21.15.86:0/1421138653 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe4004500 con 0x55c8e16d7780 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdbe2a00700 1 -- 172.21.15.86:0/1421138653 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4005020 con 0x55c8e16d7780 2024-04-25T19:01:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 -- 172.21.15.86:0/1421138653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d7780 msgr2=0x55c8e16de250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d7780 0x55c8e16de250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55c8e16ed0b0 tx=0x7fdbe4009260 comp rx=0 tx=0).stop 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 -- 172.21.15.86:0/1421138653 shutdown_connections 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16db4b0 0x55c8e16cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16d1ef0 0x55c8e16d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/1421138653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d7780 0x55c8e16de250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 -- 172.21.15.86:0/1421138653 >> 172.21.15.86:0/1421138653 conn(0x55c8e15fff00 msgr2=0x55c8e16ea450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.064+0000 7fdc0765eb80 1 -- 172.21.15.86:0/1421138653 shutdown_connections 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.065+0000 7fdc0765eb80 1 -- 172.21.15.86:0/1421138653 wait complete. 2024-04-25T19:01:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.065+0000 7fdc0765eb80 1 -- start start 2024-04-25T19:01:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.065+0000 7fdc0765eb80 1 Processor -- start 2024-04-25T19:01:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16ec700 con 0x55c8e16d7780 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16ec030 con 0x55c8e16d1ef0 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c8e16ecdd0 con 0x55c8e16db4b0 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdc0765eb80 10 client.0 _ll_register_callbacks cb 0x55c8e15bc6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:56730/0 (socket says 172.21.15.86:56730) 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 -- 172.21.15.86:0/2022624542 learned_addr learned my addr 172.21.15.86:0/2022624542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5400700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf6800700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 -- 172.21.15.86:0/2022624542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 msgr2=0x55c8e16f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 -- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 msgr2=0x55c8e16f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.066+0000 7fdbf5e00700 1 -- 172.21.15.86:0/2022624542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e16f1580 con 0x55c8e16d1ef0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdbf5e00700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16f5d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdbe809b180 tx=0x7fdbe8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9211f7a46aed246d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdbd7e00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdbd7e00700 1 -- 172.21.15.86:0/2022624542 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe809c830 con 0x55c8e16d1ef0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe4004500 con 0x55c8e16d1ef0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdbd7e00700 1 -- 172.21.15.86:0/2022624542 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fdbe80b60d0 con 0x55c8e16d1ef0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdbd7e00700 1 -- 172.21.15.86:0/2022624542 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdbe8098f70 con 0x55c8e16d1ef0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdc0765eb80 10 client.24929 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c8e16f1580 con 0x55c8e16d1ef0 2024-04-25T19:01:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.067+0000 7fdbd7e00700 1 -- 172.21.15.86:0/2022624542 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe4004500 con 0x55c8e16d1ef0 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdbd7e00700 1 -- 172.21.15.86:0/2022624542 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdbe809c320 con 0x55c8e16d1ef0 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdc0765eb80 10 client.24929 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdc0765eb80 10 client.24929 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdc0765eb80 20 client.24929 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e1757940 0x55c8e1756650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdc0765eb80 4 client.24929 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 --> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] -- command(tid 0: {"prefix": "status"}) -- 0x55c8e14a4200 con 0x55c8e1757940 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.068+0000 7fdbf5400700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e1757940 0x55c8e1756650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdbf5400700 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e1757940 0x55c8e1756650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=801f573c12b53a4f server_cookie=cc3749b352b58a5a in_seq=0 out_seq=0 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdbd7e00700 10 client.24929 ms_handle_connect on v2:172.21.15.86:6838/3676043902 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdbd7e00700 1 -- 172.21.15.86:0/2022624542 <== mds.1 v2:172.21.15.86:6838/3676043902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c8e14a4200 con 0x55c8e1757940 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdbd7e00700 10 client.24929 handle_command_reply: tid=0 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 client.24929 shutdown 2024-04-25T19:01:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 msgr2=0x55c8e16f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16f5d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdbe809b180 tx=0x7fdbe8009310 comp rx=0 tx=0).stop 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 shutdown_connections 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55c8e1757940 0x55c8e1756650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c8e16db4b0 0x55c8e16f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8e16d1ef0 0x55c8e16f5d00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 --2- 172.21.15.86:0/2022624542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8e16d7780 0x55c8e16f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdbd7e00700 0 client.24929 ms_handle_reset on v2:172.21.15.86:6838/3676043902 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 >> 172.21.15.86:0/2022624542 conn(0x55c8e15fff00 msgr2=0x55c8e16de250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.069+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 shutdown_connections 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.070+0000 7fdc0765eb80 1 -- 172.21.15.86:0/2022624542 wait complete. 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.070+0000 7fdc0765eb80 20 client.24929 trim_cache size 0 max 16384 2024-04-25T19:01:46.075 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24509, 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 158.630249894, 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 202.881241292 2024-04-25T19:01:46.076 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:01:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.070+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdbdc0a3c40 msgr2=0x7fdbdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.070+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdbdc0a3c40 0x7fdbdc0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000f590 tx=0x7fdbf0009920 comp rx=0 tx=0).stop 2024-04-25T19:01:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.070+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 msgr2=0x7fdbf819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.070+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 0x7fdbf819f790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005910 tx=0x7fdbe8004270 comp rx=0 tx=0).stop 2024-04-25T19:01:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 shutdown_connections 2024-04-25T19:01:46.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fdbdc0a3c40 0x7fdbdc0a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbf810e3b0 0x7fdbf81a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbf81092d0 0x7fdbf819f790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 --2- 172.21.15.86:0/44524769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf8109c80 0x7fdbf819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 >> 172.21.15.86:0/44524769 conn(0x7fdbf8076210 msgr2=0x7fdbf8077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 shutdown_connections 2024-04-25T19:01:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:46.071+0000 7fdbfcc00700 1 -- 172.21.15.86:0/44524769 wait complete. 2024-04-25T19:01:46.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:46 smithi086 ceph-mon[27783]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:46 smithi183 ceph-mon[32344]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:46.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T19:01:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:46 smithi194 ceph-mon[32740]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:47.383 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:48.752 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:48 smithi086 ceph-mon[27783]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:48 smithi183 ceph-mon[32344]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:48 smithi194 ceph-mon[32740]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 -- 172.21.15.86:0/1787492389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 msgr2=0x7fd5941096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 --2- 172.21.15.86:0/1787492389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941096b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b0a0 tx=0x7fd58800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 -- 172.21.15.86:0/1787492389 shutdown_connections 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 --2- 172.21.15.86:0/1787492389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd59410e3b0 0x7fd594115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 --2- 172.21.15.86:0/1787492389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941096b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 --2- 172.21.15.86:0/1787492389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd594109c80 0x7fd59410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 -- 172.21.15.86:0/1787492389 >> 172.21.15.86:0/1787492389 conn(0x7fd594076210 msgr2=0x7fd594107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 -- 172.21.15.86:0/1787492389 shutdown_connections 2024-04-25T19:01:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.542+0000 7fd59b600700 1 -- 172.21.15.86:0/1787492389 wait complete. 2024-04-25T19:01:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.543+0000 7fd59b600700 1 Processor -- start 2024-04-25T19:01:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 -- start start 2024-04-25T19:01:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 0x7fd5941a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd59410e3b0 0x7fd59419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd594119290 con 0x7fd594109c80 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd594119110 con 0x7fd5941092d0 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd59b600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd594119410 con 0x7fd59410e3b0 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 0x7fd5941a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd599600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd59410e3b0 0x7fd59419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 0x7fd5941a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59876/0 (socket says 172.21.15.86:59876) 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.544+0000 7fd593e00700 1 -- 172.21.15.86:0/1478854914 learned_addr learned my addr 172.21.15.86:0/1478854914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 -- 172.21.15.86:0/1478854914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd59410e3b0 msgr2=0x7fd59419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd59410e3b0 0x7fd59419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 -- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 msgr2=0x7fd5941a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 0x7fd5941a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 -- 172.21.15.86:0/1478854914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59419fcf0 con 0x7fd5941092d0 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd593e00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 0x7fd5941a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.545+0000 7fd598c00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941a53d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b780 tx=0x7fd588004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.546+0000 7fd591600700 1 -- 172.21.15.86:0/1478854914 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588021070 con 0x7fd5941092d0 2024-04-25T19:01:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.546+0000 7fd591600700 1 -- 172.21.15.86:0/1478854914 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd588007b80 con 0x7fd5941092d0 2024-04-25T19:01:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.546+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59419ff80 con 0x7fd5941092d0 2024-04-25T19:01:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.546+0000 7fd591600700 1 -- 172.21.15.86:0/1478854914 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588005040 con 0x7fd5941092d0 2024-04-25T19:01:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.546+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941ac2a0 con 0x7fd5941092d0 2024-04-25T19:01:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.549+0000 7fd591600700 1 -- 172.21.15.86:0/1478854914 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd588024070 con 0x7fd5941092d0 2024-04-25T19:01:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.550+0000 7fd591600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd57c0a3cc0 0x7fd57c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.550+0000 7fd5a604db80 1 Processor -- start 2024-04-25T19:01:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.550+0000 7fd591600700 1 -- 172.21.15.86:0/1478854914 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fd58800f020 con 0x7fd5941092d0 2024-04-25T19:01:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd593e00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd57c0a3cc0 0x7fd57c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 -- start start 2024-04-25T19:01:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f20c10 0x562803f2b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f2bab0 0x562803f2dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562803e73e40 con 0x562803f202f0 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562803e1a6e0 con 0x562803f2bab0 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd5a604db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562803e6fe00 con 0x562803f20c10 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.551+0000 7fd593e00700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd57c0a3cc0 0x7fd57c0a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd594108880 tx=0x7fd580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f206d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59892/0 (socket says 172.21.15.86:59892) 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 -- 172.21.15.86:0/2473535846 learned_addr learned my addr 172.21.15.86:0/2473535846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd599600700 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f2bab0 0x562803f2dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd593e00700 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f20c10 0x562803f2b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 -- 172.21.15.86:0/2473535846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f20c10 msgr2=0x562803f2b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f20c10 0x562803f2b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 -- 172.21.15.86:0/2473535846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f2bab0 msgr2=0x562803f2dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f2bab0 0x562803f2dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 -- 172.21.15.86:0/2473535846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562803e10630 con 0x562803f202f0 2024-04-25T19:01:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd599600700 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f2bab0 0x562803f2dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd598c00700 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f206d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd58809d9b0 tx=0x7fd58801da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e61e5a560fdd6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.552+0000 7fd586000700 1 -- 172.21.15.86:0/2473535846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5880074e0 con 0x562803f202f0 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.553+0000 7fd586000700 1 -- 172.21.15.86:0/2473535846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58809da20 con 0x562803f202f0 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.553+0000 7fd586000700 1 -- 172.21.15.86:0/2473535846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5880a42f0 con 0x562803f202f0 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.553+0000 7fd5a604db80 1 -- 172.21.15.86:0/2473535846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 msgr2=0x562803f206d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.553+0000 7fd5a604db80 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f206d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd58809d9b0 tx=0x7fd58801da70 comp rx=0 tx=0).stop 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.553+0000 7fd5a604db80 1 -- 172.21.15.86:0/2473535846 shutdown_connections 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.553+0000 7fd5a604db80 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f2bab0 0x562803f2dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f206d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 --2- 172.21.15.86:0/2473535846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f20c10 0x562803f2b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 -- 172.21.15.86:0/2473535846 >> 172.21.15.86:0/2473535846 conn(0x562803e4f550 msgr2=0x562803f1c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 -- 172.21.15.86:0/2473535846 shutdown_connections 2024-04-25T19:01:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 -- 172.21.15.86:0/2473535846 wait complete. 2024-04-25T19:01:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 -- start start 2024-04-25T19:01:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.554+0000 7fd5a604db80 1 Processor -- start 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f202f0 0x562803f2b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 0x562803f28570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f28ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562803f2e3e0 con 0x562803f20c10 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562803e2d810 con 0x562803f202f0 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562803f2e560 con 0x562803f2bab0 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd5a604db80 10 client.0 _ll_register_callbacks cb 0x562803e0c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 0x562803f28570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd598c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f202f0 0x562803f2b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 0x562803f28570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59904/0 (socket says 172.21.15.86:59904) 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd593e00700 1 -- 172.21.15.86:0/3683130169 learned_addr learned my addr 172.21.15.86:0/3683130169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.555+0000 7fd599600700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f28ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd593e00700 1 -- 172.21.15.86:0/3683130169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 msgr2=0x562803f28ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd593e00700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f28ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd593e00700 1 -- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f202f0 msgr2=0x562803f2b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd593e00700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f202f0 0x562803f2b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd593e00700 1 -- 172.21.15.86:0/3683130169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562803dc1880 con 0x562803f20c10 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd599600700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f28ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd593e00700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 0x562803f28570 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd5800039b0 tx=0x7fd5800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ac7d4bc7fce39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd57aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd57aa00700 1 -- 172.21.15.86:0/3683130169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58001fa90 con 0x562803f20c10 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd57aa00700 1 -- 172.21.15.86:0/3683130169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fd58001fc30 con 0x562803f20c10 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.556+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58809da20 con 0x562803f20c10 2024-04-25T19:01:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd57aa00700 1 -- 172.21.15.86:0/3683130169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd580028440 con 0x562803f20c10 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 10 client.14817 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562803e0aaa0 con 0x562803f20c10 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd57aa00700 1 -- 172.21.15.86:0/3683130169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58809da20 con 0x562803f20c10 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd57aa00700 1 -- 172.21.15.86:0/3683130169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd580030020 con 0x562803f20c10 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 10 client.14817 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 10 client.14817 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 20 client.14817 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803f3b0a0 0x562803f3c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 4 client.14817 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.557+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562803e81630 con 0x562803f3b0a0 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.558+0000 7fd598c00700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803f3b0a0 0x562803f3c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.558+0000 7fd598c00700 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803f3b0a0 0x562803f3c510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5af4724561557ab server_cookie=72666a8c7a226fb4 in_seq=0 out_seq=0 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.558+0000 7fd57aa00700 10 client.14817 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.559+0000 7fd57aa00700 1 -- 172.21.15.86:0/3683130169 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562803e81630 con 0x562803f3b0a0 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.559+0000 7fd57aa00700 10 client.14817 handle_command_reply: tid=0 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.559+0000 7fd5a604db80 1 client.14817 shutdown 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 msgr2=0x562803f28570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 0x562803f28570 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd5800039b0 tx=0x7fd5800047c0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 shutdown_connections 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803f3b0a0 0x562803f3c510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f28ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f202f0 0x562803f2b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3683130169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f20c10 0x562803f28570 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 >> 172.21.15.86:0/3683130169 conn(0x562803e4f550 msgr2=0x562803f22c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd57aa00700 0 client.14817 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:01:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 shutdown_connections 2024-04-25T19:01:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 1 -- 172.21.15.86:0/3683130169 wait complete. 2024-04-25T19:01:49.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.560+0000 7fd5a604db80 20 client.14817 trim_cache size 0 max 16384 2024-04-25T19:01:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.582+0000 7fd5a604db80 1 Processor -- start 2024-04-25T19:01:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.582+0000 7fd5a604db80 1 -- start start 2024-04-25T19:01:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f28a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f1c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f1c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd5a604db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562803f3a850 con 0x562803f202f0 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd5a604db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562803f3af20 con 0x562803f20c10 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd5a604db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562803f3b5f0 con 0x562803f2bab0 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f1c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd598c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f28a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f1c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:56774/0 (socket says 172.21.15.86:56774) 2024-04-25T19:01:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 -- 172.21.15.86:0/3697609586 learned_addr learned my addr 172.21.15.86:0/3697609586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd599600700 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f1c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 -- 172.21.15.86:0/3697609586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 msgr2=0x562803f1c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f1c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 -- 172.21.15.86:0/3697609586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 msgr2=0x562803f28a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f28a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.583+0000 7fd593e00700 1 -- 172.21.15.86:0/3697609586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562803f13f70 con 0x562803f20c10 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.584+0000 7fd599600700 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f1c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.584+0000 7fd593e00700 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f1c1d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5800328b0 tx=0x7fd580014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a91f7c6fffd237d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.584+0000 7fd585600700 1 -- 172.21.15.86:0/3697609586 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580032b30 con 0x562803f20c10 2024-04-25T19:01:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.584+0000 7fd585600700 1 -- 172.21.15.86:0/3697609586 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd580004ce0 con 0x562803f20c10 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.584+0000 7fd585600700 1 -- 172.21.15.86:0/3697609586 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580005510 con 0x562803f20c10 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 -- 172.21.15.86:0/3697609586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 msgr2=0x562803f1c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f1c1d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5800328b0 tx=0x7fd580014000 comp rx=0 tx=0).stop 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 -- 172.21.15.86:0/3697609586 shutdown_connections 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f2bab0 0x562803f1c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f202f0 0x562803f28a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 --2- 172.21.15.86:0/3697609586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f1c1d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 -- 172.21.15.86:0/3697609586 >> 172.21.15.86:0/3697609586 conn(0x562803e4f550 msgr2=0x562803f28350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 -- 172.21.15.86:0/3697609586 shutdown_connections 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 -- 172.21.15.86:0/3697609586 wait complete. 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 -- start start 2024-04-25T19:01:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.585+0000 7fd5a604db80 1 Processor -- start 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f202f0 0x562803f25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f25a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f2bab0 0x562803f41e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562803f3c390 con 0x562803f2bab0 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562803f3bcc0 con 0x562803f20c10 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562803f3ca60 con 0x562803f202f0 2024-04-25T19:01:49.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd5a604db80 10 client.0 _ll_register_callbacks cb 0x562803e0c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd599600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f2bab0 0x562803f41e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd598c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f202f0 0x562803f25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd599600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f2bab0 0x562803f41e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59914/0 (socket says 172.21.15.86:59914) 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd599600700 1 -- 172.21.15.86:0/1090960356 learned_addr learned my addr 172.21.15.86:0/1090960356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.586+0000 7fd593e00700 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f25a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd598c00700 1 -- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 msgr2=0x562803f25a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd598c00700 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f25a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd598c00700 1 -- 172.21.15.86:0/1090960356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f2bab0 msgr2=0x562803f41e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd598c00700 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f2bab0 0x562803f41e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd598c00700 1 -- 172.21.15.86:0/1090960356 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562803f42370 con 0x562803f202f0 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd598c00700 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f202f0 0x562803f25550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd588009fd0 tx=0x7fd588009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a8975dfe329b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:49.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd57aa00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd57aa00700 1 -- 172.21.15.86:0/1090960356 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58809da20 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd580004ce0 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd57aa00700 1 -- 172.21.15.86:0/1090960356 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fd58809bd20 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd57aa00700 1 -- 172.21.15.86:0/1090960356 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5880a1dd0 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.587+0000 7fd5a604db80 10 client.24946 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562803fa2810 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd57aa00700 1 -- 172.21.15.86:0/1090960356 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd580004ce0 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd57aa00700 1 -- 172.21.15.86:0/1090960356 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd588011070 con 0x562803f202f0 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 10 client.24946 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 10 client.24946 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 20 client.24946 populate_metadata read hostname 'smithi086' 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803fa6fe0 0x562803fa5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 4 client.24946 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:01:49.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "status"}) -- 0x562803e7b6d0 con 0x562803fa6fe0 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.588+0000 7fd593e00700 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803fa6fe0 0x562803fa5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd57aa00700 10 client.24946 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd593e00700 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803fa6fe0 0x562803fa5cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870c1a0a3ad4faf5 server_cookie=2c802d70744e64f7 in_seq=0 out_seq=0 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd57aa00700 1 -- 172.21.15.86:0/1090960356 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562803e7b6d0 con 0x562803fa6fe0 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd57aa00700 10 client.24946 handle_command_reply: tid=0 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd5a604db80 1 client.24946 shutdown 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f202f0 msgr2=0x562803f25550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.589+0000 7fd5a604db80 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f202f0 0x562803f25550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd588009fd0 tx=0x7fd588009e20 comp rx=0 tx=0).stop 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 shutdown_connections 2024-04-25T19:01:49.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562803fa6fe0 0x562803fa5cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562803f2bab0 0x562803f41e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562803f202f0 0x562803f25550 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 --2- 172.21.15.86:0/1090960356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562803f20c10 0x562803f25a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 >> 172.21.15.86:0/1090960356 conn(0x562803e4f550 msgr2=0x562803f1c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd57aa00700 0 client.24946 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 shutdown_connections 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 1 -- 172.21.15.86:0/1090960356 wait complete. 2024-04-25T19:01:49.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd5a604db80 20 client.24946 trim_cache size 0 max 16384 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24527, 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 178.013078984, 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:01:49.596 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:01:49.597 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:01:49.597 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 198.521193342 2024-04-25T19:01:49.597 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:01:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd57c0a3cc0 msgr2=0x7fd57c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.590+0000 7fd59b600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd57c0a3cc0 0x7fd57c0a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd594108880 tx=0x7fd580009260 comp rx=0 tx=0).stop 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 msgr2=0x7fd5941a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941a53d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b780 tx=0x7fd588004d00 comp rx=0 tx=0).stop 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 shutdown_connections 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fd57c0a3cc0 0x7fd57c0a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd59410e3b0 0x7fd59419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5941092d0 0x7fd5941a53d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 --2- 172.21.15.86:0/1478854914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd594109c80 0x7fd5941a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 >> 172.21.15.86:0/1478854914 conn(0x7fd594076210 msgr2=0x7fd594077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 shutdown_connections 2024-04-25T19:01:49.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:49.591+0000 7fd59b600700 1 -- 172.21.15.86:0/1478854914 wait complete. 2024-04-25T19:01:50.420 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T19:01:50.421 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T19:01:50.430 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:01:50.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T19:01:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:50 smithi086 ceph-mon[27783]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:50.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:50 smithi183 ceph-mon[32344]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:50.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:50 smithi194 ceph-mon[32740]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:51 smithi086 ceph-mon[27783]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:51.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:51 smithi183 ceph-mon[32344]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:51.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:51 smithi194 ceph-mon[32740]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:01:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 -- 172.21.15.86:0/3604890849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f991010a970 msgr2=0x7f991010ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 --2- 172.21.15.86:0/3604890849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f991010a970 0x7f991010ce00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9908009f90 tx=0x7f990800b330 comp rx=0 tx=0).stop 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 -- 172.21.15.86:0/3604890849 shutdown_connections 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 --2- 172.21.15.86:0/3604890849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f991010a970 0x7f991010ce00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 --2- 172.21.15.86:0/3604890849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9910076a70 0x7f9910074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 --2- 172.21.15.86:0/3604890849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99100754a0 0x7f9910075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.067+0000 7f9916800700 1 -- 172.21.15.86:0/3604890849 >> 172.21.15.86:0/3604890849 conn(0x7f99100fe040 msgr2=0x7f9910100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.068+0000 7f9916800700 1 -- 172.21.15.86:0/3604890849 shutdown_connections 2024-04-25T19:01:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.068+0000 7f9916800700 1 -- 172.21.15.86:0/3604890849 wait complete. 2024-04-25T19:01:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.069+0000 7f9916800700 1 Processor -- start 2024-04-25T19:01:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.069+0000 7f9916800700 1 -- start start 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f9916800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 0x7f991019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f9916800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9910076a70 0x7f991019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f9916800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f991010a970 0x7f99101a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f9916800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9910110780 con 0x7f9910076a70 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f9916800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9910110600 con 0x7f99100754a0 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f9916800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9910110900 con 0x7f991010a970 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9910076a70 0x7f991019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 0x7f991019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 0x7f991019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:52364/0 (socket says 172.21.15.86:52364) 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f991010a970 0x7f99101a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990f400700 1 -- 172.21.15.86:0/2367583643 learned_addr learned my addr 172.21.15.86:0/2367583643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990f400700 1 -- 172.21.15.86:0/2367583643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f991010a970 msgr2=0x7f99101a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.070+0000 7f990f400700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f991010a970 0x7f99101a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f990f400700 1 -- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9910076a70 msgr2=0x7f991019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f990f400700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9910076a70 0x7f991019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f990f400700 1 -- 172.21.15.86:0/2367583643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99101a2a50 con 0x7f99100754a0 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f990fe00700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f991010a970 0x7f99101a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f990ea00700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9910076a70 0x7f991019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f990f400700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 0x7f991019ae40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f990000e970 tx=0x7f990000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900004a30 con 0x7f99100754a0 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f990000cc70 con 0x7f99100754a0 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101a2d40 con 0x7f99100754a0 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900005550 con 0x7f99100754a0 2024-04-25T19:01:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.071+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101a31c0 con 0x7f99100754a0 2024-04-25T19:01:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.072+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9910069730 con 0x7f99100754a0 2024-04-25T19:01:53.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.075+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f990000c7c0 con 0x7f99100754a0 2024-04-25T19:01:53.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.076+0000 7f9907e00700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f98f40a3c40 0x7f98f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:53.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.076+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f99000d0840 con 0x7f99100754a0 2024-04-25T19:01:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.076+0000 7f990ea00700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f98f40a3c40 0x7f98f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.077+0000 7f990ea00700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f98f40a3c40 0x7f98f40a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99100767a0 tx=0x7f98fc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.078+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9900053ff0 con 0x7f99100754a0 2024-04-25T19:01:53.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.324+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9910062e80 con 0x7f98f40a3c40 2024-04-25T19:01:53.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.326+0000 7f9907e00700 1 -- 172.21.15.86:0/2367583643 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9910062e80 con 0x7f98f40a3c40 2024-04-25T19:01:53.328 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-04-25T19:01:53.328 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-04-25T19:01:53.328 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f98f40a3c40 msgr2=0x7f98f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f98f40a3c40 0x7f98f40a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99100767a0 tx=0x7f98fc009310 comp rx=0 tx=0).stop 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 msgr2=0x7f991019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 0x7f991019ae40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f990000e970 tx=0x7f990000ec80 comp rx=0 tx=0).stop 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 shutdown_connections 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f98f40a3c40 0x7f98f40a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f991010a970 0x7f99101a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99100754a0 0x7f991019ae40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 --2- 172.21.15.86:0/2367583643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9910076a70 0x7f991019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 >> 172.21.15.86:0/2367583643 conn(0x7f99100fe040 msgr2=0x7f9910106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.329+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 shutdown_connections 2024-04-25T19:01:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:53.330+0000 7f9916800700 1 -- 172.21.15.86:0/2367583643 wait complete. 2024-04-25T19:01:54.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T19:01:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:54.727 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:54 smithi086 ceph-mon[27783]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:01:54.727 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:54 smithi086 ceph-mon[27783]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:54 smithi194 ceph-mon[32740]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:01:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:54 smithi194 ceph-mon[32740]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:54 smithi183 ceph-mon[32344]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:01:54.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:54 smithi183 ceph-mon[32344]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:56 smithi194 ceph-mon[32740]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.749+0000 7f024ec00700 1 -- 172.21.15.86:0/4052015981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 msgr2=0x7f0248105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.749+0000 7f024ec00700 1 --2- 172.21.15.86:0/4052015981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f0248105920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0238009990 tx=0x7f0238009ca0 comp rx=0 tx=0).stop 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 -- 172.21.15.86:0/4052015981 shutdown_connections 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 --2- 172.21.15.86:0/4052015981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0248108790 0x7f024810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 --2- 172.21.15.86:0/4052015981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f0248105920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 --2- 172.21.15.86:0/4052015981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0248105e60 0x7f0248108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 -- 172.21.15.86:0/4052015981 >> 172.21.15.86:0/4052015981 conn(0x7f02480fcf40 msgr2=0x7f02480ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 -- 172.21.15.86:0/4052015981 shutdown_connections 2024-04-25T19:01:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.750+0000 7f024ec00700 1 -- 172.21.15.86:0/4052015981 wait complete. 2024-04-25T19:01:56.753 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:56 smithi086 ceph-mon[27783]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:56.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.751+0000 7f024ec00700 1 Processor -- start 2024-04-25T19:01:56.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.751+0000 7f024ec00700 1 -- start start 2024-04-25T19:01:56.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.751+0000 7f024ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f024819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f024ec00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0248105e60 0x7f024819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f024ec00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0248108790 0x7f02481a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f024ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f024810e520 con 0x7f0248103530 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f024ec00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f024810e3a0 con 0x7f0248105e60 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f024ec00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f024810e6a0 con 0x7f0248108790 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f024819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f024819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51008/0 (socket says 172.21.15.86:51008) 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 -- 172.21.15.86:0/1135694165 learned_addr learned my addr 172.21.15.86:0/1135694165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f023f400700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0248105e60 0x7f024819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f024cc00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0248108790 0x7f02481a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 -- 172.21.15.86:0/1135694165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0248108790 msgr2=0x7f02481a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0248108790 0x7f02481a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 -- 172.21.15.86:0/1135694165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0248105e60 msgr2=0x7f024819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0248105e60 0x7f024819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.752+0000 7f0247e00700 1 -- 172.21.15.86:0/1135694165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02481a2aa0 con 0x7f0248103530 2024-04-25T19:01:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.753+0000 7f0247e00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f024819ae40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f02380057f0 tx=0x7f0238009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.753+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238026070 con 0x7f0248103530 2024-04-25T19:01:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.753+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0238021410 con 0x7f0248103530 2024-04-25T19:01:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.753+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238017650 con 0x7f0248103530 2024-04-25T19:01:56.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.753+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02481a2d30 con 0x7f0248103530 2024-04-25T19:01:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.753+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02481a3190 con 0x7f0248103530 2024-04-25T19:01:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.755+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0248069730 con 0x7f0248103530 2024-04-25T19:01:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.757+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0238029070 con 0x7f0248103530 2024-04-25T19:01:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.759+0000 7f0245600700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f022c0a3cc0 0x7f022c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.759+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f02380d1300 con 0x7f0248103530 2024-04-25T19:01:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.759+0000 7f023f400700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f022c0a3cc0 0x7f022c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:56.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.760+0000 7f023f400700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f022c0a3cc0 0x7f022c0a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0230009c60 tx=0x7f0230009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:56.764+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f023805dff0 con 0x7f0248103530 2024-04-25T19:01:56.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:56 smithi183 ceph-mon[32344]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:01:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.002+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0248101c90 con 0x7f022c0a3cc0 2024-04-25T19:01:57.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.043+0000 7f0245600700 1 -- 172.21.15.86:0/1135694165 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f0248101c90 con 0x7f022c0a3cc0 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (3m) 50s ago 3m 13.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 14e2c7e5f3b6 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi183 running (3m) 65s ago 3m 29.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9ff187b36c4e 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi194 running (3m) 65s ago 3m 22.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 73eed52809e2 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (3m) 50s ago 3m 18.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3f9e16f47238 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi183 running (3m) 65s ago 3m 21.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 aa9aefaa080e 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi194 running (3m) 65s ago 3m 18.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 aebf8b3de1cc 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (3m) 50s ago 3m 18.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5e4f719ae3d2 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi183 running (3m) 65s ago 3m 21.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 877b444a2ecd 2024-04-25T19:01:57.046 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi194 running (3m) 65s ago 3m 22.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 68818f808e5f 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (3m) 50s ago 3m 13.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d1c7615f0661 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi183 running (4m) 65s ago 4m 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 99f24bcc8c82 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi194 running (4m) 65s ago 4m 21.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bcbf93062965 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (12m) 50s ago 12m 516M - 19.0.0-3156-gd69f13d8 da4b004c09e8 06ae06a75995 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi183 *:8443,8765 running (10m) 65s ago 9m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f26e03817af6 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi194 *:8443,8765 running (10m) 65s ago 10m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c92e9a25aeb6 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (12m) 50s ago 12m 50.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 2c6bbf4e3091 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi183 running (10m) 65s ago 10m 45.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 e7a7f96bbe25 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi194 running (10m) 65s ago 10m 42.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 556dd49b81d0 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (9m) 50s ago 9m 46.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d1370c9d29ee 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi183 running (9m) 65s ago 9m 47.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f0ef81040bdf 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi194 running (8m) 65s ago 8m 59.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a79bc65e24bb 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (8m) 50s ago 8m 55.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8ed25e25e24b 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi183 running (7m) 65s ago 7m 46.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d9d9c13d1a56 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi194 running (7m) 65s ago 7m 47.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6f613c386e9b 2024-04-25T19:01:57.047 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (6m) 50s ago 6m 52.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a3303775e4b9 2024-04-25T19:01:57.048 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi183 running (6m) 65s ago 6m 48.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 2576ec10982d 2024-04-25T19:01:57.048 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi194 running (5m) 65s ago 5m 45.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f95b0a6d7aa0 2024-04-25T19:01:57.048 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (5m) 50s ago 5m 42.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b4d06c7b4088 2024-04-25T19:01:57.048 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi183 running (4m) 65s ago 4m 59.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f8fe1d8b3a7a 2024-04-25T19:01:57.048 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi194 running (4m) 65s ago 4m 54.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c8e0be7f52a3 2024-04-25T19:01:57.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.046+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f022c0a3cc0 msgr2=0x7f022c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.046+0000 7f024ec00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f022c0a3cc0 0x7f022c0a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0230009c60 tx=0x7f0230009260 comp rx=0 tx=0).stop 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.046+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 msgr2=0x7f024819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.046+0000 7f024ec00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f024819ae40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f02380057f0 tx=0x7f0238009510 comp rx=0 tx=0).stop 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 shutdown_connections 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f022c0a3cc0 0x7f022c0a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0248108790 0x7f02481a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0248103530 0x7f024819ae40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 --2- 172.21.15.86:0/1135694165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0248105e60 0x7f024819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 >> 172.21.15.86:0/1135694165 conn(0x7f02480fcf40 msgr2=0x7f0248101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 shutdown_connections 2024-04-25T19:01:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:01:57.047+0000 7f024ec00700 1 -- 172.21.15.86:0/1135694165 wait complete. 2024-04-25T19:01:57.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T19:01:58.312 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:58 smithi194 ceph-mon[32740]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:01:58.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:01:58 smithi194 ceph-mon[32740]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:58.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:58 smithi086 ceph-mon[27783]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:01:58.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:01:58 smithi086 ceph-mon[27783]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:01:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:58 smithi183 ceph-mon[32344]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:01:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:01:58 smithi183 ceph-mon[32344]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:02:00.509 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:00 smithi086 ceph-mon[27783]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.572+0000 7fb9d4200700 1 -- 172.21.15.86:0/134265929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc10a2e0 msgr2=0x7fb9cc10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.572+0000 7fb9d4200700 1 --2- 172.21.15.86:0/134265929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc10a2e0 0x7fb9cc10a740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009990 tx=0x7fb9b8009ca0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 -- 172.21.15.86:0/134265929 shutdown_connections 2024-04-25T19:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 --2- 172.21.15.86:0/134265929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9cc10ac80 0x7fb9cc111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 --2- 172.21.15.86:0/134265929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 --2- 172.21.15.86:0/134265929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc10a2e0 0x7fb9cc10a740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 -- 172.21.15.86:0/134265929 >> 172.21.15.86:0/134265929 conn(0x7fb9cc0780b0 msgr2=0x7fb9cc07a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 -- 172.21.15.86:0/134265929 shutdown_connections 2024-04-25T19:02:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.573+0000 7fb9d4200700 1 -- 172.21.15.86:0/134265929 wait complete. 2024-04-25T19:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 Processor -- start 2024-04-25T19:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 -- start start 2024-04-25T19:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9cc10a2e0 0x7fb9cc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc10ac80 0x7fb9cc1a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc114b70 con 0x7fb9cc10a2e0 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1149f0 con 0x7fb9cc10ac80 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.574+0000 7fb9d4200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc114cf0 con 0x7fb9cc1090e0 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44952/0 (socket says 172.21.15.86:44952) 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 -- 172.21.15.86:0/1592171348 learned_addr learned my addr 172.21.15.86:0/1592171348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d0e00700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9cc10a2e0 0x7fb9cc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d2200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc10ac80 0x7fb9cc1a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 -- 172.21.15.86:0/1592171348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc10ac80 msgr2=0x7fb9cc1a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc10ac80 0x7fb9cc1a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 -- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9cc10a2e0 msgr2=0x7fb9cc19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9cc10a2e0 0x7fb9cc19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 -- 172.21.15.86:0/1592171348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc1a6e40 con 0x7fb9cc1090e0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d2200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc10ac80 0x7fb9cc1a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.575+0000 7fb9d1800700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc19f1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e980 tx=0x7fb9c000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.576+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000cbd0 con 0x7fb9cc1090e0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.576+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c00044e0 con 0x7fb9cc1090e0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.576+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0003e00 con 0x7fb9cc1090e0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.576+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc1a7130 con 0x7fb9cc1090e0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.576+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1a7620 con 0x7fb9cc1090e0 2024-04-25T19:02:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.577+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9cc069730 con 0x7fb9cc1090e0 2024-04-25T19:02:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.579+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9c0003fa0 con 0x7fb9cc1090e0 2024-04-25T19:02:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.581+0000 7fb9ca000700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb9b40a3cc0 0x7fb9b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.581+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fb9c00d07f0 con 0x7fb9cc1090e0 2024-04-25T19:02:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.581+0000 7fb9d0e00700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb9b40a3cc0 0x7fb9b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.582+0000 7fb9d0e00700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb9b40a3cc0 0x7fb9b40a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800b720 tx=0x7fb9b800b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.585+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9c009d110 con 0x7fb9cc1090e0 2024-04-25T19:02:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:00 smithi194 ceph-mon[32740]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.824+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb9cc062e80 con 0x7fb9b40a3cc0 2024-04-25T19:02:00.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.840+0000 7fb9ca000700 1 -- 172.21.15.86:0/1592171348 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb9cc062e80 con 0x7fb9b40a3cc0 2024-04-25T19:02:00.843 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T19:02:00.843 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 69s ago 4m smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-04-25T19:02:00.843 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 69s ago 10m smithi086=x;smithi183=y;smithi194=z;count:3 2024-04-25T19:02:00.843 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 69s ago 10m smithi086:172.21.15.86=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c;count:3 2024-04-25T19:02:00.843 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 69s ago - 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb9b40a3cc0 msgr2=0x7fb9b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb9b40a3cc0 0x7fb9b40a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800b720 tx=0x7fb9b800b7f0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 msgr2=0x7fb9cc19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc19f1e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e980 tx=0x7fb9c000ec90 comp rx=0 tx=0).stop 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 shutdown_connections 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fb9b40a3cc0 0x7fb9b40a6180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc10ac80 0x7fb9cc1a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1090e0 0x7fb9cc19f1e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 --2- 172.21.15.86:0/1592171348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9cc10a2e0 0x7fb9cc19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.843+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 >> 172.21.15.86:0/1592171348 conn(0x7fb9cc0780b0 msgr2=0x7fb9cc10d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.844+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 shutdown_connections 2024-04-25T19:02:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:00.844+0000 7fb9d4200700 1 -- 172.21.15.86:0/1592171348 wait complete. 2024-04-25T19:02:00.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:00 smithi183 ceph-mon[32344]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:01.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T19:02:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:02 smithi086 ceph-mon[27783]: from='client.24964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:02.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:02 smithi086 ceph-mon[27783]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:02 smithi194 ceph-mon[32740]: from='client.24964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:02 smithi194 ceph-mon[32740]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:02.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:02 smithi183 ceph-mon[32344]: from='client.24964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:02 smithi183 ceph-mon[32344]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 -- 172.21.15.86:0/4242251971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 msgr2=0x7f132c079880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 --2- 172.21.15.86:0/4242251971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 0x7f132c079880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1324009f90 tx=0x7f132400b330 comp rx=0 tx=0).stop 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 -- 172.21.15.86:0/4242251971 shutdown_connections 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 --2- 172.21.15.86:0/4242251971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 0x7f132c079880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 --2- 172.21.15.86:0/4242251971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 0x7f132c077810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 --2- 172.21.15.86:0/4242251971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 0x7f132c078ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.381+0000 7f1333600700 1 -- 172.21.15.86:0/4242251971 >> 172.21.15.86:0/4242251971 conn(0x7f132c100250 msgr2=0x7f132c102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.382+0000 7f1333600700 1 -- 172.21.15.86:0/4242251971 shutdown_connections 2024-04-25T19:02:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.382+0000 7f1333600700 1 -- 172.21.15.86:0/4242251971 wait complete. 2024-04-25T19:02:04.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 Processor -- start 2024-04-25T19:02:04.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 -- start start 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 0x7f132c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 0x7f132c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 0x7f132c1a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f132c114b10 con 0x7f132c078640 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f132c114990 con 0x7f132c079000 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f1333600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f132c114c90 con 0x7f132c0773f0 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f132be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 0x7f132c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f132be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 0x7f132c19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42572/0 (socket says 172.21.15.86:42572) 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.383+0000 7f132be00700 1 -- 172.21.15.86:0/3916830934 learned_addr learned my addr 172.21.15.86:0/3916830934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 0x7f132c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1331600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 0x7f132c1a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 -- 172.21.15.86:0/3916830934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 msgr2=0x7f132c1a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 0x7f132c1a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 -- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 msgr2=0x7f132c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 0x7f132c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 -- 172.21.15.86:0/3916830934 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c1a6d30 con 0x7f132c0773f0 2024-04-25T19:02:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.384+0000 7f1330c00700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 0x7f132c19f0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f132000ea20 tx=0x7f132000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.385+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132000cbd0 con 0x7f132c0773f0 2024-04-25T19:02:04.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.385+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c1a6fc0 con 0x7f132c0773f0 2024-04-25T19:02:04.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.385+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132000ee60 con 0x7f132c0773f0 2024-04-25T19:02:04.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.385+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320018680 con 0x7f132c0773f0 2024-04-25T19:02:04.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.385+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1a74c0 con 0x7f132c0773f0 2024-04-25T19:02:04.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.386+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c069730 con 0x7f132c0773f0 2024-04-25T19:02:04.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.388+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1320010430 con 0x7f132c0773f0 2024-04-25T19:02:04.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.389+0000 7f1329600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f13140a3c40 0x7f13140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.390+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f13200d0700 con 0x7f132c0773f0 2024-04-25T19:02:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.390+0000 7f132be00700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f13140a3c40 0x7f13140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.390+0000 7f132be00700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f13140a3c40 0x7f13140a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1318005d10 tx=0x7f1318009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:04.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.392+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f132009d020 con 0x7f132c0773f0 2024-04-25T19:02:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.626+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f132c062e80 con 0x7f13140a3c40 2024-04-25T19:02:04.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.629+0000 7f1329600700 1 -- 172.21.15.86:0/3916830934 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f132c062e80 con 0x7f13140a3c40 2024-04-25T19:02:04.631 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-04-25T19:02:04.632 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-04-25T19:02:04.632 INFO:teuthology.orchestra.run.smithi086.stdout:smithi183 172.21.15.183 2024-04-25T19:02:04.632 INFO:teuthology.orchestra.run.smithi086.stdout:smithi194 172.21.15.194 2024-04-25T19:02:04.632 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-04-25T19:02:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f13140a3c40 msgr2=0x7f13140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f13140a3c40 0x7f13140a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1318005d10 tx=0x7f1318009310 comp rx=0 tx=0).stop 2024-04-25T19:02:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 msgr2=0x7f132c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 0x7f132c19f0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f132000ea20 tx=0x7f132000ed30 comp rx=0 tx=0).stop 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 shutdown_connections 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f13140a3c40 0x7f13140a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f132c079000 0x7f132c1a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f132c0773f0 0x7f132c19f0d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 --2- 172.21.15.86:0/3916830934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f132c078640 0x7f132c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.631+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 >> 172.21.15.86:0/3916830934 conn(0x7f132c100250 msgr2=0x7f132c101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.632+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 shutdown_connections 2024-04-25T19:02:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:04.632+0000 7f1333600700 1 -- 172.21.15.86:0/3916830934 wait complete. 2024-04-25T19:02:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:04 smithi194 ceph-mon[32740]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:04 smithi086 ceph-mon[27783]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:04.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:04 smithi183 ceph-mon[32344]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:05.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T19:02:05.929 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:06 smithi194 ceph-mon[32740]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:06 smithi194 ceph-mon[32740]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:06 smithi086 ceph-mon[27783]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:06.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:06 smithi086 ceph-mon[27783]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:06.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:06 smithi183 ceph-mon[32344]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:06.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:06 smithi183 ceph-mon[32344]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:08.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 -- 172.21.15.86:0/2177060928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 msgr2=0x7f69f010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:08.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 --2- 172.21.15.86:0/2177060928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 0x7f69f010a740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009990 tx=0x7f69dc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 -- 172.21.15.86:0/2177060928 shutdown_connections 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 --2- 172.21.15.86:0/2177060928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f010ac80 0x7f69f0111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 --2- 172.21.15.86:0/2177060928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f01090e0 0x7f69f0109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 --2- 172.21.15.86:0/2177060928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 0x7f69f010a740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 -- 172.21.15.86:0/2177060928 >> 172.21.15.86:0/2177060928 conn(0x7f69f00780b0 msgr2=0x7f69f007a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.029+0000 7f69f5a00700 1 -- 172.21.15.86:0/2177060928 shutdown_connections 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.030+0000 7f69f5a00700 1 -- 172.21.15.86:0/2177060928 wait complete. 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 Processor -- start 2024-04-25T19:02:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 -- start start 2024-04-25T19:02:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 0x7f69f019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 0x7f69f019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f010ac80 0x7f69f01a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0114b50 con 0x7f69f01090e0 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01149d0 con 0x7f69f010ac80 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.031+0000 7f69f5a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0114cd0 con 0x7f69f010a2e0 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 0x7f69f019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 0x7f69f019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42586/0 (socket says 172.21.15.86:42586) 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 -- 172.21.15.86:0/871934418 learned_addr learned my addr 172.21.15.86:0/871934418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69ef400700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f010ac80 0x7f69f01a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 -- 172.21.15.86:0/871934418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 msgr2=0x7f69f019f6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 0x7f69f019f6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 -- 172.21.15.86:0/871934418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f010ac80 msgr2=0x7f69f01a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f010ac80 0x7f69f01a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 -- 172.21.15.86:0/871934418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f01a6df0 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69ef400700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f010ac80 0x7f69f01a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.032+0000 7f69eea00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 0x7f69f019f190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e9a0 tx=0x7f69e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.033+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000cbd0 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.033+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e00044d0 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.033+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0003d80 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.033+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01a70e0 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.033+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f01a7620 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.034+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f0069730 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.036+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f69e0003f20 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.037+0000 7f69e7400700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f69d40a3cc0 0x7f69d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.037+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f69e00d06f0 con 0x7f69f01090e0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.038+0000 7f69ee000700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f69d40a3cc0 0x7f69d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.038+0000 7f69ee000700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f69d40a3cc0 0x7f69d40a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b2e0 tx=0x7f69dc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.042+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69e009d010 con 0x7f69f01090e0 2024-04-25T19:02:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.283+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 --> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f69f0062e80 con 0x7f69d40a3cc0 2024-04-25T19:02:08.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.285+0000 7f69e7400700 1 -- 172.21.15.86:0/871934418 <== mgr.14152 v2:172.21.15.86:6800/237697797 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f69f0062e80 con 0x7f69d40a3cc0 2024-04-25T19:02:08.288 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T19:02:08.288 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-25T19:02:08.289 INFO:teuthology.orchestra.run.smithi086.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T19:02:08.289 INFO:teuthology.orchestra.run.smithi086.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T19:02:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f69d40a3cc0 msgr2=0x7f69d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f69d40a3cc0 0x7f69d40a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00b2e0 tx=0x7f69dc005a90 comp rx=0 tx=0).stop 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 msgr2=0x7f69f019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 0x7f69f019f190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e9a0 tx=0x7f69e000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 shutdown_connections 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f69d40a3cc0 0x7f69d40a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f010ac80 0x7f69f01a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69f01090e0 0x7f69f019f190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 --2- 172.21.15.86:0/871934418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69f010a2e0 0x7f69f019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 >> 172.21.15.86:0/871934418 conn(0x7f69f00780b0 msgr2=0x7f69f010d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 shutdown_connections 2024-04-25T19:02:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:08.288+0000 7f69f5a00700 1 -- 172.21.15.86:0/871934418 wait complete. 2024-04-25T19:02:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:08 smithi086 ceph-mon[27783]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:02:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:08 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:08 smithi183 ceph-mon[32344]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:02:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:08 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:08 smithi194 ceph-mon[32740]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T19:02:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:08 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:09.084 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T19:02:09.093 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:02:09.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T19:02:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:09.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:09.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:02:09.798 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:09 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:02:09.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:02:09.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:09 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:02:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:09.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:09.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:02:09.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:09 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:02:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:10 smithi086 ceph-mon[27783]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:10 smithi086 ceph-mon[27783]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:10 smithi183 ceph-mon[32344]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:10 smithi183 ceph-mon[32344]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:10 smithi194 ceph-mon[32740]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:10 smithi194 ceph-mon[32740]: from='mgr.14152 172.21.15.86:0/2300239499' entity='mgr.x' 2024-04-25T19:02:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 -- 172.21.15.86:0/2596491245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0105ef0 msgr2=0x7f09a0106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 --2- 172.21.15.86:0/2596491245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0105ef0 0x7f09a0106350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f098c009990 tx=0x7f098c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 -- 172.21.15.86:0/2596491245 shutdown_connections 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 --2- 172.21.15.86:0/2596491245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0106890 0x7f09a010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 --2- 172.21.15.86:0/2596491245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 0x7f09a0105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 --2- 172.21.15.86:0/2596491245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0105ef0 0x7f09a0106350 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 -- 172.21.15.86:0/2596491245 >> 172.21.15.86:0/2596491245 conn(0x7f09a0100290 msgr2=0x7f09a01026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 -- 172.21.15.86:0/2596491245 shutdown_connections 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.658+0000 7f09a6000700 1 -- 172.21.15.86:0/2596491245 wait complete. 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.659+0000 7f09a6000700 1 Processor -- start 2024-04-25T19:02:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.659+0000 7f09a6000700 1 -- start start 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.659+0000 7f09a6000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 0x7f09a019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.659+0000 7f09a6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 0x7f09a019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.659+0000 7f09a6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0106890 0x7f09a01a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f09a6000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01107b0 con 0x7f09a0105ef0 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f09a6000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0110630 con 0x7f09a0104cf0 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f09a6000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0110930 con 0x7f09a0106890 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 0x7f09a019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 0x7f09a019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0106890 0x7f09a01a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 0x7f09a019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36044/0 (socket says 172.21.15.86:36044) 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 -- 172.21.15.86:0/3147289126 learned_addr learned my addr 172.21.15.86:0/3147289126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 -- 172.21.15.86:0/3147289126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0106890 msgr2=0x7f09a01a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:11.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0106890 0x7f09a01a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 -- 172.21.15.86:0/3147289126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 msgr2=0x7f09a019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 0x7f09a019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099ea00700 1 -- 172.21.15.86:0/3147289126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a01a6e40 con 0x7f09a0105ef0 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099fe00700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0106890 0x7f09a01a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.660+0000 7f099f400700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 0x7f09a019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.661+0000 7f099ea00700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 0x7f09a019f720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f098c000c00 tx=0x7f098c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.661+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c026070 con 0x7f09a0105ef0 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.661+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c021410 con 0x7f09a0105ef0 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.661+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c017650 con 0x7f09a0105ef0 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.661+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a01a70d0 con 0x7f09a0105ef0 2024-04-25T19:02:11.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.661+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a01a74d0 con 0x7f09a0105ef0 2024-04-25T19:02:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.663+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a0069730 con 0x7f09a0105ef0 2024-04-25T19:02:11.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.665+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f098c009d10 con 0x7f09a0105ef0 2024-04-25T19:02:11.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.666+0000 7f0997e00700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f09840a3c40 0x7f09840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:11.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.666+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f098c0d1270 con 0x7f09a0105ef0 2024-04-25T19:02:11.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.667+0000 7f099f400700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f09840a3c40 0x7f09840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:11.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.667+0000 7f099f400700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f09840a3c40 0x7f09840a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09a0105d50 tx=0x7f0990009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:11.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.669+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f098c013070 con 0x7f09a0105ef0 2024-04-25T19:02:11.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.937+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f09a01a79b0 con 0x7f09a0105ef0 2024-04-25T19:02:11.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.938+0000 7f0997e00700 1 -- 172.21.15.86:0/3147289126 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3094 (secure 0 0 0) 0x7f098c09dc20 con 0x7f09a0105ef0 2024-04-25T19:02:11.940 INFO:teuthology.orchestra.run.smithi086.stdout:e28 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 28 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-25T18:58:51.446663+0000 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-25T18:59:15.320078+0000 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-04-25T19:02:11.941 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=24527,1=24509,2=24494,3=24493,4=24478} 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3] 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-04-25T19:02:11.942 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:qdb_cluster leader: 24527 members: 24478,24527,24493,24494,24509 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:24527} state up:active seq 7 addr [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{0:24473} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/4174239962,v1:172.21.15.183:6835/4174239962] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{1:24509} state up:active seq 13 addr [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{1:24472} state up:standby-replay seq 1 addr [v2:172.21.15.194:6834/425079996,v1:172.21.15.194:6835/425079996] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{2:24494} state up:active seq 15 addr [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{2:24461} state up:standby-replay seq 1 addr [v2:172.21.15.86:6836/3883478130,v1:172.21.15.86:6837/3883478130] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{3:24493} state up:active seq 14 addr [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{3:24443} state up:standby-replay seq 1 addr [v2:172.21.15.183:6832/2135214296,v1:172.21.15.183:6833/2135214296] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{4:24478} state up:active seq 16 addr [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{4:24430} state up:standby-replay seq 1 addr [v2:172.21.15.194:6832/45783241,v1:172.21.15.194:6833/45783241] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:11.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-04-25T19:02:11.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14361} state up:standby seq 1 addr [v2:172.21.15.86:6834/3780909923,v1:172.21.15.86:6835/3780909923] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.944 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{-1:14409} state up:standby seq 1 addr [v2:172.21.15.86:6840/2332559651,v1:172.21.15.86:6841/2332559651] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.941+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f09840a3c40 msgr2=0x7f09840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.941+0000 7f09a6000700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f09840a3c40 0x7f09840a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09a0105d50 tx=0x7f0990009260 comp rx=0 tx=0).stop 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.941+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 msgr2=0x7f09a019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.941+0000 7f09a6000700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 0x7f09a019f720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f098c000c00 tx=0x7f098c0057c0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 shutdown_connections 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f09840a3c40 0x7f09840a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09a0106890 0x7f09a01a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a0104cf0 0x7f09a019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 --2- 172.21.15.86:0/3147289126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a0105ef0 0x7f09a019f720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:11.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 >> 172.21.15.86:0/3147289126 conn(0x7f09a0100290 msgr2=0x7f09a0078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:11.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 shutdown_connections 2024-04-25T19:02:11.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:11.942+0000 7f09a6000700 1 -- 172.21.15.86:0/3147289126 wait complete. 2024-04-25T19:02:11.946 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T19:02:12.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:12 smithi086 ceph-mon[27783]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:12.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:12 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3147289126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T19:02:12.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T19:02:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:12 smithi183 ceph-mon[32344]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:12 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3147289126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T19:02:12.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:12 smithi194 ceph-mon[32740]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:12.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:12 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3147289126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T19:02:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:14 smithi086 ceph-mon[27783]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:14 smithi183 ceph-mon[32344]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:14 smithi194 ceph-mon[32740]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 -- 172.21.15.86:0/1329509040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c105ee0 msgr2=0x7f4b6c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 --2- 172.21.15.86:0/1329509040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c106340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b580099c0 tx=0x7f4b58009cd0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 -- 172.21.15.86:0/1329509040 shutdown_connections 2024-04-25T19:02:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 --2- 172.21.15.86:0/1329509040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c106880 0x7f4b6c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 --2- 172.21.15.86:0/1329509040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 0x7f4b6c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 --2- 172.21.15.86:0/1329509040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c106340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 -- 172.21.15.86:0/1329509040 >> 172.21.15.86:0/1329509040 conn(0x7f4b6c1002a0 msgr2=0x7f4b6c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 -- 172.21.15.86:0/1329509040 shutdown_connections 2024-04-25T19:02:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.425+0000 7f4b71200700 1 -- 172.21.15.86:0/1329509040 wait complete. 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.426+0000 7f4b71200700 1 Processor -- start 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.426+0000 7f4b71200700 1 -- start start 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.426+0000 7f4b71200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 0x7f4b6c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.426+0000 7f4b71200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b71200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c106880 0x7f4b6c1a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b71200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1107a0 con 0x7f4b6c106880 2024-04-25T19:02:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b71200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c110620 con 0x7f4b6c105ee0 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b71200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c110920 con 0x7f4b6c104ce0 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 0x7f4b6c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b6a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c106880 0x7f4b6c1a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:39312/0 (socket says 172.21.15.86:39312) 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 -- 172.21.15.86:0/3005094212 learned_addr learned my addr 172.21.15.86:0/3005094212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 0x7f4b6c19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:52650/0 (socket says 172.21.15.86:52650) 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 -- 172.21.15.86:0/3005094212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 msgr2=0x7f4b6c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 0x7f4b6c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 -- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c106880 msgr2=0x7f4b6c1a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c106880 0x7f4b6c1a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.427+0000 7f4b69400700 1 -- 172.21.15.86:0/3005094212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1a2a40 con 0x7f4b6c105ee0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.428+0000 7f4b69400700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c19b320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b580036f0 tx=0x7f4b58003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.428+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58009e60 con 0x7f4b6c105ee0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.428+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5800be50 con 0x7f4b6c105ee0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.428+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1a2cd0 con 0x7f4b6c105ee0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.428+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800fda0 con 0x7f4b6c105ee0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.428+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a3180 con 0x7f4b6c105ee0 2024-04-25T19:02:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.429+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c069730 con 0x7f4b6c105ee0 2024-04-25T19:02:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.431+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b5800f460 con 0x7f4b6c105ee0 2024-04-25T19:02:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.433+0000 7f4b62a00700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b500a3cc0 0x7f4b500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.433+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4b580d12d0 con 0x7f4b6c105ee0 2024-04-25T19:02:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.433+0000 7f4b69e00700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b500a3cc0 0x7f4b500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.434+0000 7f4b69e00700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b500a3cc0 0x7f4b500a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c105d40 tx=0x7f4b5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.435+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b58013070 con 0x7f4b6c105ee0 2024-04-25T19:02:15.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.665+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4b6c02e090 con 0x7f4b6c105ee0 2024-04-25T19:02:15.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.667+0000 7f4b62a00700 1 -- 172.21.15.86:0/3005094212 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5392 (secure 0 0 0) 0x7f4b5809dc80 con 0x7f4b6c105ee0 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 93 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-25T18:49:23.294231+0000 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-25T19:00:38.512884+0000 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 26 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-04-25T19:02:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release squid 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/2684261173,v1:172.21.15.86:6803/2684261173] [v2:172.21.15.86:6804/2684261173,v1:172.21.15.86:6805/2684261173] exists,up 983ae9a3-f0a7-436b-996b-e9916864d30e 2024-04-25T19:02:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/4147786843,v1:172.21.15.183:6801/4147786843] [v2:172.21.15.183:6802/4147786843,v1:172.21.15.183:6803/4147786843] exists,up cf82beae-9a79-46f2-9ea8-59032823bc00 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/1582539385,v1:172.21.15.194:6801/1582539385] [v2:172.21.15.194:6802/1582539385,v1:172.21.15.194:6803/1582539385] exists,up 768f0112-52c8-452e-b56c-1d19c065d486 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/2203765843,v1:172.21.15.86:6811/2203765843] [v2:172.21.15.86:6812/2203765843,v1:172.21.15.86:6813/2203765843] exists,up a91c123c-a363-482a-9d1a-c6eb6f121b33 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/3004862386,v1:172.21.15.183:6809/3004862386] [v2:172.21.15.183:6810/3004862386,v1:172.21.15.183:6811/3004862386] exists,up deb085ac-6e63-4429-a71a-e9e35e2c27f4 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6808/1967519671,v1:172.21.15.194:6809/1967519671] [v2:172.21.15.194:6810/1967519671,v1:172.21.15.194:6811/1967519671] exists,up 74a891b6-ac08-4b3a-bd2a-5b8d735a77de 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/1838185370,v1:172.21.15.86:6819/1838185370] [v2:172.21.15.86:6820/1838185370,v1:172.21.15.86:6821/1838185370] exists,up 7b207644-8c34-4045-88b6-7cb304dcccec 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 52 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/2750054847,v1:172.21.15.183:6817/2750054847] [v2:172.21.15.183:6818/2750054847,v1:172.21.15.183:6819/2750054847] exists,up 75d0252d-dd63-4aa7-982b-bd260501a0e1 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 60 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6816/1969375394,v1:172.21.15.194:6817/1969375394] [v2:172.21.15.194:6818/1969375394,v1:172.21.15.194:6819/1969375394] exists,up dcd61ecf-016d-40fd-8a13-e4aa1828d49e 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 68 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/3110000530,v1:172.21.15.86:6827/3110000530] [v2:172.21.15.86:6828/3110000530,v1:172.21.15.86:6829/3110000530] exists,up 8006e686-e97b-4cda-a577-6b8dc8f9d5f7 2024-04-25T19:02:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 73 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/291756027,v1:172.21.15.183:6825/291756027] [v2:172.21.15.183:6826/291756027,v1:172.21.15.183:6827/291756027] exists,up 5f7367bb-2998-47a7-ae18-fd713be7d07e 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 81 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6824/745573568,v1:172.21.15.194:6825/745573568] [v2:172.21.15.194:6826/745573568,v1:172.21.15.194:6827/745573568] exists,up dc1df1f5-bf92-4d45-8c5d-e80fef77cccf 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.16 [2,4] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.1d [2,4,8,11] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.0 [5,9] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.e [5,9] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.14 [4,6] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.17 [4,6] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.19 [3,8] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.1b [3,8] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.37 [1,9] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.3a [4,8] 2024-04-25T19:02:15.672 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1094517747 expires 2024-04-26T18:50:37.175300+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/378785417 expires 2024-04-26T18:50:37.175300+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/4264123395 expires 2024-04-26T18:50:37.175300+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3778539307 expires 2024-04-26T18:50:04.542688+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/73406505 expires 2024-04-26T18:50:04.542688+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/777604946 expires 2024-04-26T18:50:04.542688+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/777604946 expires 2024-04-26T18:50:04.542688+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2055127838 expires 2024-04-26T18:50:37.175300+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3781450984 expires 2024-04-26T18:50:04.542688+0000 2024-04-25T19:02:15.673 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/4264123395 expires 2024-04-26T18:50:37.175300+0000 2024-04-25T19:02:15.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b500a3cc0 msgr2=0x7f4b500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b500a3cc0 0x7f4b500a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c105d40 tx=0x7f4b5c009260 comp rx=0 tx=0).stop 2024-04-25T19:02:15.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 msgr2=0x7f4b6c19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:15.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c19b320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b580036f0 tx=0x7f4b58003e60 comp rx=0 tx=0).stop 2024-04-25T19:02:15.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 shutdown_connections 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4b500a3cc0 0x7f4b500a6180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c106880 0x7f4b6c1a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b6c104ce0 0x7f4b6c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 --2- 172.21.15.86:0/3005094212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c105ee0 0x7f4b6c19b320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 >> 172.21.15.86:0/3005094212 conn(0x7f4b6c1002a0 msgr2=0x7f4b6c109260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 shutdown_connections 2024-04-25T19:02:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:15.670+0000 7f4b71200700 1 -- 172.21.15.86:0/3005094212 wait complete. 2024-04-25T19:02:16.528 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T19:02:16.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:02:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:16 smithi086 ceph-mon[27783]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:16 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3005094212' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T19:02:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:16 smithi183 ceph-mon[32344]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:16 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3005094212' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T19:02:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:16 smithi194 ceph-mon[32740]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:16 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3005094212' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T19:02:16.985 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:18.672 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:18 smithi086 ceph-mon[27783]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:18 smithi183 ceph-mon[32344]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:18 smithi194 ceph-mon[32740]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:19.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 -- 172.21.15.86:0/1845784153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc109c80 msgr2=0x7f86cc10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 --2- 172.21.15.86:0/1845784153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc109c80 0x7f86cc10dcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009920 tx=0x7f86bc009c30 comp rx=0 tx=0).stop 2024-04-25T19:02:19.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 -- 172.21.15.86:0/1845784153 shutdown_connections 2024-04-25T19:02:19.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 --2- 172.21.15.86:0/1845784153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc10e3b0 0x7f86cc115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 --2- 172.21.15.86:0/1845784153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86cc1092d0 0x7f86cc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 --2- 172.21.15.86:0/1845784153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc109c80 0x7f86cc10dcd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 -- 172.21.15.86:0/1845784153 >> 172.21.15.86:0/1845784153 conn(0x7f86cc076210 msgr2=0x7f86cc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:19.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.189+0000 7f86d3c00700 1 -- 172.21.15.86:0/1845784153 shutdown_connections 2024-04-25T19:02:19.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.190+0000 7f86d3c00700 1 -- 172.21.15.86:0/1845784153 wait complete. 2024-04-25T19:02:19.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 Processor -- start 2024-04-25T19:02:19.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 -- start start 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 0x7f86cc1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86cc109c80 0x7f86cc1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc10e3b0 0x7f86cc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc119270 con 0x7f86cc1092d0 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc1190f0 con 0x7f86cc109c80 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.191+0000 7f86d3c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc1193f0 con 0x7f86cc10e3b0 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 0x7f86cc1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 0x7f86cc1a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36080/0 (socket says 172.21.15.86:36080) 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1200700 1 -- 172.21.15.86:0/255208821 learned_addr learned my addr 172.21.15.86:0/255208821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc10e3b0 0x7f86cc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86cbe00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86cc109c80 0x7f86cc1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1c00700 1 -- 172.21.15.86:0/255208821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86cc109c80 msgr2=0x7f86cc1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86cc109c80 0x7f86cc1a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1c00700 1 -- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 msgr2=0x7f86cc1a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 0x7f86cc1a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1c00700 1 -- 172.21.15.86:0/255208821 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc19ff50 con 0x7f86cc10e3b0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.192+0000 7f86d1200700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 0x7f86cc1a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.193+0000 7f86d1c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc10e3b0 0x7f86cc19f670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009cd0 tx=0x7f86c400c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.193+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c400cd10 con 0x7f86cc10e3b0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.193+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c4003a00 con 0x7f86cc10e3b0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.193+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc1a01e0 con 0x7f86cc10e3b0 2024-04-25T19:02:19.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.193+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4004520 con 0x7f86cc10e3b0 2024-04-25T19:02:19.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.193+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1ac520 con 0x7f86cc10e3b0 2024-04-25T19:02:19.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.194+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86cc10a7b0 con 0x7f86cc10e3b0 2024-04-25T19:02:19.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.196+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f86c401a020 con 0x7f86cc10e3b0 2024-04-25T19:02:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.198+0000 7f86c9600700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f86b40a3cc0 0x7f86b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.198+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f86c40cfc80 con 0x7f86cc10e3b0 2024-04-25T19:02:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.198+0000 7f86d1200700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f86b40a3cc0 0x7f86b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.198+0000 7f86d1200700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f86b40a3cc0 0x7f86b40a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86c0011550 tx=0x7f86c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.202+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86c4010040 con 0x7f86cc10e3b0 2024-04-25T19:02:19.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.479+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86cc1a0520 con 0x7f86cc10e3b0 2024-04-25T19:02:19.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.481+0000 7f86c9600700 1 -- 172.21.15.86:0/255208821 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11937 (secure 0 0 0) 0x7f86c409c5a0 con 0x7f86cc10e3b0 2024-04-25T19:02:19.483 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:19.483 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T18:59:15.320078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24478,24527,24493,24494,24509]},"id":1}]} 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f86b40a3cc0 msgr2=0x7f86b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f86b40a3cc0 0x7f86b40a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86c0011550 tx=0x7f86c0009260 comp rx=0 tx=0).stop 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc10e3b0 msgr2=0x7f86cc19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc10e3b0 0x7f86cc19f670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009cd0 tx=0x7f86c400c620 comp rx=0 tx=0).stop 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 shutdown_connections 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f86b40a3cc0 0x7f86b40a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86cc10e3b0 0x7f86cc19f670 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86cc1092d0 0x7f86cc1a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 --2- 172.21.15.86:0/255208821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86cc109c80 0x7f86cc1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:19.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 >> 172.21.15.86:0/255208821 conn(0x7f86cc076210 msgr2=0x7f86cc1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 shutdown_connections 2024-04-25T19:02:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:19.484+0000 7f86d3c00700 1 -- 172.21.15.86:0/255208821 wait complete. 2024-04-25T19:02:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T19:02:19.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:19 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/255208821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:19 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/255208821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:19.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:19 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/255208821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:20.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:02:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:20 smithi086 ceph-mon[27783]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:20.883 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:20 smithi194 ceph-mon[32740]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:20 smithi183 ceph-mon[32344]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:22.809 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:22 smithi086 ceph-mon[27783]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:22 smithi183 ceph-mon[32344]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 -- 172.21.15.86:0/4275736284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 msgr2=0x7f82300751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 --2- 172.21.15.86:0/4275736284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 0x7f82300751f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82240098f0 tx=0x7f8224009c00 comp rx=0 tx=0).stop 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 -- 172.21.15.86:0/4275736284 shutdown_connections 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 --2- 172.21.15.86:0/4275736284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823010f200 0x7f8230111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 --2- 172.21.15.86:0/4275736284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 0x7f82300751f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 --2- 172.21.15.86:0/4275736284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 -- 172.21.15.86:0/4275736284 >> 172.21.15.86:0/4275736284 conn(0x7f82300fe490 msgr2=0x7f82301008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 -- 172.21.15.86:0/4275736284 shutdown_connections 2024-04-25T19:02:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.977+0000 7f8236800700 1 -- 172.21.15.86:0/4275736284 wait complete. 2024-04-25T19:02:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.978+0000 7f8236800700 1 Processor -- start 2024-04-25T19:02:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 -- start start 2024-04-25T19:02:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 0x7f823019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823010f200 0x7f823019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8230114d90 con 0x7f823010f200 2024-04-25T19:02:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8230114c10 con 0x7f8230075730 2024-04-25T19:02:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.979+0000 7f8236800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8230114f10 con 0x7f8230074e10 2024-04-25T19:02:22.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:22 smithi194 ceph-mon[32740]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8235e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823010f200 0x7f823019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823019b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:34412/0 (socket says 172.21.15.86:34412) 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 -- 172.21.15.86:0/865712814 learned_addr learned my addr 172.21.15.86:0/865712814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8235400700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 0x7f823019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 -- 172.21.15.86:0/865712814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 msgr2=0x7f823019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 0x7f823019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 -- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823010f200 msgr2=0x7f823019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823010f200 0x7f823019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.980+0000 7f8234a00700 1 -- 172.21.15.86:0/865712814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82301a02e0 con 0x7f8230075730 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.981+0000 7f8234a00700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823019b860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9a0 tx=0x7f821c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.981+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00ca10 con 0x7f8230075730 2024-04-25T19:02:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.981+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c00cbb0 con 0x7f8230075730 2024-04-25T19:02:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.981+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82301a05d0 con 0x7f8230075730 2024-04-25T19:02:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.981+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c018680 con 0x7f8230075730 2024-04-25T19:02:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.981+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82301a7e50 con 0x7f8230075730 2024-04-25T19:02:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.982+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8230069730 con 0x7f8230075730 2024-04-25T19:02:22.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.984+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f821c010430 con 0x7f8230075730 2024-04-25T19:02:22.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.985+0000 7f822e000700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f82180a3c40 0x7f82180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:22.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.986+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f821c0d06d0 con 0x7f8230075730 2024-04-25T19:02:22.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.986+0000 7f8235400700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f82180a3c40 0x7f82180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:22.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.986+0000 7f8235400700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f82180a3c40 0x7f82180a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82240098c0 tx=0x7f8224019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:22.988+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f821c09cff0 con 0x7f8230075730 2024-04-25T19:02:23.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.262+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f823002df80 con 0x7f8230075730 2024-04-25T19:02:23.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.263+0000 7f822e000700 1 -- 172.21.15.86:0/865712814 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11937 (secure 0 0 0) 0x7f821c09c8f0 con 0x7f8230075730 2024-04-25T19:02:23.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:23.265 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T18:59:15.320078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24478,24527,24493,24494,24509]},"id":1}]} 2024-04-25T19:02:23.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f82180a3c40 msgr2=0x7f82180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:23.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f82180a3c40 0x7f82180a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82240098c0 tx=0x7f8224019040 comp rx=0 tx=0).stop 2024-04-25T19:02:23.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 msgr2=0x7f823019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:23.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823019b860 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9a0 tx=0x7f821c00ecb0 comp rx=0 tx=0).stop 2024-04-25T19:02:23.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 shutdown_connections 2024-04-25T19:02:23.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f82180a3c40 0x7f82180a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823010f200 0x7f823019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8230074e10 0x7f823019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 --2- 172.21.15.86:0/865712814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8230075730 0x7f823019b860 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 >> 172.21.15.86:0/865712814 conn(0x7f82300fe490 msgr2=0x7f82300ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 shutdown_connections 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:23.266+0000 7f8236800700 1 -- 172.21.15.86:0/865712814 wait complete. 2024-04-25T19:02:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-25T19:02:23.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:23 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/865712814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:23.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:23 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/865712814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:23.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:23 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/865712814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:24.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T19:02:24.524 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:24 smithi086 ceph-mon[27783]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:24 smithi183 ceph-mon[32344]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:24 smithi194 ceph-mon[32740]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 -- 172.21.15.86:0/433898564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 msgr2=0x7fc42c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 --2- 172.21.15.86:0/433898564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 0x7fc42c115f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc42400b0a0 tx=0x7fc42400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 -- 172.21.15.86:0/433898564 shutdown_connections 2024-04-25T19:02:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 --2- 172.21.15.86:0/433898564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 0x7fc42c115f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 --2- 172.21.15.86:0/433898564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 0x7fc42c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 --2- 172.21.15.86:0/433898564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 -- 172.21.15.86:0/433898564 >> 172.21.15.86:0/433898564 conn(0x7fc42c076210 msgr2=0x7fc42c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.727+0000 7fc430a00700 1 -- 172.21.15.86:0/433898564 shutdown_connections 2024-04-25T19:02:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.728+0000 7fc430a00700 1 -- 172.21.15.86:0/433898564 wait complete. 2024-04-25T19:02:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 Processor -- start 2024-04-25T19:02:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 -- start start 2024-04-25T19:02:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 0x7fc42c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 0x7fc42c1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1192c0 con 0x7fc42c1092d0 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c119140 con 0x7fc42c10e3b0 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc430a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c119440 con 0x7fc42c109c80 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.729+0000 7fc429e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 0x7fc42c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34526/0 (socket says 172.21.15.86:34526) 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc42a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 0x7fc42c1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 -- 172.21.15.86:0/2563779435 learned_addr learned my addr 172.21.15.86:0/2563779435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 0x7fc42c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55590/0 (socket says 172.21.15.86:55590) 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 -- 172.21.15.86:0/2563779435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 msgr2=0x7fc42c1a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 0x7fc42c1a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 -- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 msgr2=0x7fc42c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 0x7fc42c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 -- 172.21.15.86:0/2563779435 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a4720 con 0x7fc42c109c80 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.730+0000 7fc429400700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c19fc50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e9f0 tx=0x7fc41800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.731+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418004a30 con 0x7fc42c109c80 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.731+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a4a10 con 0x7fc42c109c80 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.731+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41800cc70 con 0x7fc42c109c80 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.731+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418005530 con 0x7fc42c109c80 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.731+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1ac290 con 0x7fc42c109c80 2024-04-25T19:02:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.732+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c069730 con 0x7fc42c109c80 2024-04-25T19:02:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.734+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc418004bd0 con 0x7fc42c109c80 2024-04-25T19:02:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.735+0000 7fc422a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4100a3c40 0x7fc4100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.735+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fc4180d09c0 con 0x7fc42c109c80 2024-04-25T19:02:26.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.736+0000 7fc429e00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4100a3c40 0x7fc4100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.736+0000 7fc429e00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4100a3c40 0x7fc4100a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009c60 tx=0x7fc41c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.738+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc41809d2e0 con 0x7fc42c109c80 2024-04-25T19:02:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:26 smithi086 ceph-mon[27783]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:26.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:26 smithi183 ceph-mon[32344]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:26.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.963+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc42c02df80 con 0x7fc42c109c80 2024-04-25T19:02:26.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.966+0000 7fc422a00700 1 -- 172.21.15.86:0/2563779435 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22867 (secure 0 0 0) 0x7fc41809cbe0 con 0x7fc42c109c80 2024-04-25T19:02:26.968 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:02:26.968 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T19:00:38.512884+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4100a3c40 msgr2=0x7fc4100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4100a3c40 0x7fc4100a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009c60 tx=0x7fc41c009260 comp rx=0 tx=0).stop 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 msgr2=0x7fc42c19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c19fc50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e9f0 tx=0x7fc41800ed00 comp rx=0 tx=0).stop 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 shutdown_connections 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fc4100a3c40 0x7fc4100a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc42c10e3b0 0x7fc42c1a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c1092d0 0x7fc42c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 --2- 172.21.15.86:0/2563779435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc42c109c80 0x7fc42c19fc50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 >> 172.21.15.86:0/2563779435 conn(0x7fc42c076210 msgr2=0x7fc42c076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 shutdown_connections 2024-04-25T19:02:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:26.969+0000 7fc430a00700 1 -- 172.21.15.86:0/2563779435 wait complete. 2024-04-25T19:02:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:26 smithi194 ceph-mon[32740]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:27.704 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:27 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2563779435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:02:27.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:58:51.446663+0000', 'modified': '2024-04-25T18:59:15.320078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24509, 'mds_2': 24494, 'mds_3': 24493, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/45783241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 45783241}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 45783241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2135214296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2135214296}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2135214296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/3883478130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3883478130}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3883478130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/425079996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 425079996}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 425079996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/4174239962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 4174239962}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 4174239962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.194:6837/3328875881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3328875881}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3328875881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.194:6839/2085744336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 2085744336}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 2085744336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.183:6837/3273542947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3273542947}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3273542947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.86:6839/3676043902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3676043902}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3676043902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.183:6839/703487979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 703487979}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 703487979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24478, 24527, 24493, 24494, 24509]} 2024-04-25T19:02:27.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T19:02:27.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-25T19:02:27.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:27 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2563779435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:02:27.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:27 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2563779435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:02:28.230 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:28.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:28 smithi086 ceph-mon[27783]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:28 smithi183 ceph-mon[32344]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:28 smithi194 ceph-mon[32740]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 -- 172.21.15.86:0/2456214143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 msgr2=0x7f4588111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 --2- 172.21.15.86:0/2456214143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 0x7f4588111a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4580009f90 tx=0x7f458000b330 comp rx=0 tx=0).stop 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 -- 172.21.15.86:0/2456214143 shutdown_connections 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 --2- 172.21.15.86:0/2456214143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 0x7f4588111a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 --2- 172.21.15.86:0/2456214143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 0x7f4588105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 --2- 172.21.15.86:0/2456214143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 0x7f45881097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 -- 172.21.15.86:0/2456214143 >> 172.21.15.86:0/2456214143 conn(0x7f4588100680 msgr2=0x7f4588102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 -- 172.21.15.86:0/2456214143 shutdown_connections 2024-04-25T19:02:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.429+0000 7f458d200700 1 -- 172.21.15.86:0/2456214143 wait complete. 2024-04-25T19:02:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.431+0000 7f458d200700 1 Processor -- start 2024-04-25T19:02:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.431+0000 7f458d200700 1 -- start start 2024-04-25T19:02:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.431+0000 7f458d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 0x7f458819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.431+0000 7f458d200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 0x7f458819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.431+0000 7f458d200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 0x7f458819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f458d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4588114d40 con 0x7f4588104db0 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f458d200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4588114bc0 con 0x7f4588109e90 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f458d200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4588114ec0 con 0x7f4588105760 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f4587400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 0x7f458819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f457fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 0x7f458819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f4587400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 0x7f458819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33232/0 (socket says 172.21.15.86:33232) 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f4587400700 1 -- 172.21.15.86:0/790017333 learned_addr learned my addr 172.21.15.86:0/790017333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f4587e00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 0x7f458819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f457fe00700 1 -- 172.21.15.86:0/790017333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 msgr2=0x7f458819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f457fe00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 0x7f458819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f457fe00700 1 -- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 msgr2=0x7f458819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f457fe00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 0x7f458819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.432+0000 7f457fe00700 1 -- 172.21.15.86:0/790017333 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45881a0260 con 0x7f4588105760 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.433+0000 7f457fe00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 0x7f458819b790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f457000e9c0 tx=0x7f457000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.433+0000 7f4584c00700 1 -- 172.21.15.86:0/790017333 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570004220 con 0x7f4588105760 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.433+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45881a0550 con 0x7f4588105760 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.433+0000 7f4584c00700 1 -- 172.21.15.86:0/790017333 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45700043c0 con 0x7f4588105760 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.433+0000 7f4584c00700 1 -- 172.21.15.86:0/790017333 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570005020 con 0x7f4588105760 2024-04-25T19:02:30.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.433+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45881a7dd0 con 0x7f4588105760 2024-04-25T19:02:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.437+0000 7f4584c00700 1 -- 172.21.15.86:0/790017333 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4570004750 con 0x7f4588105760 2024-04-25T19:02:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 Processor -- start 2024-04-25T19:02:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 -- start start 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4584c00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f456c0a3c40 0x7f456c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0ef27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 0x5574b0efd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0f00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574b0e45e40 con 0x5574b0efdc40 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574b0dec6e0 con 0x5574b0ef2d00 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4599176b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574b0e41e00 con 0x5574b0ef23e0 2024-04-25T19:02:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.438+0000 7f4584c00700 1 -- 172.21.15.86:0/790017333 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f45700d03a0 con 0x7f4588105760 2024-04-25T19:02:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587e00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f456c0a3c40 0x7f456c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f457fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 0x5574b0efd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0ef27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0f00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f457fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 0x5574b0efd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37308/0 (socket says 172.21.15.86:37308) 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0ef27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41120/0 (socket says 172.21.15.86:41120) 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f457fe00700 1 -- 172.21.15.86:0/1977089942 learned_addr learned my addr 172.21.15.86:0/1977089942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 -- 172.21.15.86:0/1977089942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 msgr2=0x5574b0efd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 0x5574b0efd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 -- 172.21.15.86:0/1977089942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 msgr2=0x5574b0f00030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0f00030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f4587400700 1 -- 172.21.15.86:0/1977089942 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b0de2630 con 0x5574b0ef23e0 2024-04-25T19:02:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.439+0000 7f457fe00700 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 0x5574b0efd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.440+0000 7f4587400700 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0ef27c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4578009d40 tx=0x7f4578011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef6d3e481c76a08a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.440+0000 7f4587e00700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f456c0a3c40 0x7f456c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f45800060d0 tx=0x7f45800093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.440+0000 7f457cc00700 1 -- 172.21.15.86:0/1977089942 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578004250 con 0x5574b0ef23e0 2024-04-25T19:02:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.440+0000 7f457cc00700 1 -- 172.21.15.86:0/1977089942 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45780043f0 con 0x5574b0ef23e0 2024-04-25T19:02:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.440+0000 7f457cc00700 1 -- 172.21.15.86:0/1977089942 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578019750 con 0x5574b0ef23e0 2024-04-25T19:02:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 -- 172.21.15.86:0/1977089942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 msgr2=0x5574b0ef27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0ef27c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4578009d40 tx=0x7f4578011720 comp rx=0 tx=0).stop 2024-04-25T19:02:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 -- 172.21.15.86:0/1977089942 shutdown_connections 2024-04-25T19:02:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0f00030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0ef27c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 --2- 172.21.15.86:0/1977089942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef2d00 0x5574b0efd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 -- 172.21.15.86:0/1977089942 >> 172.21.15.86:0/1977089942 conn(0x5574b0e21550 msgr2=0x5574b0eeea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 -- 172.21.15.86:0/1977089942 shutdown_connections 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 -- 172.21.15.86:0/1977089942 wait complete. 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.441+0000 7f4599176b80 1 -- start start 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 Processor -- start 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef23e0 0x5574b0efd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0efabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f007c0 con 0x5574b0efdc40 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574b0dff810 con 0x5574b0ef23e0 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f00940 con 0x5574b0ef2d00 2024-04-25T19:02:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.442+0000 7f4599176b80 10 client.0 _ll_register_callbacks cb 0x5574b0dde6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f4587e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0efabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41128/0 (socket says 172.21.15.86:41128) 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 -- 172.21.15.86:0/2361883316 learned_addr learned my addr 172.21.15.86:0/2361883316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f4587400700 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef23e0 0x5574b0efd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 -- 172.21.15.86:0/2361883316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef23e0 msgr2=0x5574b0efd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef23e0 0x5574b0efd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 -- 172.21.15.86:0/2361883316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 msgr2=0x5574b0efabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0efabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 -- 172.21.15.86:0/2361883316 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b0d93880 con 0x5574b0ef2d00 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f457fe00700 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efa690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f457000f010 tx=0x7f457001fd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f7232cda72fae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.443+0000 7f4575600700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4575600700 1 -- 172.21.15.86:0/2361883316 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001c040 con 0x5574b0ef2d00 2024-04-25T19:02:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4575600700 1 -- 172.21.15.86:0/2361883316 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f45700cdef0 con 0x5574b0ef2d00 2024-04-25T19:02:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45780043f0 con 0x5574b0ef2d00 2024-04-25T19:02:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4575600700 1 -- 172.21.15.86:0/2361883316 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4570017ce0 con 0x5574b0ef2d00 2024-04-25T19:02:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4599176b80 10 client.25015 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574b0ddcaa0 con 0x5574b0ef2d00 2024-04-25T19:02:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4575600700 1 -- 172.21.15.86:0/2361883316 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45780043f0 con 0x5574b0ef2d00 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4575600700 1 -- 172.21.15.86:0/2361883316 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f457001f3a0 con 0x5574b0ef2d00 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.444+0000 7f4599176b80 10 client.25015 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.445+0000 7f4599176b80 10 client.25015 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.445+0000 7f4599176b80 20 client.25015 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.445+0000 7f4599176b80 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f0d570 0x5574b0f0e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.445+0000 7f4599176b80 4 client.25015 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.445+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 --> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574b0e53630 con 0x5574b0f0d570 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.445+0000 7f4587400700 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f0d570 0x5574b0f0e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.446+0000 7f4587400700 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f0d570 0x5574b0f0e9e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8a46380fe739778 server_cookie=c7dc94dca5f33abc in_seq=0 out_seq=0 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.446+0000 7f4575600700 10 client.25015 ms_handle_connect on v2:172.21.15.194:6836/3328875881 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.446+0000 7f4575600700 1 -- 172.21.15.86:0/2361883316 <== mds.4 v2:172.21.15.194:6836/3328875881 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574b0e53630 con 0x5574b0f0d570 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.446+0000 7f4575600700 10 client.25015 handle_command_reply: tid=0 2024-04-25T19:02:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.446+0000 7f4599176b80 1 client.25015 shutdown 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 msgr2=0x5574b0efa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efa690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f457000f010 tx=0x7f457001fd20 comp rx=0 tx=0).stop 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 shutdown_connections 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f0d570 0x5574b0f0e9e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0efdc40 0x5574b0efabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0ef23e0 0x5574b0efd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 --2- 172.21.15.86:0/2361883316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efa690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4575600700 0 client.25015 ms_handle_reset on v2:172.21.15.194:6836/3328875881 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 >> 172.21.15.86:0/2361883316 conn(0x5574b0e21550 msgr2=0x5574b0eee680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 shutdown_connections 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 1 -- 172.21.15.86:0/2361883316 wait complete. 2024-04-25T19:02:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.447+0000 7f4599176b80 20 client.25015 trim_cache size 0 max 16384 2024-04-25T19:02:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.469+0000 7f4599176b80 1 Processor -- start 2024-04-25T19:02:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.469+0000 7f4599176b80 1 -- start start 2024-04-25T19:02:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4599176b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef23e0 0x5574b0eeeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4599176b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4599176b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f17e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4599176b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f0cd20 con 0x5574b0ef23e0 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4599176b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f0d3f0 con 0x5574b0efdc40 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4599176b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f0dac0 con 0x5574b0ef2d00 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4587e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f17e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4587400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef23e0 0x5574b0eeeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41144/0 (socket says 172.21.15.86:41144) 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 -- 172.21.15.86:0/3616397301 learned_addr learned my addr 172.21.15.86:0/3616397301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f4587e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f17e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37328/0 (socket says 172.21.15.86:37328) 2024-04-25T19:02:30.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 -- 172.21.15.86:0/3616397301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 msgr2=0x5574b0f17e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f17e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 -- 172.21.15.86:0/3616397301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef23e0 msgr2=0x5574b0eeeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.470+0000 7f457fe00700 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef23e0 0x5574b0eeeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.471+0000 7f457fe00700 1 -- 172.21.15.86:0/3616397301 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b0efb660 con 0x5574b0ef2d00 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.471+0000 7f4587e00700 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f17e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.471+0000 7f457fe00700 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efab80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4570098f90 tx=0x7f457000c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667c13ae1df16baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.471+0000 7f4577e00700 1 -- 172.21.15.86:0/3616397301 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001c040 con 0x5574b0ef2d00 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.471+0000 7f4577e00700 1 -- 172.21.15.86:0/3616397301 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457009c5c0 con 0x5574b0ef2d00 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.471+0000 7f4577e00700 1 -- 172.21.15.86:0/3616397301 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45700a5000 con 0x5574b0ef2d00 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 -- 172.21.15.86:0/3616397301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 msgr2=0x5574b0efab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efab80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4570098f90 tx=0x7f457000c930 comp rx=0 tx=0).stop 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 -- 172.21.15.86:0/3616397301 shutdown_connections 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f17e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef23e0 0x5574b0eeeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 --2- 172.21.15.86:0/3616397301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef2d00 0x5574b0efab80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 -- 172.21.15.86:0/3616397301 >> 172.21.15.86:0/3616397301 conn(0x5574b0e21550 msgr2=0x5574b0efa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 -- 172.21.15.86:0/3616397301 shutdown_connections 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 -- 172.21.15.86:0/3616397301 wait complete. 2024-04-25T19:02:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.472+0000 7f4599176b80 1 -- start start 2024-04-25T19:02:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 Processor -- start 2024-04-25T19:02:30.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0f17dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef2d00 0x5574b0f18300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f0e860 con 0x5574b0ef2d00 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f0e190 con 0x5574b0efdc40 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574b0f0ef30 con 0x5574b0ef23e0 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4599176b80 10 client.0 _ll_register_callbacks cb 0x5574b0dde6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4587400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0f17dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f457fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef2d00 0x5574b0f18300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4587400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0f17dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41158/0 (socket says 172.21.15.86:41158) 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.473+0000 7f4587400700 1 -- 172.21.15.86:0/3951999740 learned_addr learned my addr 172.21.15.86:0/3951999740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587e00700 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587400700 1 -- 172.21.15.86:0/3951999740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 msgr2=0x5574b0f12220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587400700 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f12220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587400700 1 -- 172.21.15.86:0/3951999740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef2d00 msgr2=0x5574b0f18300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587400700 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef2d00 0x5574b0f18300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587400700 1 -- 172.21.15.86:0/3951999740 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b0f12a60 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4587400700 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0f17dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4578000c00 tx=0x7f4578003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91de1471ceaaa01f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4575600700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4575600700 1 -- 172.21.15.86:0/3951999740 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578035060 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457009c5c0 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.474+0000 7f4575600700 1 -- 172.21.15.86:0/3951999740 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f45780051e0 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4575600700 1 -- 172.21.15.86:0/3951999740 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45780053e0 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 10 client.25021 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574b0f752c0 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4575600700 1 -- 172.21.15.86:0/3951999740 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457009c5c0 con 0x5574b0ef23e0 2024-04-25T19:02:30.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4575600700 1 -- 172.21.15.86:0/3951999740 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4578039030 con 0x5574b0ef23e0 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 10 client.25021 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 10 client.25021 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 20 client.25021 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f79d10 0x5574b0f78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 4 client.25021 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.475+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 --> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] -- command(tid 0: {"prefix": "status"}) -- 0x5574b0e4d6d0 con 0x5574b0f79d10 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.476+0000 7f457fe00700 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f79d10 0x5574b0f78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.476+0000 7f457fe00700 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f79d10 0x5574b0f78a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9447502a5aa54f6d server_cookie=3238700e65a88681 in_seq=0 out_seq=0 2024-04-25T19:02:30.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.476+0000 7f4575600700 10 client.25021 ms_handle_connect on v2:172.21.15.194:6836/3328875881 2024-04-25T19:02:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4575600700 1 -- 172.21.15.86:0/3951999740 <== mds.4 v2:172.21.15.194:6836/3328875881 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5574b0e4d6d0 con 0x5574b0f79d10 2024-04-25T19:02:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4575600700 10 client.25021 handle_command_reply: tid=0 2024-04-25T19:02:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 client.25021 shutdown 2024-04-25T19:02:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 msgr2=0x5574b0f17dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0f17dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4578000c00 tx=0x7f4578003e70 comp rx=0 tx=0).stop 2024-04-25T19:02:30.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 shutdown_connections 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x5574b0f79d10 0x5574b0f78a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574b0efdc40 0x5574b0f12220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574b0ef23e0 0x5574b0f17dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4575600700 0 client.25021 ms_handle_reset on v2:172.21.15.194:6836/3328875881 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 --2- 172.21.15.86:0/3951999740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574b0ef2d00 0x5574b0f18300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 >> 172.21.15.86:0/3951999740 conn(0x5574b0e21550 msgr2=0x5574b0efbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 shutdown_connections 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 1 -- 172.21.15.86:0/3951999740 wait complete. 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.477+0000 7f4599176b80 20 client.25021 trim_cache size 0 max 16384 2024-04-25T19:02:30.482 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24478, 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 199.99746999300001, 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 258.58185867200001 2024-04-25T19:02:30.483 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f456c0a3c40 msgr2=0x7f456c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f456c0a3c40 0x7f456c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f45800060d0 tx=0x7f45800093e0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 msgr2=0x7f458819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 0x7f458819b790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f457000e9c0 tx=0x7f457000ecd0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 shutdown_connections 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f456c0a3c40 0x7f456c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4588109e90 0x7f458819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4588104db0 0x7f458819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 --2- 172.21.15.86:0/790017333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4588105760 0x7f458819b790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:30.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 >> 172.21.15.86:0/790017333 conn(0x7f4588100680 msgr2=0x7f45881029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 shutdown_connections 2024-04-25T19:02:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:30.478+0000 7f458d200700 1 -- 172.21.15.86:0/790017333 wait complete. 2024-04-25T19:02:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:30 smithi086 ceph-mon[27783]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:30.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:30 smithi183 ceph-mon[32344]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:30 smithi194 ceph-mon[32740]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:31.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T19:02:31.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:31 smithi086 ceph-mon[27783]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:31.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:31 smithi183 ceph-mon[32344]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:31.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:31 smithi194 ceph-mon[32740]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:34.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.146+0000 7fcdf7000700 1 -- 172.21.15.86:0/2585758311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0108a50 msgr2=0x7fcdf0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.146+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2585758311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0108a50 0x7fcdf0108e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009990 tx=0x7fcde0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 -- 172.21.15.86:0/2585758311 shutdown_connections 2024-04-25T19:02:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2585758311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdf0103450 0x7fcdf010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2585758311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0108a50 0x7fcdf0108e30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2585758311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 -- 172.21.15.86:0/2585758311 >> 172.21.15.86:0/2585758311 conn(0x7fcdf0079ba0 msgr2=0x7fcdf0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 -- 172.21.15.86:0/2585758311 shutdown_connections 2024-04-25T19:02:34.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.147+0000 7fcdf7000700 1 -- 172.21.15.86:0/2585758311 wait complete. 2024-04-25T19:02:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 Processor -- start 2024-04-25T19:02:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 -- start start 2024-04-25T19:02:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0103450 0x7fcdf0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdf0108a50 0x7fcdf00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0112c30 con 0x7fcdf0102a90 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0112ab0 con 0x7fcdf0108a50 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.148+0000 7fcdf7000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0112db0 con 0x7fcdf0103450 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33286/0 (socket says 172.21.15.86:33286) 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 -- 172.21.15.86:0/2936812952 learned_addr learned my addr 172.21.15.86:0/2936812952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcde7400700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0103450 0x7fcdf0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdf5000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdf0108a50 0x7fcdf00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 -- 172.21.15.86:0/2936812952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0103450 msgr2=0x7fcdf0075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0103450 0x7fcdf0075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 -- 172.21.15.86:0/2936812952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdf0108a50 msgr2=0x7fcdf00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdf0108a50 0x7fcdf00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 -- 172.21.15.86:0/2936812952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf01a76f0 con 0x7fcdf0102a90 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.149+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0074d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.150+0000 7fcded600700 1 -- 172.21.15.86:0/2936812952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0027070 con 0x7fcdf0102a90 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.150+0000 7fcded600700 1 -- 172.21.15.86:0/2936812952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde0022470 con 0x7fcdf0102a90 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.150+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf01a7980 con 0x7fcdf0102a90 2024-04-25T19:02:34.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.150+0000 7fcded600700 1 -- 172.21.15.86:0/2936812952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0017650 con 0x7fcdf0102a90 2024-04-25T19:02:34.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.150+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01a7eb0 con 0x7fcdf0102a90 2024-04-25T19:02:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.155+0000 7fcded600700 1 -- 172.21.15.86:0/2936812952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcde002a070 con 0x7fcdf0102a90 2024-04-25T19:02:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 Processor -- start 2024-04-25T19:02:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 -- start start 2024-04-25T19:02:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fcded600700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcdd40a3cc0 0x7fcdd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e18a00b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a010a0 0x563e189fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e189fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fcded600700 1 -- 172.21.15.86:0/2936812952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fcde00d10e0 con 0x7fcdf0102a90 2024-04-25T19:02:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e1894ce40 con 0x563e18a010a0 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e188f36e0 con 0x563e189faef0 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.156+0000 7fce018d1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e18948e00 con 0x563e18a00780 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdf5000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcdd40a3cc0 0x7fcdd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e18a00b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e18a00b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41184/0 (socket says 172.21.15.86:41184) 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 -- 172.21.15.86:0/3217949546 learned_addr learned my addr 172.21.15.86:0/3217949546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcde7400700 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a010a0 0x563e189fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdf5000700 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e189fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 -- 172.21.15.86:0/3217949546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 msgr2=0x563e189fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e189fb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 -- 172.21.15.86:0/3217949546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a010a0 msgr2=0x563e189fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a010a0 0x563e189fa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 -- 172.21.15.86:0/3217949546 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e188e9630 con 0x563e18a00780 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdf5000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcdd40a3cc0 0x7fcdd40a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009c10 tx=0x7fcde8004040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.157+0000 7fcdefe00700 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e18a00b60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a5aa0 tx=0x7fcde0022cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50bef1628cb88765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.158+0000 7fcde5600700 1 -- 172.21.15.86:0/3217949546 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00a17a0 con 0x563e18a00780 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.158+0000 7fcde5600700 1 -- 172.21.15.86:0/3217949546 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde00a5fd0 con 0x563e18a00780 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.158+0000 7fcde5600700 1 -- 172.21.15.86:0/3217949546 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde009d8b0 con 0x563e18a00780 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 -- 172.21.15.86:0/3217949546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 msgr2=0x563e18a00b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e18a00b60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a5aa0 tx=0x7fcde0022cc0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 -- 172.21.15.86:0/3217949546 shutdown_connections 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e189fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e18a00b60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 --2- 172.21.15.86:0/3217949546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a010a0 0x563e189fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 -- 172.21.15.86:0/3217949546 >> 172.21.15.86:0/3217949546 conn(0x563e18928f00 msgr2=0x563e189f58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 -- 172.21.15.86:0/3217949546 shutdown_connections 2024-04-25T19:02:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 -- 172.21.15.86:0/3217949546 wait complete. 2024-04-25T19:02:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.159+0000 7fce018d1b80 1 -- start start 2024-04-25T19:02:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 Processor -- start 2024-04-25T19:02:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e189faef0 0x563e18a06cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a03fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a04990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e18a076d0 con 0x563e18a00780 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e18906810 con 0x563e18a04510 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e18a07850 con 0x563e189faef0 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.160+0000 7fce018d1b80 10 client.0 _ll_register_callbacks cb 0x563e188e56d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcde7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a03fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcde7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a03fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33296/0 (socket says 172.21.15.86:33296) 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcde7400700 1 -- 172.21.15.86:0/2150662475 learned_addr learned my addr 172.21.15.86:0/2150662475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdf5000700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a04990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e189faef0 0x563e18a06cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 -- 172.21.15.86:0/2150662475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 msgr2=0x563e18a04990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a04990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 -- 172.21.15.86:0/2150662475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 msgr2=0x563e18a03fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a03fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 -- 172.21.15.86:0/2150662475 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1889a880 con 0x563e189faef0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdefe00700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e189faef0 0x563e18a06cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x563e18a075e0 tx=0x7fcde00a17a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c54345a673b440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.161+0000 7fcdde000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fcdde000700 1 -- 172.21.15.86:0/2150662475 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00bb060 con 0x563e189faef0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde00a5fd0 con 0x563e189faef0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fcdde000700 1 -- 172.21.15.86:0/2150662475 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fcde009d860 con 0x563e189faef0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fcdde000700 1 -- 172.21.15.86:0/2150662475 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcde0009570 con 0x563e189faef0 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fce018d1b80 10 client.25033 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e188e3aa0 con 0x563e189faef0 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.162+0000 7fcdde000700 1 -- 172.21.15.86:0/2150662475 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde00a5fd0 con 0x563e189faef0 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fcdde000700 1 -- 172.21.15.86:0/2150662475 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcde009c9b0 con 0x563e189faef0 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fce018d1b80 10 client.25033 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fce018d1b80 10 client.25033 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fce018d1b80 20 client.25033 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fce018d1b80 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a14380 0x563e18a157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fce018d1b80 4 client.25033 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.163+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 --> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e1895a630 con 0x563e18a14380 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.164+0000 7fcde7400700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a14380 0x563e18a157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.164+0000 7fcde7400700 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a14380 0x563e18a157f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9fd47f01f06d2c11 server_cookie=74b2248c6434360f in_seq=0 out_seq=0 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.164+0000 7fcdde000700 10 client.25033 ms_handle_connect on v2:172.21.15.194:6838/2085744336 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fcdde000700 1 -- 172.21.15.86:0/2150662475 <== mds.3 v2:172.21.15.194:6838/2085744336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e1895a630 con 0x563e18a14380 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fcdde000700 10 client.25033 handle_command_reply: tid=0 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 client.25033 shutdown 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e189faef0 msgr2=0x563e18a06cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e189faef0 0x563e18a06cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x563e18a075e0 tx=0x7fcde00a17a0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 shutdown_connections 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a14380 0x563e18a157f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a04990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e189faef0 0x563e18a06cd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 --2- 172.21.15.86:0/2150662475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a03fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 >> 172.21.15.86:0/2150662475 conn(0x563e18928f00 msgr2=0x563e189f4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fcdde000700 0 client.25033 ms_handle_reset on v2:172.21.15.194:6838/2085744336 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.165+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 shutdown_connections 2024-04-25T19:02:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.166+0000 7fce018d1b80 1 -- 172.21.15.86:0/2150662475 wait complete. 2024-04-25T19:02:34.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.166+0000 7fce018d1b80 20 client.25033 trim_cache size 0 max 16384 2024-04-25T19:02:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 Processor -- start 2024-04-25T19:02:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 -- start start 2024-04-25T19:02:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e189faef0 0x563e189fb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e189f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a02790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e18a13b30 con 0x563e189faef0 2024-04-25T19:02:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e18a14200 con 0x563e18a04510 2024-04-25T19:02:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fce018d1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e18a148d0 con 0x563e18a00780 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fcde7400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e189f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.188+0000 7fcdf5000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a02790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e189f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41210/0 (socket says 172.21.15.86:41210) 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 -- 172.21.15.86:0/1716031371 learned_addr learned my addr 172.21.15.86:0/1716031371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcdefe00700 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e189faef0 0x563e189fb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 -- 172.21.15.86:0/1716031371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 msgr2=0x563e18a02790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a02790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 -- 172.21.15.86:0/1716031371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e189faef0 msgr2=0x563e189fb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e189faef0 0x563e189fb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 -- 172.21.15.86:0/1716031371 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e188e3aa0 con 0x563e18a00780 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.189+0000 7fcde7400700 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e189f55d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563e18a16020 tx=0x7fcdd8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b97ced948e69ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fcde4c00700 1 -- 172.21.15.86:0/1716031371 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8004830 con 0x563e18a00780 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fcde4c00700 1 -- 172.21.15.86:0/1716031371 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd8024030 con 0x563e18a00780 2024-04-25T19:02:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fcde4c00700 1 -- 172.21.15.86:0/1716031371 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8020330 con 0x563e18a00780 2024-04-25T19:02:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 -- 172.21.15.86:0/1716031371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 msgr2=0x563e189f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e189f55d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563e18a16020 tx=0x7fcdd8009260 comp rx=0 tx=0).stop 2024-04-25T19:02:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 -- 172.21.15.86:0/1716031371 shutdown_connections 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e18a04510 0x563e18a02790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e189faef0 0x563e189fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 --2- 172.21.15.86:0/1716031371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a00780 0x563e189f55d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 -- 172.21.15.86:0/1716031371 >> 172.21.15.86:0/1716031371 conn(0x563e18928f00 msgr2=0x563e189f6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.190+0000 7fce018d1b80 1 -- 172.21.15.86:0/1716031371 shutdown_connections 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.191+0000 7fce018d1b80 1 -- 172.21.15.86:0/1716031371 wait complete. 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.191+0000 7fce018d1b80 1 -- start start 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.191+0000 7fce018d1b80 1 Processor -- start 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e18a1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a19c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a04510 0x563e18a1a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e18a15670 con 0x563e18a00780 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e18a14fa0 con 0x563e189faef0 2024-04-25T19:02:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e18a15d40 con 0x563e18a04510 2024-04-25T19:02:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fce018d1b80 10 client.0 _ll_register_callbacks cb 0x563e188e56d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcde7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a19c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcde7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a19c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33322/0 (socket says 172.21.15.86:33322) 2024-04-25T19:02:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcde7400700 1 -- 172.21.15.86:0/864931445 learned_addr learned my addr 172.21.15.86:0/864931445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdefe00700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e18a1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdf5000700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a04510 0x563e18a1a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdefe00700 1 -- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a04510 msgr2=0x563e18a1a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdefe00700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a04510 0x563e18a1a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdefe00700 1 -- 172.21.15.86:0/864931445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 msgr2=0x563e18a19c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdefe00700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a19c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.192+0000 7fcdefe00700 1 -- 172.21.15.86:0/864931445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e18a1a6a0 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdefe00700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e18a1ee50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcde0099dc0 tx=0x7fcde0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc42b78eaf3a5cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdde000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdde000700 1 -- 172.21.15.86:0/864931445 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00bb040 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd8024030 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdde000700 1 -- 172.21.15.86:0/864931445 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fcde00a5b40 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdde000700 1 -- 172.21.15.86:0/864931445 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcde00a5fd0 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fce018d1b80 10 client.25043 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e18a1a6a0 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdde000700 1 -- 172.21.15.86:0/864931445 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd8024030 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.193+0000 7fcdde000700 1 -- 172.21.15.86:0/864931445 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcde00050f0 con 0x563e189faef0 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fce018d1b80 10 client.25043 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fce018d1b80 10 client.25043 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fce018d1b80 20 client.25043 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fce018d1b80 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a808b0 0x563e18a7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fce018d1b80 4 client.25043 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 --> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] -- command(tid 0: {"prefix": "status"}) -- 0x563e189546d0 con 0x563e18a808b0 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.194+0000 7fcde7400700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a808b0 0x563e18a7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fcdde000700 10 client.25043 ms_handle_connect on v2:172.21.15.194:6838/2085744336 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fcde7400700 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a808b0 0x563e18a7f5c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=141a692d537505ed server_cookie=35f720e70f46726d in_seq=0 out_seq=0 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fcdde000700 1 -- 172.21.15.86:0/864931445 <== mds.3 v2:172.21.15.194:6838/2085744336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563e189546d0 con 0x563e18a808b0 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fcdde000700 10 client.25043 handle_command_reply: tid=0 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 client.25043 shutdown 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 msgr2=0x563e18a1ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e18a1ee50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcde0099dc0 tx=0x7fcde0005830 comp rx=0 tx=0).stop 2024-04-25T19:02:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 shutdown_connections 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e18a04510 0x563e18a1a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x563e18a808b0 0x563e18a7f5c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e189faef0 0x563e18a1ee50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fcdde000700 0 client.25043 ms_handle_reset on v2:172.21.15.194:6838/2085744336 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 --2- 172.21.15.86:0/864931445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e18a00780 0x563e18a19c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 >> 172.21.15.86:0/864931445 conn(0x563e18928f00 msgr2=0x563e18a02790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 shutdown_connections 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 1 -- 172.21.15.86:0/864931445 wait complete. 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.195+0000 7fce018d1b80 20 client.25043 trim_cache size 0 max 16384 2024-04-25T19:02:34.200 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24493, 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 204.72976874400001, 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 256.231037028 2024-04-25T19:02:34.201 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:02:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcdd40a3cc0 msgr2=0x7fcdd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcdd40a3cc0 0x7fcdd40a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009c10 tx=0x7fcde8004040 comp rx=0 tx=0).stop 2024-04-25T19:02:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 msgr2=0x7fcdf0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0074d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde00057c0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 shutdown_connections 2024-04-25T19:02:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fcdd40a3cc0 0x7fcdd40a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdf0108a50 0x7fcdf00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf0102a90 0x7fcdf0074d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 --2- 172.21.15.86:0/2936812952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcdf0103450 0x7fcdf0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 >> 172.21.15.86:0/2936812952 conn(0x7fcdf0079ba0 msgr2=0x7fcdf0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 shutdown_connections 2024-04-25T19:02:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:34.196+0000 7fcdf7000700 1 -- 172.21.15.86:0/2936812952 wait complete. 2024-04-25T19:02:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:34 smithi194 ceph-mon[32740]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:34 smithi086 ceph-mon[27783]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:34.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:34 smithi183 ceph-mon[32344]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:35.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T19:02:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:36 smithi194 ceph-mon[32740]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:36.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:36 smithi086 ceph-mon[27783]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:36 smithi183 ceph-mon[32344]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.930+0000 7f2847400700 1 -- 172.21.15.86:0/2910072217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 msgr2=0x7f284010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.930+0000 7f2847400700 1 --2- 172.21.15.86:0/2910072217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284010f8b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2838009f90 tx=0x7f283800b330 comp rx=0 tx=0).stop 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.930+0000 7f2847400700 1 -- 172.21.15.86:0/2910072217 shutdown_connections 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.930+0000 7f2847400700 1 --2- 172.21.15.86:0/2910072217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284010f8b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.930+0000 7f2847400700 1 --2- 172.21.15.86:0/2910072217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2840108a50 0x7f2840108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.931+0000 7f2847400700 1 --2- 172.21.15.86:0/2910072217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 0x7f2840102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.931+0000 7f2847400700 1 -- 172.21.15.86:0/2910072217 >> 172.21.15.86:0/2910072217 conn(0x7f2840079ba0 msgr2=0x7f2840079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.931+0000 7f2847400700 1 -- 172.21.15.86:0/2910072217 shutdown_connections 2024-04-25T19:02:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.931+0000 7f2847400700 1 -- 172.21.15.86:0/2910072217 wait complete. 2024-04-25T19:02:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 Processor -- start 2024-04-25T19:02:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 -- start start 2024-04-25T19:02:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 0x7f284019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2840108a50 0x7f284019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2840112be0 con 0x7f2840103450 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2840112a60 con 0x7f2840102a90 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.932+0000 7f2847400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2840112d60 con 0x7f2840108a50 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f283fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2844a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 0x7f284019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f283fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33340/0 (socket says 172.21.15.86:33340) 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f283fe00700 1 -- 172.21.15.86:0/3395271357 learned_addr learned my addr 172.21.15.86:0/3395271357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2845400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2840108a50 0x7f284019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2844a00700 1 -- 172.21.15.86:0/3395271357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2840108a50 msgr2=0x7f284019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2844a00700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2840108a50 0x7f284019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2844a00700 1 -- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 msgr2=0x7f284019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2844a00700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f2844a00700 1 -- 172.21.15.86:0/3395271357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28401a02c0 con 0x7f2840102a90 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.933+0000 7f283fe00700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.934+0000 7f2844a00700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 0x7f284019b2b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f283400e9c0 tx=0x7f283400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.934+0000 7f283d600700 1 -- 172.21.15.86:0/3395271357 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400c9c0 con 0x7f2840102a90 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.934+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28401a05b0 con 0x7f2840102a90 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.934+0000 7f283d600700 1 -- 172.21.15.86:0/3395271357 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f283400cb60 con 0x7f2840102a90 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.934+0000 7f283d600700 1 -- 172.21.15.86:0/3395271357 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834018420 con 0x7f2840102a90 2024-04-25T19:02:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.934+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a7e30 con 0x7f2840102a90 2024-04-25T19:02:37.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.937+0000 7f283d600700 1 -- 172.21.15.86:0/3395271357 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2834010430 con 0x7f2840102a90 2024-04-25T19:02:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.939+0000 7f2851df7b80 1 Processor -- start 2024-04-25T19:02:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f283d600700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f28280a3c40 0x7f28280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 -- start start 2024-04-25T19:02:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f283d600700 1 -- 172.21.15.86:0/3395271357 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f28340d0520 con 0x7f2840102a90 2024-04-25T19:02:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2845400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f28280a3c40 0x7f28280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6150b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be61510a0 0x558be614a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be614b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558be609ce40 con 0x558be61510a0 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558be60436e0 con 0x558be614aef0 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.940+0000 7f2851df7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558be6098e00 con 0x558be6150780 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f283fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be61510a0 0x558be614a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41224/0 (socket says 172.21.15.86:41224) 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 -- 172.21.15.86:0/2918939827 learned_addr learned my addr 172.21.15.86:0/2918939827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2845400700 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be614b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 -- 172.21.15.86:0/2918939827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 msgr2=0x558be614b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be614b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 -- 172.21.15.86:0/2918939827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be61510a0 msgr2=0x558be614a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2845400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f28280a3c40 0x7f28280a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2838009da0 tx=0x7f28380096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be61510a0 0x558be614a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 -- 172.21.15.86:0/2918939827 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be6039630 con 0x558be6150780 2024-04-25T19:02:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.941+0000 7f2844a00700 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6150b60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2834099110 tx=0x7f2834098b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ae2393b68e7fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2832000700 1 -- 172.21.15.86:0/2918939827 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283409c740 con 0x558be6150780 2024-04-25T19:02:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2832000700 1 -- 172.21.15.86:0/2918939827 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28340a0af0 con 0x558be6150780 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2832000700 1 -- 172.21.15.86:0/2918939827 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340d00b0 con 0x558be6150780 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 -- 172.21.15.86:0/2918939827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 msgr2=0x558be6150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6150b60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2834099110 tx=0x7f2834098b10 comp rx=0 tx=0).stop 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 -- 172.21.15.86:0/2918939827 shutdown_connections 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be614b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6150b60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2918939827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be61510a0 0x558be614a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.942+0000 7f2851df7b80 1 -- 172.21.15.86:0/2918939827 >> 172.21.15.86:0/2918939827 conn(0x558be6078f00 msgr2=0x558be61458a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.943+0000 7f2851df7b80 1 -- 172.21.15.86:0/2918939827 shutdown_connections 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.943+0000 7f2851df7b80 1 -- 172.21.15.86:0/2918939827 wait complete. 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.943+0000 7f2851df7b80 1 -- start start 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.943+0000 7f2851df7b80 1 Processor -- start 2024-04-25T19:02:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be6156d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6154020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be61549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558be6157920 con 0x558be6154560 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558be6056810 con 0x558be614aef0 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558be6157aa0 con 0x558be6150780 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2851df7b80 10 client.0 _ll_register_callbacks cb 0x558be60356d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2845400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be61549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2845400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be61549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33352/0 (socket says 172.21.15.86:33352) 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f283fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2845400700 1 -- 172.21.15.86:0/2440810602 learned_addr learned my addr 172.21.15.86:0/2440810602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.944+0000 7f2844a00700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be6156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2845400700 1 -- 172.21.15.86:0/2440810602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 msgr2=0x558be6154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2845400700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2845400700 1 -- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 msgr2=0x558be6156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2845400700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be6156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2845400700 1 -- 172.21.15.86:0/2440810602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be5fea880 con 0x558be6154560 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f283fe00700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2844a00700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be6156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2845400700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be61549e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f283800bd70 tx=0x7f283800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f20a459d3c479e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2826a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2826a00700 1 -- 172.21.15.86:0/2440810602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838007830 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2826a00700 1 -- 172.21.15.86:0/2440810602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f28380079d0 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.945+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28340a0af0 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2826a00700 1 -- 172.21.15.86:0/2440810602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2838007bd0 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2851df7b80 10 client.14931 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558be6033aa0 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2826a00700 1 -- 172.21.15.86:0/2440810602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28340a0af0 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2826a00700 1 -- 172.21.15.86:0/2440810602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2838011070 con 0x558be6154560 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2851df7b80 10 client.14931 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2851df7b80 10 client.14931 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2851df7b80 20 client.14931 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.946+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be6164b70 0x558be6165fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.947+0000 7f2851df7b80 4 client.14931 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.947+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 --> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be60aa630 con 0x558be6164b70 2024-04-25T19:02:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.947+0000 7f2844a00700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be6164b70 0x558be6165fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.947+0000 7f2844a00700 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be6164b70 0x558be6165fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36047057e2c63c25 server_cookie=37d8641b5fc7bf08 in_seq=0 out_seq=0 2024-04-25T19:02:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.947+0000 7f2826a00700 10 client.14931 ms_handle_connect on v2:172.21.15.183:6836/3273542947 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.948+0000 7f2826a00700 1 -- 172.21.15.86:0/2440810602 <== mds.2 v2:172.21.15.183:6836/3273542947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558be60aa630 con 0x558be6164b70 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.948+0000 7f2826a00700 10 client.14931 handle_command_reply: tid=0 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.948+0000 7f2851df7b80 1 client.14931 shutdown 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 msgr2=0x558be61549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be61549e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f283800bd70 tx=0x7f283800b330 comp rx=0 tx=0).stop 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 shutdown_connections 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be6164b70 0x558be6165fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be61549e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be6156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2440810602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be6154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 >> 172.21.15.86:0/2440810602 conn(0x558be6078f00 msgr2=0x558be6144e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2826a00700 0 client.14931 ms_handle_reset on v2:172.21.15.183:6836/3273542947 2024-04-25T19:02:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 shutdown_connections 2024-04-25T19:02:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 1 -- 172.21.15.86:0/2440810602 wait complete. 2024-04-25T19:02:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.949+0000 7f2851df7b80 20 client.14931 trim_cache size 0 max 16384 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.971+0000 7f2851df7b80 1 Processor -- start 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.971+0000 7f2851df7b80 1 -- start start 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be614aef0 0x558be614a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be6150780 0x558be61455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2851df7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558be6164320 con 0x558be6154560 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2851df7b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558be61649f0 con 0x558be6150780 2024-04-25T19:02:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2851df7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558be61650c0 con 0x558be614aef0 2024-04-25T19:02:37.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33368/0 (socket says 172.21.15.86:33368) 2024-04-25T19:02:37.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f283fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be6150780 0x558be61455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 -- 172.21.15.86:0/2259208125 learned_addr learned my addr 172.21.15.86:0/2259208125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:37.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2844a00700 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be614aef0 0x558be614a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 -- 172.21.15.86:0/2259208125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be614aef0 msgr2=0x558be614a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be614aef0 0x558be614a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 -- 172.21.15.86:0/2259208125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be6150780 msgr2=0x558be61455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be6150780 0x558be61455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2845400700 1 -- 172.21.15.86:0/2259208125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be61612e0 con 0x558be6154560 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f283fe00700 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be6150780 0x558be61455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.972+0000 7f2844a00700 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be614aef0 0x558be614a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.973+0000 7f2845400700 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616f440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d70 tx=0x7f28380099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2f38e703c86e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.973+0000 7f2831600700 1 -- 172.21.15.86:0/2259208125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283803f070 con 0x558be6154560 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.973+0000 7f2831600700 1 -- 172.21.15.86:0/2259208125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f283802c070 con 0x558be6154560 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.973+0000 7f2831600700 1 -- 172.21.15.86:0/2259208125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28380050b0 con 0x558be6154560 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 -- 172.21.15.86:0/2259208125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 msgr2=0x558be616f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616f440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d70 tx=0x7f28380099c0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 -- 172.21.15.86:0/2259208125 shutdown_connections 2024-04-25T19:02:37.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616f440 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be614aef0 0x558be614a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 --2- 172.21.15.86:0/2259208125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be6150780 0x558be61455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 -- 172.21.15.86:0/2259208125 >> 172.21.15.86:0/2259208125 conn(0x558be6078f00 msgr2=0x558be614b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 -- 172.21.15.86:0/2259208125 shutdown_connections 2024-04-25T19:02:37.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 -- 172.21.15.86:0/2259208125 wait complete. 2024-04-25T19:02:37.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.974+0000 7f2851df7b80 1 -- start start 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 Processor -- start 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be616f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be616a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558be6165e60 con 0x558be6154560 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558be6165790 con 0x558be614aef0 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558be6166530 con 0x558be6150780 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.975+0000 7f2851df7b80 10 client.0 _ll_register_callbacks cb 0x558be60356d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f283fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be616a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33376/0 (socket says 172.21.15.86:33376) 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 -- 172.21.15.86:0/3999005556 learned_addr learned my addr 172.21.15.86:0/3999005556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2844a00700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be616f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 -- 172.21.15.86:0/3999005556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 msgr2=0x558be616a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be616a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 -- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 msgr2=0x558be616f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be616f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 -- 172.21.15.86:0/3999005556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be616ae90 con 0x558be6154560 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2844a00700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be616f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2845400700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616a950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2838031630 tx=0x7f28380318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30ff383ec3cbb9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2826a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2826a00700 1 -- 172.21.15.86:0/3999005556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283803f070 con 0x558be6154560 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2826a00700 1 -- 172.21.15.86:0/3999005556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f2838031ce0 con 0x558be6154560 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.976+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283802c070 con 0x558be6154560 2024-04-25T19:02:37.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2826a00700 1 -- 172.21.15.86:0/3999005556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28380050c0 con 0x558be6154560 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 10 client.14937 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558be61cc9c0 con 0x558be6154560 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2826a00700 1 -- 172.21.15.86:0/3999005556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f283802c070 con 0x558be6154560 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2826a00700 1 -- 172.21.15.86:0/3999005556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f283804c050 con 0x558be6154560 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 10 client.14937 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 10 client.14937 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 20 client.14937 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be61d10c0 0x558be61cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 4 client.14937 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.977+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 --> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] -- command(tid 0: {"prefix": "status"}) -- 0x558be60a46d0 con 0x558be61d10c0 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.978+0000 7f2844a00700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be61d10c0 0x558be61cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.978+0000 7f2844a00700 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be61d10c0 0x558be61cfdd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a50d6c9f9479bb19 server_cookie=6441f003b8e2da98 in_seq=0 out_seq=0 2024-04-25T19:02:37.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.978+0000 7f2826a00700 10 client.14937 ms_handle_connect on v2:172.21.15.183:6836/3273542947 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2826a00700 1 -- 172.21.15.86:0/3999005556 <== mds.2 v2:172.21.15.183:6836/3273542947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558be60a46d0 con 0x558be61d10c0 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2826a00700 10 client.14937 handle_command_reply: tid=0 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 client.14937 shutdown 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 msgr2=0x558be616a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616a950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2838031630 tx=0x7f28380318e0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 shutdown_connections 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x558be61d10c0 0x558be61cfdd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558be6154560 0x558be616a950 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558be614aef0 0x558be616f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 --2- 172.21.15.86:0/3999005556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558be6150780 0x558be616a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2826a00700 0 client.14937 ms_handle_reset on v2:172.21.15.183:6836/3273542947 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 >> 172.21.15.86:0/3999005556 conn(0x558be6078f00 msgr2=0x558be61528b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 shutdown_connections 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 1 -- 172.21.15.86:0/3999005556 wait complete. 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.979+0000 7f2851df7b80 20 client.14937 trim_cache size 0 max 16384 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24494, 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 209.531421929, 2024-04-25T19:02:37.985 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 263.13172210599998 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f28280a3c40 msgr2=0x7f28280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f28280a3c40 0x7f28280a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2838009da0 tx=0x7f28380096c0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 msgr2=0x7f284019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 0x7f284019b2b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f283400e9c0 tx=0x7f283400ecd0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 shutdown_connections 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f28280a3c40 0x7f28280a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2840108a50 0x7f284019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2840102a90 0x7f284019b2b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 --2- 172.21.15.86:0/3395271357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2840103450 0x7f284019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 >> 172.21.15.86:0/3395271357 conn(0x7f2840079ba0 msgr2=0x7f2840101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 shutdown_connections 2024-04-25T19:02:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:37.980+0000 7f2847400700 1 -- 172.21.15.86:0/3395271357 wait complete. 2024-04-25T19:02:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:38 smithi194 ceph-mon[32740]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:38.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:38 smithi086 ceph-mon[27783]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:38.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.g status 2024-04-25T19:02:38.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:38 smithi183 ceph-mon[32344]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T19:02:39.339 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:40.813 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:40 smithi086 ceph-mon[27783]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:40 smithi183 ceph-mon[32344]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:40 smithi194 ceph-mon[32740]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:41.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.637+0000 7f26b5600700 1 -- 172.21.15.86:0/2810675674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0104e90 msgr2=0x7f26b0105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.637+0000 7f26b5600700 1 --2- 172.21.15.86:0/2810675674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0104e90 0x7f26b0105270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26a00099c0 tx=0x7f26a0009cd0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 -- 172.21.15.86:0/2810675674 shutdown_connections 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 --2- 172.21.15.86:0/2810675674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b0109f70 0x7f26b0111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 --2- 172.21.15.86:0/2810675674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0104e90 0x7f26b0105270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 --2- 172.21.15.86:0/2810675674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0105840 0x7f26b0109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 -- 172.21.15.86:0/2810675674 >> 172.21.15.86:0/2810675674 conn(0x7f26b0100720 msgr2=0x7f26b0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 -- 172.21.15.86:0/2810675674 shutdown_connections 2024-04-25T19:02:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.638+0000 7f26b5600700 1 -- 172.21.15.86:0/2810675674 wait complete. 2024-04-25T19:02:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 Processor -- start 2024-04-25T19:02:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 -- start start 2024-04-25T19:02:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0104e90 0x7f26b019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b0105840 0x7f26b019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0109f70 0x7f26b019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0114e00 con 0x7f26b0104e90 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0114c80 con 0x7f26b0105840 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.639+0000 7f26b5600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0114f80 con 0x7f26b0109f70 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0104e90 0x7f26b019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0104e90 0x7f26b019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42160/0 (socket says 172.21.15.86:42160) 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26aea00700 1 -- 172.21.15.86:0/3661202447 learned_addr learned my addr 172.21.15.86:0/3661202447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26af400700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0109f70 0x7f26b019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26a7e00700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b0105840 0x7f26b019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26af400700 1 -- 172.21.15.86:0/3661202447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b0105840 msgr2=0x7f26b019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26af400700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b0105840 0x7f26b019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26af400700 1 -- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0104e90 msgr2=0x7f26b019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26af400700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0104e90 0x7f26b019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.640+0000 7f26af400700 1 -- 172.21.15.86:0/3661202447 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b01a0300 con 0x7f26b0109f70 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.641+0000 7f26af400700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0109f70 0x7f26b019fbc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26a800efc0 tx=0x7f26a800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.641+0000 7f26a7400700 1 -- 172.21.15.86:0/3661202447 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80092e0 con 0x7f26b0109f70 2024-04-25T19:02:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.641+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b01a0590 con 0x7f26b0109f70 2024-04-25T19:02:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.641+0000 7f26a7400700 1 -- 172.21.15.86:0/3661202447 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a8004bb0 con 0x7f26b0109f70 2024-04-25T19:02:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.641+0000 7f26a7400700 1 -- 172.21.15.86:0/3661202447 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8004020 con 0x7f26b0109f70 2024-04-25T19:02:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.641+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a7e70 con 0x7f26b0109f70 2024-04-25T19:02:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.644+0000 7f26a7400700 1 -- 172.21.15.86:0/3661202447 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26a801b030 con 0x7f26b0109f70 2024-04-25T19:02:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.645+0000 7f26a7400700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26940a3c40 0x7f26940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.646+0000 7f26a7400700 1 -- 172.21.15.86:0/3661202447 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f26a80cfe70 con 0x7f26b0109f70 2024-04-25T19:02:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.646+0000 7f26bffa1b80 1 Processor -- start 2024-04-25T19:02:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.646+0000 7f26aea00700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26940a3c40 0x7f26940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.646+0000 7f26bffa1b80 1 -- start start 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 0x5599d51fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fe0a0 0x5599d51f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26aea00700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26940a3c40 0x7f26940a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009990 tx=0x7f26a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51f7ef0 0x5599d51f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d5149e40 con 0x5599d51fd780 2024-04-25T19:02:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599d50f06e0 con 0x5599d51fe0a0 2024-04-25T19:02:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599d5145e00 con 0x5599d51f7ef0 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 0x5599d51fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26af400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51f7ef0 0x5599d51f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26af400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51f7ef0 0x5599d51f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59320/0 (socket says 172.21.15.86:59320) 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 0x5599d51fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42172/0 (socket says 172.21.15.86:42172) 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26af400700 1 -- 172.21.15.86:0/3475126410 learned_addr learned my addr 172.21.15.86:0/3475126410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.647+0000 7f26a7e00700 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fe0a0 0x5599d51f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f26aea00700 1 -- 172.21.15.86:0/3475126410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51f7ef0 msgr2=0x5599d51f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f26aea00700 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51f7ef0 0x5599d51f8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f26aea00700 1 -- 172.21.15.86:0/3475126410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fe0a0 msgr2=0x5599d51f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f26aea00700 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fe0a0 0x5599d51f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f26aea00700 1 -- 172.21.15.86:0/3475126410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d50e6630 con 0x5599d51fd780 2024-04-25T19:02:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f26aea00700 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 0x5599d51fdb60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26a00059d0 tx=0x7f26a001a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b94b2cbd07bd9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f269fe00700 1 -- 172.21.15.86:0/3475126410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0003c20 con 0x5599d51fd780 2024-04-25T19:02:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f269fe00700 1 -- 172.21.15.86:0/3475126410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a0003dc0 con 0x5599d51fd780 2024-04-25T19:02:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.648+0000 7f269fe00700 1 -- 172.21.15.86:0/3475126410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0023ae0 con 0x5599d51fd780 2024-04-25T19:02:41.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3475126410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 msgr2=0x5599d51fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 0x5599d51fdb60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26a00059d0 tx=0x7f26a001a070 comp rx=0 tx=0).stop 2024-04-25T19:02:41.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3475126410 shutdown_connections 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51f7ef0 0x5599d51f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51fd780 0x5599d51fdb60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3475126410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fe0a0 0x5599d51f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3475126410 >> 172.21.15.86:0/3475126410 conn(0x5599d5125f00 msgr2=0x5599d51f28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3475126410 shutdown_connections 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.649+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3475126410 wait complete. 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.650+0000 7f26bffa1b80 1 -- start start 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.650+0000 7f26bffa1b80 1 Processor -- start 2024-04-25T19:02:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5203d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fd780 0x5599d5201040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d5201580 0x5599d5201a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d5204860 con 0x5599d51f7ef0 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599d5103810 con 0x5599d51fd780 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599d52049e0 con 0x5599d5201580 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26bffa1b80 10 client.0 _ll_register_callbacks cb 0x5599d50e26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5203d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5203d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42182/0 (socket says 172.21.15.86:42182) 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 -- 172.21.15.86:0/3070264194 learned_addr learned my addr 172.21.15.86:0/3070264194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26a7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fd780 0x5599d5201040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26af400700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d5201580 0x5599d5201a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 -- 172.21.15.86:0/3070264194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d5201580 msgr2=0x5599d5201a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d5201580 0x5599d5201a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 -- 172.21.15.86:0/3070264194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fd780 msgr2=0x5599d5201040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.651+0000 7f26aea00700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fd780 0x5599d5201040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f26aea00700 1 -- 172.21.15.86:0/3070264194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d5097880 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f26a7e00700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fd780 0x5599d5201040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f26af400700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d5201580 0x5599d5201a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f26aea00700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5203d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009350 tx=0x7f26a000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd70352e2da8fa73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f269cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f269cc00700 1 -- 172.21.15.86:0/3070264194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a003c060 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f269cc00700 1 -- 172.21.15.86:0/3070264194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f26a0023670 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a0003dc0 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.652+0000 7f269cc00700 1 -- 172.21.15.86:0/3070264194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26a0023870 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 10 client.14949 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599d50e0aa0 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f269cc00700 1 -- 172.21.15.86:0/3070264194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a0003dc0 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f269cc00700 1 -- 172.21.15.86:0/3070264194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f26a0040030 con 0x5599d51f7ef0 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 10 client.14949 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 10 client.14949 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T19:02:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 20 client.14949 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d5210720 0x5599d5212890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 4 client.14949 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.653+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 --> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599d5157630 con 0x5599d5210720 2024-04-25T19:02:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.654+0000 7f26a7e00700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d5210720 0x5599d5212890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.654+0000 7f26a7e00700 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d5210720 0x5599d5212890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6f1184ca123eaa3 server_cookie=3d97757e4ee8748b in_seq=0 out_seq=0 2024-04-25T19:02:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.654+0000 7f269cc00700 10 client.14949 ms_handle_connect on v2:172.21.15.86:6838/3676043902 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.655+0000 7f269cc00700 1 -- 172.21.15.86:0/3070264194 <== mds.1 v2:172.21.15.86:6838/3676043902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599d5157630 con 0x5599d5210720 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.655+0000 7f269cc00700 10 client.14949 handle_command_reply: tid=0 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.655+0000 7f26bffa1b80 1 client.14949 shutdown 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 msgr2=0x5599d5203d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5203d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009350 tx=0x7f26a000be10 comp rx=0 tx=0).stop 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 shutdown_connections 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d5210720 0x5599d5212890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d5201580 0x5599d5201a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5203d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3070264194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51fd780 0x5599d5201040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 >> 172.21.15.86:0/3070264194 conn(0x5599d5125f00 msgr2=0x5599d51f1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f269cc00700 0 client.14949 ms_handle_reset on v2:172.21.15.86:6838/3676043902 2024-04-25T19:02:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 shutdown_connections 2024-04-25T19:02:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3070264194 wait complete. 2024-04-25T19:02:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.656+0000 7f26bffa1b80 20 client.14949 trim_cache size 0 max 16384 2024-04-25T19:02:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.678+0000 7f26bffa1b80 1 Processor -- start 2024-04-25T19:02:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.678+0000 7f26bffa1b80 1 -- start start 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5212890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d52019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 0x5599d520de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d5210bd0 con 0x5599d51f7ef0 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599d52112a0 con 0x5599d5201580 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599d5211970 con 0x5599d51fd780 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5212890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26af400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 0x5599d520de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26af400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 0x5599d520de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:44564/0 (socket says 172.21.15.86:44564) 2024-04-25T19:02:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26af400700 1 -- 172.21.15.86:0/3143873417 learned_addr learned my addr 172.21.15.86:0/3143873417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5212890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42190/0 (socket says 172.21.15.86:42190) 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.679+0000 7f26a7e00700 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d52019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f26af400700 1 -- 172.21.15.86:0/3143873417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 msgr2=0x5599d52019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f26af400700 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d52019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f26af400700 1 -- 172.21.15.86:0/3143873417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 msgr2=0x5599d5212890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f26af400700 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5212890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f26af400700 1 -- 172.21.15.86:0/3143873417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d5212530 con 0x5599d5201580 2024-04-25T19:02:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f26af400700 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 0x5599d520de60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f26a80090a0 tx=0x7f26a8004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60fd91a1cb0cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f269f400700 1 -- 172.21.15.86:0/3143873417 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80b4070 con 0x5599d5201580 2024-04-25T19:02:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f269f400700 1 -- 172.21.15.86:0/3143873417 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a809c120 con 0x5599d5201580 2024-04-25T19:02:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.680+0000 7f269f400700 1 -- 172.21.15.86:0/3143873417 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80a4870 con 0x5599d5201580 2024-04-25T19:02:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3143873417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 msgr2=0x5599d520de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 0x5599d520de60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f26a80090a0 tx=0x7f26a8004d60 comp rx=0 tx=0).stop 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3143873417 shutdown_connections 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d5201580 0x5599d520de60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d51f7ef0 0x5599d5212890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/3143873417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d52019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3143873417 >> 172.21.15.86:0/3143873417 conn(0x5599d5125f00 msgr2=0x5599d51eeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3143873417 shutdown_connections 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.681+0000 7f26bffa1b80 1 -- 172.21.15.86:0/3143873417 wait complete. 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.682+0000 7f26bffa1b80 1 -- start start 2024-04-25T19:02:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.682+0000 7f26bffa1b80 1 Processor -- start 2024-04-25T19:02:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.682+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51f7ef0 0x5599d51f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.682+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d51f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26bffa1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d5201580 0x5599d51f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d5212710 con 0x5599d5201580 2024-04-25T19:02:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599d5212040 con 0x5599d51f7ef0 2024-04-25T19:02:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26bffa1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599d5212de0 con 0x5599d51fd780 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26bffa1b80 10 client.0 _ll_register_callbacks cb 0x5599d50e26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d51f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d5201580 0x5599d51f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d51f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59356/0 (socket says 172.21.15.86:59356) 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 -- 172.21.15.86:0/221190597 learned_addr learned my addr 172.21.15.86:0/221190597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26aea00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51f7ef0 0x5599d51f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 -- 172.21.15.86:0/221190597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51f7ef0 msgr2=0x5599d51f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51f7ef0 0x5599d51f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 -- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d5201580 msgr2=0x5599d51f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d5201580 0x5599d51f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26a7e00700 1 -- 172.21.15.86:0/221190597 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d5219fa0 con 0x5599d51fd780 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.683+0000 7f26aea00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51f7ef0 0x5599d51f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f26a7e00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d51f20e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26b0101bb0 tx=0x7f2698009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33472ac63d76a8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f269cc00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:02:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f269cc00700 1 -- 172.21.15.86:0/221190597 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698004250 con 0x5599d51fd780 2024-04-25T19:02:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f269cc00700 1 -- 172.21.15.86:0/221190597 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f26980043f0 con 0x5599d51fd780 2024-04-25T19:02:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a809c120 con 0x5599d51fd780 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f269cc00700 1 -- 172.21.15.86:0/221190597 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2698003680 con 0x5599d51fd780 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f26bffa1b80 10 client.25072 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.684+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599d52788e0 con 0x5599d51fd780 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f269cc00700 1 -- 172.21.15.86:0/221190597 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a809c120 con 0x5599d51fd780 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f269cc00700 1 -- 172.21.15.86:0/221190597 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f26980203b0 con 0x5599d51fd780 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26bffa1b80 10 client.25072 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26bffa1b80 10 client.25072 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26bffa1b80 20 client.25072 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d527d590 0x5599d527c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26bffa1b80 4 client.25072 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:02:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 --> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] -- command(tid 0: {"prefix": "status"}) -- 0x5599d51516d0 con 0x5599d527d590 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.685+0000 7f26aea00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d527d590 0x5599d527c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f26aea00700 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d527d590 0x5599d527c2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d906c1f4b137deed server_cookie=3502bce7b9760cbd in_seq=0 out_seq=0 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f269cc00700 10 client.25072 ms_handle_connect on v2:172.21.15.86:6838/3676043902 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f269cc00700 1 -- 172.21.15.86:0/221190597 <== mds.1 v2:172.21.15.86:6838/3676043902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5599d51516d0 con 0x5599d527d590 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f269cc00700 10 client.25072 handle_command_reply: tid=0 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f26bffa1b80 1 client.25072 shutdown 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 msgr2=0x5599d51f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.686+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d51f20e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26b0101bb0 tx=0x7f2698009820 comp rx=0 tx=0).stop 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 shutdown_connections 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x5599d527d590 0x5599d527c2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d5201580 0x5599d51f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d51f7ef0 0x5599d51f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 --2- 172.21.15.86:0/221190597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d51fd780 0x5599d51f20e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f269cc00700 0 client.25072 ms_handle_reset on v2:172.21.15.86:6838/3676043902 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 >> 172.21.15.86:0/221190597 conn(0x5599d5125f00 msgr2=0x5599d51fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 shutdown_connections 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 1 -- 172.21.15.86:0/221190597 wait complete. 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.687+0000 7f26bffa1b80 20 client.25072 trim_cache size 0 max 16384 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24509, 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:02:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 214.24755317399999, 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 258.49854551300001 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:02:41.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26940a3c40 msgr2=0x7f26940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26940a3c40 0x7f26940a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009990 tx=0x7f26a0005960 comp rx=0 tx=0).stop 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0109f70 msgr2=0x7f26b019fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0109f70 0x7f26b019fbc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26a800efc0 tx=0x7f26a800c3a0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 shutdown_connections 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f26940a3c40 0x7f26940a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26b0109f70 0x7f26b019fbc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0104e90 0x7f26b019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 --2- 172.21.15.86:0/3661202447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b0105840 0x7f26b019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 >> 172.21.15.86:0/3661202447 conn(0x7f26b0100720 msgr2=0x7f26b0109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 shutdown_connections 2024-04-25T19:02:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:41.688+0000 7f26b5600700 1 -- 172.21.15.86:0/3661202447 wait complete. 2024-04-25T19:02:42.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T19:02:42.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:42 smithi086 ceph-mon[27783]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:42.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:42 smithi183 ceph-mon[32344]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:42 smithi194 ceph-mon[32740]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T19:02:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:44.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:44 smithi086 ceph-mon[27783]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:44.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:44 smithi183 ceph-mon[32344]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:44 smithi194 ceph-mon[32740]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T19:02:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 -- 172.21.15.86:0/181562237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006b980 msgr2=0x7f4c7010f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 --2- 172.21.15.86:0/181562237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006b980 0x7f4c7010f8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800b0a0 tx=0x7f4c6800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 -- 172.21.15.86:0/181562237 shutdown_connections 2024-04-25T19:02:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 --2- 172.21.15.86:0/181562237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006b980 0x7f4c7010f8f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 --2- 172.21.15.86:0/181562237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c70108a20 0x7f4c70108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 --2- 172.21.15.86:0/181562237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c7006afe0 0x7f4c7006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 -- 172.21.15.86:0/181562237 >> 172.21.15.86:0/181562237 conn(0x7f4c700788b0 msgr2=0x7f4c70078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 -- 172.21.15.86:0/181562237 shutdown_connections 2024-04-25T19:02:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.345+0000 7f4c75000700 1 -- 172.21.15.86:0/181562237 wait complete. 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.346+0000 7f4c75000700 1 Processor -- start 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.346+0000 7f4c75000700 1 -- start start 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c75000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006afe0 0x7f4c70101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c75000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 0x7f4c70101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c75000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c70108a20 0x7f4c70102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c75000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70112bc0 con 0x7f4c7006afe0 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c75000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70112a40 con 0x7f4c70108a20 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c75000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70112d40 con 0x7f4c7006b980 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006afe0 0x7f4c70101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 0x7f4c70101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006afe0 0x7f4c70101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42214/0 (socket says 172.21.15.86:42214) 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 0x7f4c70101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59382/0 (socket says 172.21.15.86:59382) 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c6de00700 1 -- 172.21.15.86:0/2213589137 learned_addr learned my addr 172.21.15.86:0/2213589137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.347+0000 7f4c6e800700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c70108a20 0x7f4c70102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c6d400700 1 -- 172.21.15.86:0/2213589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c70108a20 msgr2=0x7f4c70102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c6d400700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c70108a20 0x7f4c70102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c6d400700 1 -- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006afe0 msgr2=0x7f4c70101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c6d400700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006afe0 0x7f4c70101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c6d400700 1 -- 172.21.15.86:0/2213589137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c701a75f0 con 0x7f4c7006b980 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c6d400700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 0x7f4c70101f40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d750 tx=0x7f4c5c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c66a00700 1 -- 172.21.15.86:0/2213589137 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c014030 con 0x7f4c7006b980 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c66a00700 1 -- 172.21.15.86:0/2213589137 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c0044d0 con 0x7f4c7006b980 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c66a00700 1 -- 172.21.15.86:0/2213589137 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c005420 con 0x7f4c7006b980 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c701a78e0 con 0x7f4c7006b980 2024-04-25T19:02:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.348+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701a7dd0 con 0x7f4c7006b980 2024-04-25T19:02:45.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.352+0000 7f4c66a00700 1 -- 172.21.15.86:0/2213589137 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c5c004a50 con 0x7f4c7006b980 2024-04-25T19:02:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 Processor -- start 2024-04-25T19:02:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 -- start start 2024-04-25T19:02:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c66a00700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c540a3cc0 0x7f4c540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 0x55e12e80fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e8100a0 0x55e12e8099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e809ef0 0x55e12e80a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e12e75be40 con 0x55e12e809ef0 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e12e7026e0 con 0x55e12e8100a0 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e12e757e00 con 0x55e12e80f780 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.353+0000 7f4c66a00700 1 -- 172.21.15.86:0/2213589137 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4c5c0cfff0 con 0x7f4c7006b980 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 0x55e12e80fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6e800700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c540a3cc0 0x7f4c540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 0x55e12e80fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59394/0 (socket says 172.21.15.86:59394) 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 -- 172.21.15.86:0/4107286968 learned_addr learned my addr 172.21.15.86:0/4107286968 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6e800700 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e809ef0 0x55e12e80a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6d400700 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e8100a0 0x55e12e8099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 -- 172.21.15.86:0/4107286968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e8100a0 msgr2=0x55e12e8099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e8100a0 0x55e12e8099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 -- 172.21.15.86:0/4107286968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e809ef0 msgr2=0x55e12e80a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e809ef0 0x55e12e80a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6de00700 1 -- 172.21.15.86:0/4107286968 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12e6f8630 con 0x55e12e80f780 2024-04-25T19:02:45.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.354+0000 7f4c6d400700 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e8100a0 0x55e12e8099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.355+0000 7f4c6de00700 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 0x55e12e80fb60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009e30 tx=0x7f4c60011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354ce69c423d78b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.355+0000 7f4c6e800700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c540a3cc0 0x7f4c540a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009e60 tx=0x7f4c68009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.355+0000 7f4c5b400700 1 -- 172.21.15.86:0/4107286968 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6001e070 con 0x55e12e80f780 2024-04-25T19:02:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.355+0000 7f4c5b400700 1 -- 172.21.15.86:0/4107286968 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c60019ae0 con 0x55e12e80f780 2024-04-25T19:02:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.355+0000 7f4c5b400700 1 -- 172.21.15.86:0/4107286968 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000fa90 con 0x55e12e80f780 2024-04-25T19:02:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 -- 172.21.15.86:0/4107286968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 msgr2=0x55e12e80fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 0x55e12e80fb60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009e30 tx=0x7f4c60011720 comp rx=0 tx=0).stop 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 -- 172.21.15.86:0/4107286968 shutdown_connections 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e809ef0 0x55e12e80a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e80f780 0x55e12e80fb60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/4107286968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e8100a0 0x55e12e8099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 -- 172.21.15.86:0/4107286968 >> 172.21.15.86:0/4107286968 conn(0x55e12e737f00 msgr2=0x55e12e8048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 -- 172.21.15.86:0/4107286968 shutdown_connections 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.356+0000 7f4c7f976b80 1 -- 172.21.15.86:0/4107286968 wait complete. 2024-04-25T19:02:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 -- start start 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 Processor -- start 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e815c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e812f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e8138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e12e816920 con 0x55e12e80f780 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e12e715810 con 0x55e12e813450 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.357+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e12e816aa0 con 0x55e12e809ef0 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c7f976b80 10 client.0 _ll_register_callbacks cb 0x55e12e6f46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e815c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e812f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e815c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59410/0 (socket says 172.21.15.86:59410) 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6de00700 1 -- 172.21.15.86:0/3154699224 learned_addr learned my addr 172.21.15.86:0/3154699224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6e800700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e8138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6e800700 1 -- 172.21.15.86:0/3154699224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 msgr2=0x55e12e815c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6e800700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e815c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6e800700 1 -- 172.21.15.86:0/3154699224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 msgr2=0x55e12e812f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6e800700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e812f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6e800700 1 -- 172.21.15.86:0/3154699224 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12e6a9880 con 0x55e12e813450 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.358+0000 7f4c6d400700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e812f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c6e800700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e8138d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009fd0 tx=0x7f4c6800b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fab689fde168063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c4fe00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c4fe00700 1 -- 172.21.15.86:0/3154699224 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68008dc0 con 0x55e12e813450 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c60019ae0 con 0x55e12e813450 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c4fe00700 1 -- 172.21.15.86:0/3154699224 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4c68008020 con 0x55e12e813450 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c4fe00700 1 -- 172.21.15.86:0/3154699224 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c68008220 con 0x55e12e813450 2024-04-25T19:02:45.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c7f976b80 10 client.25091 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.359+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e12e6f2aa0 con 0x55e12e813450 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c4fe00700 1 -- 172.21.15.86:0/3154699224 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c60019ae0 con 0x55e12e813450 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c4fe00700 1 -- 172.21.15.86:0/3154699224 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4c68034050 con 0x55e12e813450 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c7f976b80 10 client.25091 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c7f976b80 10 client.25091 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c7f976b80 20 client.25091 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e8239a0 0x55e12e824e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c7f976b80 4 client.25091 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.360+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e12e769630 con 0x55e12e8239a0 2024-04-25T19:02:45.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.361+0000 7f4c6de00700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e8239a0 0x55e12e824e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.361+0000 7f4c6de00700 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e8239a0 0x55e12e824e10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd2f72ef37add9b server_cookie=4f144a363dd86d6 in_seq=0 out_seq=0 2024-04-25T19:02:45.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.361+0000 7f4c4fe00700 10 client.25091 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c4fe00700 1 -- 172.21.15.86:0/3154699224 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e12e769630 con 0x55e12e8239a0 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c4fe00700 10 client.25091 handle_command_reply: tid=0 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 client.25091 shutdown 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 msgr2=0x55e12e8138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e8138d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009fd0 tx=0x7f4c6800b720 comp rx=0 tx=0).stop 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 shutdown_connections 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e8239a0 0x55e12e824e10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e8138d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c4fe00700 0 client.25091 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e815c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/3154699224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e812f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.362+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 >> 172.21.15.86:0/3154699224 conn(0x55e12e737f00 msgr2=0x55e12e803e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.363+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 shutdown_connections 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.363+0000 7f4c7f976b80 1 -- 172.21.15.86:0/3154699224 wait complete. 2024-04-25T19:02:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.363+0000 7f4c7f976b80 20 client.25091 trim_cache size 0 max 16384 2024-04-25T19:02:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.385+0000 7f4c7f976b80 1 Processor -- start 2024-04-25T19:02:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.385+0000 7f4c7f976b80 1 -- start start 2024-04-25T19:02:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.385+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e809ef0 0x55e12e81fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e813450 0x55e12e82e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e12e823150 con 0x55e12e80f780 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e12e823820 con 0x55e12e809ef0 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e12e823ef0 con 0x55e12e813450 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42244/0 (socket says 172.21.15.86:42244) 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 -- 172.21.15.86:0/525835994 learned_addr learned my addr 172.21.15.86:0/525835994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6de00700 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e809ef0 0x55e12e81fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6e800700 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e813450 0x55e12e82e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 -- 172.21.15.86:0/525835994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e813450 msgr2=0x55e12e82e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e813450 0x55e12e82e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 -- 172.21.15.86:0/525835994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e809ef0 msgr2=0x55e12e81fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e809ef0 0x55e12e81fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.386+0000 7f4c6d400700 1 -- 172.21.15.86:0/525835994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12e806080 con 0x55e12e80f780 2024-04-25T19:02:45.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.387+0000 7f4c6d400700 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005e90 tx=0x7f4c5c0cd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e024c4439e4b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.387+0000 7f4c5aa00700 1 -- 172.21.15.86:0/525835994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0ba070 con 0x55e12e80f780 2024-04-25T19:02:45.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.387+0000 7f4c5aa00700 1 -- 172.21.15.86:0/525835994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c09a530 con 0x55e12e80f780 2024-04-25T19:02:45.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.387+0000 7f4c5aa00700 1 -- 172.21.15.86:0/525835994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c01d040 con 0x55e12e80f780 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.387+0000 7f4c7f976b80 1 -- 172.21.15.86:0/525835994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 msgr2=0x55e12e8045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.387+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005e90 tx=0x7f4c5c0cd920 comp rx=0 tx=0).stop 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 -- 172.21.15.86:0/525835994 shutdown_connections 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e813450 0x55e12e82e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e809ef0 0x55e12e81fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/525835994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 -- 172.21.15.86:0/525835994 >> 172.21.15.86:0/525835994 conn(0x55e12e737f00 msgr2=0x55e12e80a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 -- 172.21.15.86:0/525835994 shutdown_connections 2024-04-25T19:02:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 -- 172.21.15.86:0/525835994 wait complete. 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 -- start start 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.388+0000 7f4c7f976b80 1 Processor -- start 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e8040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e82c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e12e824c90 con 0x55e12e80f780 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e12e8245c0 con 0x55e12e813450 2024-04-25T19:02:45.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e12e825360 con 0x55e12e809ef0 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c7f976b80 10 client.0 _ll_register_callbacks cb 0x55e12e6f46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c6de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e8040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c6d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42252/0 (socket says 172.21.15.86:42252) 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c6d400700 1 -- 172.21.15.86:0/1959600816 learned_addr learned my addr 172.21.15.86:0/1959600816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.389+0000 7f4c6e800700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e82c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6d400700 1 -- 172.21.15.86:0/1959600816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 msgr2=0x55e12e8040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6d400700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e8040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6d400700 1 -- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 msgr2=0x55e12e82c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6d400700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e82c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6d400700 1 -- 172.21.15.86:0/1959600816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12e82ce00 con 0x55e12e80f780 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6e800700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e82c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c6d400700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c09a780 tx=0x7f4c5c098600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5321b0eac546c0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c4fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c4fe00700 1 -- 172.21.15.86:0/1959600816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0ba070 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c4fe00700 1 -- 172.21.15.86:0/1959600816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f4c5c09c690 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c09a530 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.390+0000 7f4c4fe00700 1 -- 172.21.15.86:0/1959600816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c01d050 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 10 client.14973 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e12e893aa0 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c4fe00700 1 -- 172.21.15.86:0/1959600816 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c09a530 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c4fe00700 1 -- 172.21.15.86:0/1959600816 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4c5c09b680 con 0x55e12e80f780 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 10 client.14973 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 10 client.14973 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 20 client.14973 populate_metadata read hostname 'smithi086' 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e898110 0x55e12e896e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 4 client.14973 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:02:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.391+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "status"}) -- 0x55e12e7636d0 con 0x55e12e898110 2024-04-25T19:02:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.392+0000 7f4c6de00700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e898110 0x55e12e896e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.392+0000 7f4c6de00700 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e898110 0x55e12e896e20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55dfa8aa45f75f78 server_cookie=ef3d93a7aa5f2c74 in_seq=0 out_seq=0 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.392+0000 7f4c4fe00700 10 client.14973 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.392+0000 7f4c4fe00700 1 -- 172.21.15.86:0/1959600816 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e12e7636d0 con 0x55e12e898110 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.392+0000 7f4c4fe00700 10 client.14973 handle_command_reply: tid=0 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.392+0000 7f4c7f976b80 1 client.14973 shutdown 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 msgr2=0x55e12e8045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c09a780 tx=0x7f4c5c098600 comp rx=0 tx=0).stop 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 shutdown_connections 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e12e898110 0x55e12e896e20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e12e813450 0x55e12e82c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e12e809ef0 0x55e12e8040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c4fe00700 0 client.14973 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 --2- 172.21.15.86:0/1959600816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e12e80f780 0x55e12e8045e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 >> 172.21.15.86:0/1959600816 conn(0x55e12e737f00 msgr2=0x55e12e811760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 shutdown_connections 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 1 -- 172.21.15.86:0/1959600816 wait complete. 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.393+0000 7f4c7f976b80 20 client.14973 trim_cache size 0 max 16384 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T19:02:45.400 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24527, 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 233.817215299, 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 254.325329404 2024-04-25T19:02:45.401 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c540a3cc0 msgr2=0x7f4c540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c540a3cc0 0x7f4c540a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009e60 tx=0x7f4c68009360 comp rx=0 tx=0).stop 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 msgr2=0x7f4c70101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 0x7f4c70101f40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d750 tx=0x7f4c5c00da60 comp rx=0 tx=0).stop 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 shutdown_connections 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f4c540a3cc0 0x7f4c540a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c70108a20 0x7f4c70102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c7006afe0 0x7f4c70101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 --2- 172.21.15.86:0/2213589137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c7006b980 0x7f4c70101f40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:45.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 >> 172.21.15.86:0/2213589137 conn(0x7f4c700788b0 msgr2=0x7f4c700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:45.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 shutdown_connections 2024-04-25T19:02:45.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:45.394+0000 7f4c75000700 1 -- 172.21.15.86:0/2213589137 wait complete. 2024-04-25T19:02:46.244 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T19:02:46.254 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T19:02:46.254 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T19:02:46.254 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T19:02:46.254 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T19:02:46.255 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T19:02:46.255 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:02:46.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 -- 172.21.15.86:0/1108651373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 msgr2=0x7f4654105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 --2- 172.21.15.86:0/1108651373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f4654105850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4640009900 tx=0x7f464002a410 comp rx=0 tx=0).stop 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 -- 172.21.15.86:0/1108651373 shutdown_connections 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 --2- 172.21.15.86:0/1108651373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4654105d90 0x7f465410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 --2- 172.21.15.86:0/1108651373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f4654105850 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 --2- 172.21.15.86:0/1108651373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46541041e0 0x7f4654104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 -- 172.21.15.86:0/1108651373 >> 172.21.15.86:0/1108651373 conn(0x7f46540ff770 msgr2=0x7f4654101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 -- 172.21.15.86:0/1108651373 shutdown_connections 2024-04-25T19:02:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.453+0000 7f4659800640 1 -- 172.21.15.86:0/1108651373 wait complete. 2024-04-25T19:02:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.454+0000 7f4659800640 1 Processor -- start 2024-04-25T19:02:46.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.454+0000 7f4659800640 1 -- start start 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4659800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46541041e0 0x7f465419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4659800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f465419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4659800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4654105d90 0x7f46541a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4659800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f465410fd70 con 0x7f46541053d0 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4659800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f465410fbf0 con 0x7f4654105d90 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4659800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f465410fef0 con 0x7f46541041e0 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f465419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46541041e0 0x7f465419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4653400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4654105d90 0x7f46541a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f465419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42270/0 (socket says 172.21.15.86:42270) 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 -- 172.21.15.86:0/829831313 learned_addr learned my addr 172.21.15.86:0/829831313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 -- 172.21.15.86:0/829831313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46541041e0 msgr2=0x7f465419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46541041e0 0x7f465419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 -- 172.21.15.86:0/829831313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4654105d90 msgr2=0x7f46541a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4654105d90 0x7f46541a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.455+0000 7f4652000640 1 -- 172.21.15.86:0/829831313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46541a1f50 con 0x7f46541053d0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.456+0000 7f4652000640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f465419a870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4640005ab0 tx=0x7f4640003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.456+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464003f070 con 0x7f46541053d0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.456+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4640031c50 con 0x7f46541053d0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.456+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640035030 con 0x7f46541053d0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.456+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46541a21e0 con 0x7f46541053d0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.456+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46541a2680 con 0x7f46541053d0 2024-04-25T19:02:46.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.458+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465410b500 con 0x7f46541053d0 2024-04-25T19:02:46.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.460+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4640042070 con 0x7f46541053d0 2024-04-25T19:02:46.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.461+0000 7f464b400640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f46380a3cd0 0x7f46380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.462+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f46400f0b20 con 0x7f46541053d0 2024-04-25T19:02:46.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.462+0000 7f4652a00640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f46380a3cd0 0x7f46380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:46.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.462+0000 7f4652a00640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f46380a3cd0 0x7f46380a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4654105230 tx=0x7f4644005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.466+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46400bcdf0 con 0x7f46541053d0 2024-04-25T19:02:46.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:46.720+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f4654062b80 con 0x7f46541053d0 2024-04-25T19:02:46.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:46 smithi086 ceph-mon[27783]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:46.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:46 smithi183 ceph-mon[32344]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:46 smithi194 ceph-mon[32740]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T19:02:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.511+0000 7f464b400640 1 -- 172.21.15.86:0/829831313 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f46400c10f0 con 0x7f46541053d0 2024-04-25T19:02:47.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.514+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f46380a3cd0 msgr2=0x7f46380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.514+0000 7f4659800640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f46380a3cd0 0x7f46380a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4654105230 tx=0x7f4644005f90 comp rx=0 tx=0).stop 2024-04-25T19:02:47.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.514+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 msgr2=0x7f465419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.514+0000 7f4659800640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f465419a870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4640005ab0 tx=0x7f4640003b90 comp rx=0 tx=0).stop 2024-04-25T19:02:47.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 shutdown_connections 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f46380a3cd0 0x7f46380a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4654105d90 0x7f46541a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46541053d0 0x7f465419a870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 --2- 172.21.15.86:0/829831313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46541041e0 0x7f465419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 >> 172.21.15.86:0/829831313 conn(0x7f46540ff770 msgr2=0x7f46541086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 shutdown_connections 2024-04-25T19:02:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.515+0000 7f4659800640 1 -- 172.21.15.86:0/829831313 wait complete. 2024-04-25T19:02:47.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T19:02:47.606 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:47 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/829831313' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T19:02:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 -- 172.21.15.86:0/3291517857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50076910 msgr2=0x7f0d50076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 --2- 172.21.15.86:0/3291517857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50076910 0x7f0d50076d30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009860 tx=0x7f0d4402a340 comp rx=0 tx=0).stop 2024-04-25T19:02:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 -- 172.21.15.86:0/3291517857 shutdown_connections 2024-04-25T19:02:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 --2- 172.21.15.86:0/3291517857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d500784f0 0x7f0d50078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 --2- 172.21.15.86:0/3291517857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d50077b50 0x7f0d50077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 --2- 172.21.15.86:0/3291517857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50076910 0x7f0d50076d30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 -- 172.21.15.86:0/3291517857 >> 172.21.15.86:0/3291517857 conn(0x7f0d500ff770 msgr2=0x7f0d50101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 -- 172.21.15.86:0/3291517857 shutdown_connections 2024-04-25T19:02:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.730+0000 7f0d58200640 1 -- 172.21.15.86:0/3291517857 wait complete. 2024-04-25T19:02:47.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.732+0000 7f0d58200640 1 Processor -- start 2024-04-25T19:02:47.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.732+0000 7f0d58200640 1 -- start start 2024-04-25T19:02:47.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.732+0000 7f0d58200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 0x7f0d5019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.732+0000 7f0d58200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50077b50 0x7f0d5019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.732+0000 7f0d58200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d500784f0 0x7f0d501a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.732+0000 7f0d58200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501140e0 con 0x7f0d500784f0 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d58200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50113f60 con 0x7f0d50077b50 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d58200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50114260 con 0x7f0d50076910 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 0x7f0d5019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 0x7f0d5019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59472/0 (socket says 172.21.15.86:59472) 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 -- 172.21.15.86:0/1775472950 learned_addr learned my addr 172.21.15.86:0/1775472950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d54e00640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50077b50 0x7f0d5019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d56200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d500784f0 0x7f0d501a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 -- 172.21.15.86:0/1775472950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50077b50 msgr2=0x7f0d5019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50077b50 0x7f0d5019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 -- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d500784f0 msgr2=0x7f0d501a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d500784f0 0x7f0d501a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 -- 172.21.15.86:0/1775472950 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a6330 con 0x7f0d50076910 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.733+0000 7f0d55800640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 0x7f0d5019e760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005940 tx=0x7f0d4402ae20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.734+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44040020 con 0x7f0d50076910 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.734+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d44030e50 con 0x7f0d50076910 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.734+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44037040 con 0x7f0d50076910 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.734+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a65c0 con 0x7f0d50076910 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.734+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a6b00 con 0x7f0d50076910 2024-04-25T19:02:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.735+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5004fce0 con 0x7f0d50076910 2024-04-25T19:02:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.737+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0d44043020 con 0x7f0d50076910 2024-04-25T19:02:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.738+0000 7f0d4e000640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 0x7f0d380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.738+0000 7f0d54e00640 1 -- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 msgr2=0x7f0d380a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.738+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f0d440f05c0 con 0x7f0d50076910 2024-04-25T19:02:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.738+0000 7f0d54e00640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 0x7f0d380a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.742+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d4404a020 con 0x7f0d50076910 2024-04-25T19:02:47.882 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/829831313' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T19:02:47.882 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: ignoring --setuser ceph since I am not root 2024-04-25T19:02:47.883 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: ignoring --setgroup ceph since I am not root 2024-04-25T19:02:47.883 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-mgr[33729]: -- 172.21.15.183:0/1748042733 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0f86e65a0 con 0x55b0f86bb000 2024-04-25T19:02:47.883 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:47.695+0000 7f90c73f7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:02:47.883 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:47.880+0000 7f90c73f7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:47.885 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/829831313' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T19:02:47.885 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: ignoring --setuser ceph since I am not root 2024-04-25T19:02:47.885 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: ignoring --setgroup ceph since I am not root 2024-04-25T19:02:47.885 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-mgr[34607]: -- 172.21.15.194:0/2742471865 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e3100a5a0 con 0x562e30fdec00 2024-04-25T19:02:47.886 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:47.697+0000 7f0b37a63200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:02:47.886 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:47.883+0000 7f0b37a63200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.939+0000 7f0d54e00640 1 -- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 msgr2=0x7f0d380a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.939+0000 7f0d54e00640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 0x7f0d380a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:47.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.948+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f0d50110b50 con 0x7f0d50076910 2024-04-25T19:02:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.955+0000 7f0d4e000640 1 -- 172.21.15.86:0/1775472950 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f0d440bcee0 con 0x7f0d50076910 2024-04-25T19:02:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 msgr2=0x7f0d380a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:02:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 0x7f0d380a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 msgr2=0x7f0d5019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 0x7f0d5019e760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005940 tx=0x7f0d4402ae20 comp rx=0 tx=0).stop 2024-04-25T19:02:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 shutdown_connections 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f0d380a3d20 0x7f0d380a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d500784f0 0x7f0d501a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.957+0000 7f0d58200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50077b50 0x7f0d5019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.958+0000 7f0d58200640 1 --2- 172.21.15.86:0/1775472950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d50076910 0x7f0d5019e760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.958+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 >> 172.21.15.86:0/1775472950 conn(0x7f0d500ff770 msgr2=0x7f0d50101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.958+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 shutdown_connections 2024-04-25T19:02:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:47.958+0000 7f0d58200640 1 -- 172.21.15.86:0/1775472950 wait complete. 2024-04-25T19:02:47.976 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: ignoring --setuser ceph since I am not root 2024-04-25T19:02:47.977 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: ignoring --setgroup ceph since I am not root 2024-04-25T19:02:47.977 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:47.841+0000 7ff0d1fe1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:02:47.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T19:02:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 -- 172.21.15.86:0/1462653967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8102850 msgr2=0x7fe8a8104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 --2- 172.21.15.86:0/1462653967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8102850 0x7fe8a8104c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe89802a400 comp rx=0 tx=0).stop 2024-04-25T19:02:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 -- 172.21.15.86:0/1462653967 shutdown_connections 2024-04-25T19:02:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 --2- 172.21.15.86:0/1462653967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8a8107ab0 0x7fe8a8109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 --2- 172.21.15.86:0/1462653967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8105180 0x7fe8a8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 --2- 172.21.15.86:0/1462653967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8102850 0x7fe8a8104c40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 -- 172.21.15.86:0/1462653967 >> 172.21.15.86:0/1462653967 conn(0x7fe8a80fc240 msgr2=0x7fe8a80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 -- 172.21.15.86:0/1462653967 shutdown_connections 2024-04-25T19:02:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.157+0000 7fe8aca00640 1 -- 172.21.15.86:0/1462653967 wait complete. 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.158+0000 7fe8aca00640 1 Processor -- start 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.158+0000 7fe8aca00640 1 -- start start 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8aca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8a8102850 0x7fe8a819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8aca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 0x7fe8a819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8aca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8107ab0 0x7fe8a81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8aca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810d970 con 0x7fe8a8107ab0 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8aca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810d7f0 con 0x7fe8a8105180 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8aca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a810daf0 con 0x7fe8a8102850 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 0x7fe8a819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8107ab0 0x7fe8a81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8a8102850 0x7fe8a819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 0x7fe8a819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:44714/0 (socket says 172.21.15.86:44714) 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 -- 172.21.15.86:0/2565020184 learned_addr learned my addr 172.21.15.86:0/2565020184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 -- 172.21.15.86:0/2565020184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8a8102850 msgr2=0x7fe8a819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8a8102850 0x7fe8a819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 -- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8107ab0 msgr2=0x7fe8a81a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8107ab0 0x7fe8a81a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.159+0000 7fe8a6a00640 1 -- 172.21.15.86:0/2565020184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a81a1e40 con 0x7fe8a8105180 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.160+0000 7fe8a6a00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 0x7fe8a819a760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9a0 tx=0x7fe89400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.160+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400cd80 con 0x7fe8a8105180 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.160+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8940035f0 con 0x7fe8a8105180 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.160+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81a2130 con 0x7fe8a8105180 2024-04-25T19:02:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.160+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894010690 con 0x7fe8a8105180 2024-04-25T19:02:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.160+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a26d0 con 0x7fe8a8105180 2024-04-25T19:02:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.161+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8109600 con 0x7fe8a8105180 2024-04-25T19:02:48.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.164+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe894004000 con 0x7fe8a8105180 2024-04-25T19:02:48.166 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:47.984+0000 7f90c73f7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:02:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.165+0000 7fe89fe00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 0x7fe88c0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.165+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7fe8940d06f0 con 0x7fe8a8105180 2024-04-25T19:02:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.165+0000 7fe8a7400640 1 -- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 msgr2=0x7fe88c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.165+0000 7fe8a7400640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 0x7fe88c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:48.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.169+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8940042f0 con 0x7fe8a8105180 2024-04-25T19:02:48.228 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:48.088+0000 7ff0d1fe1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:48.229 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:48.224+0000 7ff0d1fe1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:02:48.231 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:47.988+0000 7f0b37a63200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:02:48.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.366+0000 7fe8a7400640 1 -- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 msgr2=0x7fe88c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:48.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.366+0000 7fe8a7400640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 0x7fe88c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:48.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.374+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe8a802d5c0 con 0x7fe8a8105180 2024-04-25T19:02:48.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.381+0000 7fe89fe00640 1 -- 172.21.15.86:0/2565020184 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fe89409d010 con 0x7fe8a8105180 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 msgr2=0x7fe88c0a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 0x7fe88c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 msgr2=0x7fe8a819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 0x7fe8a819a760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9a0 tx=0x7fe89400ee60 comp rx=0 tx=0).stop 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 shutdown_connections 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7fe88c0a3c70 0x7fe88c0a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8107ab0 0x7fe8a81a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8a8105180 0x7fe8a819a760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 --2- 172.21.15.86:0/2565020184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8a8102850 0x7fe8a819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 >> 172.21.15.86:0/2565020184 conn(0x7fe8a80fc240 msgr2=0x7fe8a8102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 shutdown_connections 2024-04-25T19:02:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:48.384+0000 7fe8aca00640 1 -- 172.21.15.86:0/2565020184 wait complete. 2024-04-25T19:02:48.403 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T19:02:48.441 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T19:02:48.441 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T19:02:48.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:48.442 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T19:02:48.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:48.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T19:02:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-25T19:02:48.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:48.613 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T19:02:48.614 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:02:48.614 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:02:48.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T19:02:48.667 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:48.294+0000 7f90c73f7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:02:48.667 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:48.378+0000 7f90c73f7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:02:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:48 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/829831313' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T19:02:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:48 smithi183 ceph-mon[32344]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-25T19:02:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:48 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' 2024-04-25T19:02:48.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:48 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' 2024-04-25T19:02:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:48 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/829831313' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T19:02:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:48 smithi194 ceph-mon[32740]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-25T19:02:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:48 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' 2024-04-25T19:02:48.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:48 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' 2024-04-25T19:02:48.733 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:48 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:48.298+0000 7f0b37a63200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:02:48.733 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:48 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:48.383+0000 7f0b37a63200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:02:48.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:48.632+0000 7ff0d1fe1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:02:48.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:48.750+0000 7ff0d1fe1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:02:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/829831313' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T19:02:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-mon[27783]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-25T19:02:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' 2024-04-25T19:02:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:48 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' 2024-04-25T19:02:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:49.910 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:49.563+0000 7f90c73f7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:02:49.910 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:49.735+0000 7f90c73f7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:49.918 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:49.573+0000 7f0b37a63200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:02:49.918 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:49.744+0000 7f0b37a63200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:50.166 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:49.908+0000 7f90c73f7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:02:50.482 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:49.916+0000 7f0b37a63200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:02:50.482 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:50 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:50.203+0000 7f0b37a63200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:02:50.482 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:50 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:50.295+0000 7f0b37a63200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:02:50.495 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:50.198+0000 7f90c73f7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:02:50.495 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:50.290+0000 7f90c73f7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:02:50.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:50.341+0000 7ff0d1fe1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:02:50.718 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:50.576+0000 7ff0d1fe1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:50.916 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:50.493+0000 7f90c73f7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:02:50.917 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:50.688+0000 7f90c73f7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:50.975 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:50.802+0000 7ff0d1fe1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:02:50.981 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:50 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:50.500+0000 7f0b37a63200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:02:50.981 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:50 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:50.694+0000 7f0b37a63200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.256+0000 7f1b64e00700 1 -- 172.21.15.86:0/3132414310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b6006b0f0 msgr2=0x7f1b6010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.256+0000 7f1b64e00700 1 --2- 172.21.15.86:0/3132414310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b6006b0f0 0x7f1b6010f900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800b0a0 tx=0x7f1b5800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 -- 172.21.15.86:0/3132414310 shutdown_connections 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 --2- 172.21.15.86:0/3132414310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b6006b0f0 0x7f1b6010f900 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 --2- 172.21.15.86:0/3132414310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b60108a30 0x7f1b60108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 --2- 172.21.15.86:0/3132414310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006a750 0x7f1b6006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 -- 172.21.15.86:0/3132414310 >> 172.21.15.86:0/3132414310 conn(0x7f1b600788f0 msgr2=0x7f1b60078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 -- 172.21.15.86:0/3132414310 shutdown_connections 2024-04-25T19:02:51.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.257+0000 7f1b64e00700 1 -- 172.21.15.86:0/3132414310 wait complete. 2024-04-25T19:02:51.261 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:51.191+0000 7ff0d1fe1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.258+0000 7f1b64e00700 1 Processor -- start 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.258+0000 7f1b64e00700 1 -- start start 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b64e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 0x7f1b60199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b64e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006b0f0 0x7f1b601995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b64e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b60108a30 0x7f1b6019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b64e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60112bd0 con 0x7f1b6006b0f0 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b64e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60112a50 con 0x7f1b6006a750 2024-04-25T19:02:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b64e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60112d50 con 0x7f1b60108a30 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 0x7f1b60199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 0x7f1b60199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50542/0 (socket says 172.21.15.86:50542) 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 -- 172.21.15.86:0/1956443758 learned_addr learned my addr 172.21.15.86:0/1956443758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5e800700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b60108a30 0x7f1b6019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5d400700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006b0f0 0x7f1b601995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 -- 172.21.15.86:0/1956443758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b60108a30 msgr2=0x7f1b6019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b60108a30 0x7f1b6019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 -- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006b0f0 msgr2=0x7f1b601995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006b0f0 0x7f1b601995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5de00700 1 -- 172.21.15.86:0/1956443758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6019e090 con 0x7f1b6006a750 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.259+0000 7f1b5d400700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006b0f0 0x7f1b601995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:51.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.260+0000 7f1b5de00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 0x7f1b60199080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e9a0 tx=0x7f1b5000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.260+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000cbd0 con 0x7f1b6006a750 2024-04-25T19:02:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.260+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6019e380 con 0x7f1b6006a750 2024-04-25T19:02:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.260+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b5000cd70 con 0x7f1b6006a750 2024-04-25T19:02:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.260+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b500186f0 con 0x7f1b6006a750 2024-04-25T19:02:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.260+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601a5c00 con 0x7f1b6006a750 2024-04-25T19:02:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.261+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b60069730 con 0x7f1b6006a750 2024-04-25T19:02:51.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.263+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1b50010a00 con 0x7f1b6006a750 2024-04-25T19:02:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.264+0000 7f1b56a00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 0x7f1b440a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.264+0000 7f1b5d400700 1 -- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 msgr2=0x7f1b440a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.265+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f1b500d0710 con 0x7f1b6006a750 2024-04-25T19:02:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.265+0000 7f1b5d400700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 0x7f1b440a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.268+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b5009d030 con 0x7f1b6006a750 2024-04-25T19:02:51.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.466+0000 7f1b5d400700 1 -- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 msgr2=0x7f1b440a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:51.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.466+0000 7f1b5d400700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 0x7f1b440a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:51.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.540+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b6002df80 con 0x7f1b6006a750 2024-04-25T19:02:51.543 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:51.319+0000 7ff0d1fe1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:02:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.541+0000 7f1b56a00700 1 -- 172.21.15.86:0/1956443758 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f1b5009c930 con 0x7f1b6006a750 2024-04-25T19:02:51.543 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T19:02:51.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 msgr2=0x7f1b440a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 0x7f1b440a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 msgr2=0x7f1b60199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 0x7f1b60199080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e9a0 tx=0x7f1b5000ed60 comp rx=0 tx=0).stop 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 shutdown_connections 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f1b440a3d10 0x7f1b440a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b60108a30 0x7f1b6019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b6006a750 0x7f1b60199080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 --2- 172.21.15.86:0/1956443758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b6006b0f0 0x7f1b601995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 >> 172.21.15.86:0/1956443758 conn(0x7f1b600788f0 msgr2=0x7f1b600ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 shutdown_connections 2024-04-25T19:02:51.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:51.544+0000 7f1b64e00700 1 -- 172.21.15.86:0/1956443758 wait complete. 2024-04-25T19:02:51.595 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:51.290+0000 7f0b37a63200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:02:51.595 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:51.287+0000 7f90c73f7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:02:51.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:51 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1956443758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T19:02:51.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:51.581+0000 7ff0d1fe1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:02:51.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:51 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1956443758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T19:02:51.917 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:51.709+0000 7f90c73f7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:02:51.917 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:51.804+0000 7f90c73f7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:02:51.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:51 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1956443758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T19:02:51.982 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:51.711+0000 7f0b37a63200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:02:51.982 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:51.806+0000 7f0b37a63200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:02:52.310 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:51.839+0000 7ff0d1fe1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:02:52.387 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:02:52.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:52.388 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T19:02:52.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:52.006+0000 7f90c73f7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:02:52.438 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41746sec preferred_lft 41746sec 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T19:02:52.439 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:02:52.440 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T19:02:52.440 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:52.440 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T19:02:52.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-25T19:02:52.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-25T19:02:52.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-25T19:02:52.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T19:02:52.441 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T19:02:52.481 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:52.007+0000 7f0b37a63200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:02:52.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:52.820 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:52.657+0000 7ff0d1fe1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:02:52.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:52.869 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:52.531+0000 7f0b37a63200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:02:52.869 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:52.625+0000 7f0b37a63200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:02:52.870 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:52.532+0000 7f90c73f7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:02:52.870 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:52.626+0000 7f90c73f7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:02:52.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:52.880 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:02:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T19:02:52.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:52.948 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T19:02:53.006 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T19:02:53.006 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:02:53.006 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:02:53.006 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:02:53.007 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:02:53.007 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:53.008 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T19:02:53.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:02:53.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:02:53.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:02:53.008 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T19:02:53.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:53.166 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:52.868+0000 7f90c73f7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:02:53.167 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:53.002+0000 7f90c73f7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:02:53.167 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:53.098+0000 7f90c73f7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:02:53.211 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:52.866+0000 7f0b37a63200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:02:53.211 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:52.999+0000 7f0b37a63200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:02:53.212 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:53.093+0000 7f0b37a63200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:02:53.212 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:53.208+0000 7f0b37a63200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:53.367 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:53.251+0000 7ff0d1fe1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:02:53.520 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:53.213+0000 7f90c73f7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:53.664 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:53.386+0000 7ff0d1fe1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:02:53.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:53.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:53.695 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T19:02:53.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:53.740 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-25T19:02:53.827 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:53.827 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T19:02:53.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:02:53.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T19:02:53.828 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T19:02:53.916 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:53.518+0000 7f90c73f7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:02:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:53.949 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:53.661+0000 7ff0d1fe1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:02:53.981 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:53.514+0000 7f0b37a63200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:02:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:54.000 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T19:02:54.000 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:54.000 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T19:02:54.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T19:02:54.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T19:02:54.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T19:02:54.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T19:02:54.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T19:02:54.001 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T19:02:54.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:54.288 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:02:54.288 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T19:02:54.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T19:02:54.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T19:02:54.288 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T19:02:54.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:54.401 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:54.115+0000 7f0b37a63200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:02:54.401 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z[34603]: 2024-04-25T19:02:54.330+0000 7f0b37a63200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:02:54.401 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:54.118+0000 7f90c73f7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:02:54.401 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y[33725]: 2024-04-25T19:02:54.333+0000 7f90c73f7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:02:54.464 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:54.340+0000 7ff0d1fe1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:02:54.464 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: Standby manager daemon z restarted 2024-04-25T19:02:54.464 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: Standby manager daemon z started 2024-04-25T19:02:54.464 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T19:02:54.464 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: Standby manager daemon y restarted 2024-04-25T19:02:54.464 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: Standby manager daemon y started 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T19:02:54.465 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-mon[27783]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T19:02:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:54.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:02:54.476 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T19:02:54.515 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:02:54.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:02:54.515 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:02:54.602 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T19:02:54.602 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T19:02:54.602 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/e156f586-1b61-4077-95bf-7820cb4279c7 2024-04-25T19:02:54.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:54.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/e156f586-1b61-4077-95bf-7820cb4279c7 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T19:02:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: Standby manager daemon z restarted 2024-04-25T19:02:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: Standby manager daemon z started 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: Standby manager daemon y restarted 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: Standby manager daemon y started 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T19:02:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T19:02:54.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:54 smithi183 ceph-mon[32344]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: Standby manager daemon z restarted 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: Standby manager daemon z started 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: Standby manager daemon y restarted 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: Standby manager daemon y started 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T19:02:54.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.194:0/1782537785' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T19:02:54.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T19:02:54.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T19:02:54.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:54 smithi194 ceph-mon[32740]: from='mgr.? 172.21.15.183:0/455986962' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T19:02:54.762 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T19:02:54.762 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-25T19:02:54.815 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:54.472+0000 7ff0d1fe1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:02:55.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:54.812+0000 7ff0d1fe1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:02:55.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:55.001+0000 7ff0d1fe1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:02:55.402 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:55.135+0000 7ff0d1fe1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:02:55.403 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:55.292+0000 7ff0d1fe1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:02:55.526 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T19:02:55.526 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-25T19:02:55.526 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.183:3300,172.21.15.194:3300" 2024-04-25T19:02:55.527 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 0@1ec2eb56-0334-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/e156f586-1b61-4077-95bf-7820cb4279c7 2024-04-25T19:02:55.527 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.86:3300/172.21.15.183:3300/172.21.15.194:3300" will pass to kernel 2024-04-25T19:02:55.529 INFO:teuthology.orchestra.run.smithi086.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T19:02:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: You just mounted a file system that supports labels which does not 2024-04-25T19:02:55.529 INFO:teuthology.orchestra.run.smithi086.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T19:02:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T19:02:55.530 INFO:teuthology.orchestra.run.smithi086.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T19:02:55.534 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T19:02:55.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:02:55.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:02:55.606 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:02:55.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:55 smithi183 ceph-mon[32344]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-25T19:02:55.679 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:55 smithi086 ceph-mon[27783]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-25T19:02:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:02:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:55.723 INFO:teuthology.orchestra.run.smithi086.stdout:client25136 2024-04-25T19:02:55.723 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T19:02:55.723 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:02:55.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:55 smithi194 ceph-mon[32740]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-25T19:02:55.769 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:02:55.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:02:55.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:55.927 INFO:teuthology.orchestra.run.smithi086.stdout:client25136 2024-04-25T19:02:55.928 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T19:02:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T19:02:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:56.038 INFO:teuthology.orchestra.run.smithi086.stdout:client25136 2024-04-25T19:02:56.039 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968.client25136/status') 2024-04-25T19:02:56.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:55.677+0000 7ff0d1fe1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:02:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:02:56.166 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.25136 (3)172.21.15.86:0/2063789426 2024-04-25T19:02:56.166 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-25T19:02:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-25T19:02:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-25T19:02:56.166 INFO:teuthology.orchestra.run.smithi086.stderr:71 bytes copied, 7.8966e-05 s, 899 kB/s 2024-04-25T19:02:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:02:56.179 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T19:02:56.189 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T19:02:56.189 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:02:56.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T19:02:56.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 -- 172.21.15.86:0/928358567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc102850 msgr2=0x7f2efc104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 --2- 172.21.15.86:0/928358567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc102850 0x7f2efc104c40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009900 tx=0x7f2eec02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 -- 172.21.15.86:0/928358567 shutdown_connections 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 --2- 172.21.15.86:0/928358567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2efc107ab0 0x7f2efc109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 --2- 172.21.15.86:0/928358567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 --2- 172.21.15.86:0/928358567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc102850 0x7f2efc104c40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 -- 172.21.15.86:0/928358567 >> 172.21.15.86:0/928358567 conn(0x7f2efc0fc240 msgr2=0x7f2efc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 -- 172.21.15.86:0/928358567 shutdown_connections 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.415+0000 7f2efbe00640 1 -- 172.21.15.86:0/928358567 wait complete. 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 Processor -- start 2024-04-25T19:02:56.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 -- start start 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2efc102850 0x7f2efc19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc107ab0 0x7f2efc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc10d930 con 0x7f2efc107ab0 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc10d7b0 con 0x7f2efc102850 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.416+0000 7f2efbe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc10dab0 con 0x7f2efc105180 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc107ab0 0x7f2efc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:37568/0 (socket says 172.21.15.86:37568) 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 -- 172.21.15.86:0/757047549 learned_addr learned my addr 172.21.15.86:0/757047549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:56.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efaa00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2efc102850 0x7f2efc19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 -- 172.21.15.86:0/757047549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2efc102850 msgr2=0x7f2efc19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2efc102850 0x7f2efc19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 -- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc107ab0 msgr2=0x7f2efc1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc107ab0 0x7f2efc1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.417+0000 7f2efa000640 1 -- 172.21.15.86:0/757047549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc1a1ec0 con 0x7f2efc105180 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.418+0000 7f2efa000640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc19a7e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800e9f0 tx=0x7f2ee800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.418+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800ccc0 con 0x7f2efc105180 2024-04-25T19:02:56.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.418+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ee80054d0 con 0x7f2efc105180 2024-04-25T19:02:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.418+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc1a21b0 con 0x7f2efc105180 2024-04-25T19:02:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.418+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8010430 con 0x7f2efc105180 2024-04-25T19:02:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.418+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc1a26b0 con 0x7f2efc105180 2024-04-25T19:02:56.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.419+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2efc109600 con 0x7f2efc105180 2024-04-25T19:02:56.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.422+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f2ee80046c0 con 0x7f2efc105180 2024-04-25T19:02:56.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.424+0000 7f2ef3400640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 0x7f2ee00a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.424+0000 7f2efaa00640 1 -- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 msgr2=0x7f2ee00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:56.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.424+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8335+0+0 (secure 0 0 0) 0x7f2ee80d0860 con 0x7f2efc105180 2024-04-25T19:02:56.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.424+0000 7f2efaa00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 0x7f2ee00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:02:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.428+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ee809d180 con 0x7f2efc105180 2024-04-25T19:02:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.625+0000 7f2efaa00640 1 -- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 msgr2=0x7f2ee00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/237697797 2024-04-25T19:02:56.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.625+0000 7f2efaa00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 0x7f2ee00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:02:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.671+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f2efc062b50 con 0x7f2efc105180 2024-04-25T19:02:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.729+0000 7f2ef3400640 1 -- 172.21.15.86:0/757047549 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f2ee809ca80 con 0x7f2efc105180 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 msgr2=0x7f2ee00a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 0x7f2ee00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 msgr2=0x7f2efc19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc19a7e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800e9f0 tx=0x7f2ee800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 shutdown_connections 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:6800/237697797,v1:172.21.15.86:6801/237697797] conn(0x7f2ee00a3bf0 0x7f2ee00a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc107ab0 0x7f2efc1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2efc105180 0x7f2efc19a7e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 --2- 172.21.15.86:0/757047549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2efc102850 0x7f2efc19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 >> 172.21.15.86:0/757047549 conn(0x7f2efc0fc240 msgr2=0x7f2efc1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 shutdown_connections 2024-04-25T19:02:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.733+0000 7f2efbe00640 1 -- 172.21.15.86:0/757047549 wait complete. 2024-04-25T19:02:56.755 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:56.501+0000 7ff0d1fe1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:02:56.755 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:56 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/757047549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T19:02:56.755 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:56 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T19:02:56.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T19:02:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 -- 172.21.15.86:0/2518008273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c101fc0 msgr2=0x7f918c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 --2- 172.21.15.86:0/2518008273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c101fc0 0x7f918c1023e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f917c009900 tx=0x7f917c02a400 comp rx=0 tx=0).stop 2024-04-25T19:02:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 -- 172.21.15.86:0/2518008273 shutdown_connections 2024-04-25T19:02:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 --2- 172.21.15.86:0/2518008273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c103b70 0x7f918c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 --2- 172.21.15.86:0/2518008273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f918c1031b0 0x7f918c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 --2- 172.21.15.86:0/2518008273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c101fc0 0x7f918c1023e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 -- 172.21.15.86:0/2518008273 >> 172.21.15.86:0/2518008273 conn(0x7f918c0fd530 msgr2=0x7f918c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 -- 172.21.15.86:0/2518008273 shutdown_connections 2024-04-25T19:02:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.951+0000 7f9190a00640 1 -- 172.21.15.86:0/2518008273 wait complete. 2024-04-25T19:02:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 Processor -- start 2024-04-25T19:02:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 -- start start 2024-04-25T19:02:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 0x7f918c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f918c1031b0 0x7f918c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c103b70 0x7f918c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f918c074630 con 0x7f918c101fc0 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f918c0744b0 con 0x7f918c1031b0 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9190a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f918c0747b0 con 0x7f918c103b70 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9189e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 0x7f918c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9189e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 0x7f918c19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39514/0 (socket says 172.21.15.86:39514) 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.953+0000 7f9189e00640 1 -- 172.21.15.86:0/1131203335 learned_addr learned my addr 172.21.15.86:0/1131203335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f918a800640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c103b70 0x7f918c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189400640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f918c1031b0 0x7f918c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189e00640 1 -- 172.21.15.86:0/1131203335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c103b70 msgr2=0x7f918c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189e00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c103b70 0x7f918c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189e00640 1 -- 172.21.15.86:0/1131203335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f918c1031b0 msgr2=0x7f918c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189e00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f918c1031b0 0x7f918c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189e00640 1 -- 172.21.15.86:0/1131203335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c06baa0 con 0x7f918c101fc0 2024-04-25T19:02:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.954+0000 7f9189e00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 0x7f918c19a3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f917c031500 tx=0x7f917c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.955+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c03f070 con 0x7f918c101fc0 2024-04-25T19:02:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.955+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917c031c50 con 0x7f918c101fc0 2024-04-25T19:02:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.955+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c06bd30 con 0x7f918c101fc0 2024-04-25T19:02:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.955+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c035030 con 0x7f918c101fc0 2024-04-25T19:02:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.955+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c06c270 con 0x7f918c101fc0 2024-04-25T19:02:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.957+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c107af0 con 0x7f918c101fc0 2024-04-25T19:02:56.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.959+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f917c042070 con 0x7f918c101fc0 2024-04-25T19:02:56.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.961+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f917c0f0690 con 0x7f918c101fc0 2024-04-25T19:02:56.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:56.966+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f917c0bd2a0 con 0x7f918c101fc0 2024-04-25T19:02:57.019 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:56 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/757047549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T19:02:57.019 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:56 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T19:02:57.020 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:56 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/757047549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T19:02:57.020 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:56 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T19:02:57.029 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:56.813+0000 7ff0d1fe1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:02:57.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.146+0000 7ff023e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.146+0000 7ff023e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.146+0000 7ff023e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.146+0000 7ff023e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.146+0000 7ff023e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.148+0000 7ff027000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.148+0000 7ff027000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.148+0000 7ff027000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.148+0000 7ff027000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.148+0000 7ff027000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:57.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.833+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f917c0b88f0 con 0x7f918c101fc0 2024-04-25T19:02:57.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.834+0000 7f9182a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f91700aabd0 0x7f91700acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:57.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.834+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f917c038e90 con 0x7f91700aabd0 2024-04-25T19:02:57.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.837+0000 7f9189400640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f91700aabd0 0x7f91700acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.841+0000 7f9189400640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f91700aabd0 0x7f91700acfc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f918c19b880 tx=0x7f9178005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.870+0000 7f9182a00640 1 -- 172.21.15.86:0/1131203335 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f917c038e90 con 0x7f91700aabd0 2024-04-25T19:02:57.872 INFO:teuthology.orchestra.run.smithi086.stdout:{} 2024-04-25T19:02:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.872+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f91700aabd0 msgr2=0x7f91700acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f91700aabd0 0x7f91700acfc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f918c19b880 tx=0x7f9178005f90 comp rx=0 tx=0).stop 2024-04-25T19:02:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 msgr2=0x7f918c19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 0x7f918c19a3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f917c031500 tx=0x7f917c003c10 comp rx=0 tx=0).stop 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 shutdown_connections 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f91700aabd0 0x7f91700acfc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c103b70 0x7f918c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f918c1031b0 0x7f918c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 --2- 172.21.15.86:0/1131203335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f918c101fc0 0x7f918c19a3c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 >> 172.21.15.86:0/1131203335 conn(0x7f918c0fd530 msgr2=0x7f918c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 shutdown_connections 2024-04-25T19:02:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:57.873+0000 7f9190a00640 1 -- 172.21.15.86:0/1131203335 wait complete. 2024-04-25T19:02:57.891 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T19:02:57.903 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T19:02:57.912 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T19:02:57.913 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:02:57.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T19:02:58.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.856+0000 7ff020c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:58.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.856+0000 7ff020c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.856+0000 7ff020c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.856+0000 7ff020c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:02:57.856+0000 7ff020c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: Active manager daemon x restarted 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: Activating manager daemon x 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T19:02:58.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: mgrmap e21: x(active, starting, since 0.014856s), standbys: z, y 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:02:58.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T19:02:58.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: Manager daemon x is now available 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T19:02:58.074 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:57 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T19:02:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 -- 172.21.15.86:0/192222075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134101fc0 msgr2=0x7f81341023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 --2- 172.21.15.86:0/192222075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134101fc0 0x7f81341023e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8124009900 tx=0x7f812402a400 comp rx=0 tx=0).stop 2024-04-25T19:02:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 -- 172.21.15.86:0/192222075 shutdown_connections 2024-04-25T19:02:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 --2- 172.21.15.86:0/192222075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8134103b70 0x7f813410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 --2- 172.21.15.86:0/192222075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 0x7f8134103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 --2- 172.21.15.86:0/192222075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134101fc0 0x7f81341023e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 -- 172.21.15.86:0/192222075 >> 172.21.15.86:0/192222075 conn(0x7f81340fd510 msgr2=0x7f81340ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:58.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.091+0000 7f813a600640 1 -- 172.21.15.86:0/192222075 shutdown_connections 2024-04-25T19:02:58.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.092+0000 7f813a600640 1 -- 172.21.15.86:0/192222075 wait complete. 2024-04-25T19:02:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.092+0000 7f813a600640 1 Processor -- start 2024-04-25T19:02:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 -- start start 2024-04-25T19:02:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8134101fc0 0x7f81341981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 0x7f8134198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 0x7f813419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f813410dac0 con 0x7f8134103b70 2024-04-25T19:02:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f813410d940 con 0x7f8134101fc0 2024-04-25T19:02:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f813a600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f813410dc40 con 0x7f81341031b0 2024-04-25T19:02:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f8133e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 0x7f813419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f8132a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 0x7f8134198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f8133e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 0x7f813419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39620/0 (socket says 172.21.15.86:39620) 2024-04-25T19:02:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f8133400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8134101fc0 0x7f81341981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.093+0000 7f8133e00640 1 -- 172.21.15.86:0/863636752 learned_addr learned my addr 172.21.15.86:0/863636752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8133e00640 1 -- 172.21.15.86:0/863636752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 msgr2=0x7f8134198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8133e00640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 0x7f8134198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8133e00640 1 -- 172.21.15.86:0/863636752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8134101fc0 msgr2=0x7f81341981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8133e00640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8134101fc0 0x7f81341981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8133e00640 1 -- 172.21.15.86:0/863636752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813419fd90 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8132a00640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 0x7f8134198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f8133e00640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 0x7f813419f720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f812c00eb10 tx=0x7f812c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c004b20 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f812c004cc0 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c010670 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.094+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81341a0080 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.095+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81341a05d0 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.096+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8134109350 con 0x7f8134103b70 2024-04-25T19:02:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.097+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f812c005020 con 0x7f8134103b70 2024-04-25T19:02:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.099+0000 7f812be00640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f81180a3ae0 0x7f81180a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.099+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f812c0d06a0 con 0x7f8134103b70 2024-04-25T19:02:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.099+0000 7f8133400640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f81180a3ae0 0x7f81180a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.100+0000 7f8133400640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f81180a3ae0 0x7f81180a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8124009230 tx=0x7f81240314a0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.103+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f812c09d200 con 0x7f8134103b70 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: Active manager daemon x restarted 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: Activating manager daemon x 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: mgrmap e21: x(active, starting, since 0.014856s), standbys: z, y 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T19:02:58.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T19:02:58.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T19:02:58.167 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: Manager daemon x is now available 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T19:02:58.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:57 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: Active manager daemon x restarted 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: Activating manager daemon x 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: mgrmap e21: x(active, starting, since 0.014856s), standbys: z, y 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T19:02:58.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T19:02:58.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T19:02:58.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: Manager daemon x is now available 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T19:02:58.172 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:57 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T19:02:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.326+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8134062390 con 0x7f81180a3ae0 2024-04-25T19:02:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.432+0000 7f812be00640 1 -- 172.21.15.86:0/863636752 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8134062390 con 0x7f81180a3ae0 2024-04-25T19:02:58.434 INFO:teuthology.orchestra.run.smithi086.stdout:Schedule set for path / 2024-04-25T19:02:58.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.434+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f81180a3ae0 msgr2=0x7f81180a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.434+0000 7f813a600640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f81180a3ae0 0x7f81180a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8124009230 tx=0x7f81240314a0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 msgr2=0x7f813419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 0x7f813419f720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f812c00eb10 tx=0x7f812c00c2d0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 shutdown_connections 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f81180a3ae0 0x7f81180a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134103b70 0x7f813419f720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81341031b0 0x7f8134198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 --2- 172.21.15.86:0/863636752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8134101fc0 0x7f81341981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 >> 172.21.15.86:0/863636752 conn(0x7f81340fd510 msgr2=0x7f8134106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 shutdown_connections 2024-04-25T19:02:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.435+0000 7f813a600640 1 -- 172.21.15.86:0/863636752 wait complete. 2024-04-25T19:02:58.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.649+0000 7faa0d200640 1 -- 172.21.15.86:0/962127936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08102a30 msgr2=0x7faa08104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.649+0000 7faa0d200640 1 --2- 172.21.15.86:0/962127936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08102a30 0x7faa08104e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009900 tx=0x7fa9f802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 -- 172.21.15.86:0/962127936 shutdown_connections 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 --2- 172.21.15.86:0/962127936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08107c90 0x7faa0810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 --2- 172.21.15.86:0/962127936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08105360 0x7faa08107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 --2- 172.21.15.86:0/962127936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08102a30 0x7faa08104e20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 -- 172.21.15.86:0/962127936 >> 172.21.15.86:0/962127936 conn(0x7faa080fc420 msgr2=0x7faa080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 -- 172.21.15.86:0/962127936 shutdown_connections 2024-04-25T19:02:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.650+0000 7faa0d200640 1 -- 172.21.15.86:0/962127936 wait complete. 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.651+0000 7faa0d200640 1 Processor -- start 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.651+0000 7faa0d200640 1 -- start start 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa0d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 0x7faa08198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa0d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08105360 0x7faa081986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa0d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08107c90 0x7faa0819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa0d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810db40 con 0x7faa08105360 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa0d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810d9c0 con 0x7faa08102a30 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa0d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810dcc0 con 0x7faa08107c90 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 0x7faa08198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 0x7faa08198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50748/0 (socket says 172.21.15.86:50748) 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 -- 172.21.15.86:0/88156420 learned_addr learned my addr 172.21.15.86:0/88156420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa06800640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08107c90 0x7faa0819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7fa9ffe00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08105360 0x7faa081986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 -- 172.21.15.86:0/88156420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08107c90 msgr2=0x7faa0819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08107c90 0x7faa0819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 -- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08105360 msgr2=0x7faa081986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08105360 0x7faa081986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7faa05e00640 1 -- 172.21.15.86:0/88156420 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0819fdb0 con 0x7faa08102a30 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.652+0000 7fa9ffe00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08105360 0x7faa081986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.653+0000 7faa05e00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 0x7faa08198190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005ab0 tx=0x7fa9f8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.653+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f803f070 con 0x7faa08102a30 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.653+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa081a0040 con 0x7faa08102a30 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.653+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8031c50 con 0x7faa08102a30 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.653+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8035030 con 0x7faa08102a30 2024-04-25T19:02:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.653+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa081a0530 con 0x7faa08102a30 2024-04-25T19:02:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.654+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa080fdf10 con 0x7faa08102a30 2024-04-25T19:02:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.656+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fa9f8042070 con 0x7faa08102a30 2024-04-25T19:02:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.657+0000 7fa9fea00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9ec0a3a60 0x7fa9ec0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.657+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fa9f80f0990 con 0x7faa08102a30 2024-04-25T19:02:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.657+0000 7fa9ffe00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9ec0a3a60 0x7fa9ec0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.658+0000 7fa9ffe00640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9ec0a3a60 0x7fa9ec0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa081996a0 tx=0x7fa9f0005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:58.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.661+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9f80bd4b0 con 0x7faa08102a30 2024-04-25T19:02:58.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.856+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fa9f803b460 con 0x7faa08102a30 2024-04-25T19:02:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.897+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7faa080624f0 con 0x7fa9ec0a3a60 2024-04-25T19:02:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.997+0000 7fa9fea00640 1 -- 172.21.15.86:0/88156420 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7faa080624f0 con 0x7fa9ec0a3a60 2024-04-25T19:02:58.999 INFO:teuthology.orchestra.run.smithi086.stdout:Retention added to path / 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9ec0a3a60 msgr2=0x7fa9ec0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9ec0a3a60 0x7fa9ec0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa081996a0 tx=0x7fa9f0005f90 comp rx=0 tx=0).stop 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 msgr2=0x7faa08198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 0x7faa08198190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005ab0 tx=0x7fa9f8003b90 comp rx=0 tx=0).stop 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 shutdown_connections 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9ec0a3a60 0x7fa9ec0a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa08107c90 0x7faa0819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa08105360 0x7faa081986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 --2- 172.21.15.86:0/88156420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa08102a30 0x7faa08198190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 >> 172.21.15.86:0/88156420 conn(0x7faa080fc420 msgr2=0x7faa08103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 shutdown_connections 2024-04-25T19:02:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:58.999+0000 7faa0d200640 1 -- 172.21.15.86:0/88156420 wait complete. 2024-04-25T19:02:59.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: [25/Apr/2024:19:02:57] ENGINE Bus STARTING 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: [25/Apr/2024:19:02:57] ENGINE Serving on http://172.21.15.86:8765 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: mgrmap e22: x(active, since 1.01997s), standbys: z, y 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: [25/Apr/2024:19:02:57] ENGINE Serving on https://172.21.15.86:7150 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: [25/Apr/2024:19:02:57] ENGINE Bus STARTED 2024-04-25T19:02:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:58 smithi183 ceph-mon[32344]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 -- 172.21.15.86:0/3150683194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc0784f0 msgr2=0x7fbabc078d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 --2- 172.21.15.86:0/3150683194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc0784f0 0x7fbabc078d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009be0 tx=0x7fbab402a6c0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 -- 172.21.15.86:0/3150683194 shutdown_connections 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 --2- 172.21.15.86:0/3150683194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc0784f0 0x7fbabc078d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 --2- 172.21.15.86:0/3150683194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbabc077b50 0x7fbabc077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 --2- 172.21.15.86:0/3150683194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc076910 0x7fbabc076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 -- 172.21.15.86:0/3150683194 >> 172.21.15.86:0/3150683194 conn(0x7fbabc0ff770 msgr2=0x7fbabc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 -- 172.21.15.86:0/3150683194 shutdown_connections 2024-04-25T19:02:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.194+0000 7fbac4200640 1 -- 172.21.15.86:0/3150683194 wait complete. 2024-04-25T19:02:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 Processor -- start 2024-04-25T19:02:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 -- start start 2024-04-25T19:02:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 0x7fbabc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc077b50 0x7fbabc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbabc0784f0 0x7fbabc1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc114120 con 0x7fbabc0784f0 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc113fa0 con 0x7fbabc077b50 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.195+0000 7fbac4200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc1142a0 con 0x7fbabc076910 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 0x7fbabc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc077b50 0x7fbabc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 0x7fbabc19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:37774/0 (socket says 172.21.15.86:37774) 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 -- 172.21.15.86:0/3490064314 learned_addr learned my addr 172.21.15.86:0/3490064314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac2200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbabc0784f0 0x7fbabc1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 -- 172.21.15.86:0/3490064314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc077b50 msgr2=0x7fbabc19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc077b50 0x7fbabc19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 -- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbabc0784f0 msgr2=0x7fbabc1a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbabc0784f0 0x7fbabc1a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 -- 172.21.15.86:0/3490064314 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc1a62d0 con 0x7fbabc076910 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.196+0000 7fbac1800640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 0x7fbabc19e700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e9c0 tx=0x7fbaa800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.197+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800ccc0 con 0x7fbabc076910 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.197+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa80054d0 con 0x7fbabc076910 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.197+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc1a65c0 con 0x7fbabc076910 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.197+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8010430 con 0x7fbabc076910 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.197+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc1a6b60 con 0x7fbabc076910 2024-04-25T19:02:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.198+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc110b50 con 0x7fbabc076910 2024-04-25T19:02:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.200+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fbaa8004f80 con 0x7fbabc076910 2024-04-25T19:02:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.202+0000 7fbaba000640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbaa40a3ee0 0x7fbaa40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.202+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fbaa80d0de0 con 0x7fbabc076910 2024-04-25T19:02:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.202+0000 7fbac0e00640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbaa40a3ee0 0x7fbaa40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.203+0000 7fbac0e00640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbaa40a3ee0 0x7fbaa40a63a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbab00095f0 tx=0x7fbab0005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.206+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbaa809d260 con 0x7fbabc076910 2024-04-25T19:02:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: [25/Apr/2024:19:02:57] ENGINE Bus STARTING 2024-04-25T19:02:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: [25/Apr/2024:19:02:57] ENGINE Serving on http://172.21.15.86:8765 2024-04-25T19:02:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: mgrmap e22: x(active, since 1.01997s), standbys: z, y 2024-04-25T19:02:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:59.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:02:59.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: [25/Apr/2024:19:02:57] ENGINE Serving on https://172.21.15.86:7150 2024-04-25T19:02:59.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: [25/Apr/2024:19:02:57] ENGINE Bus STARTED 2024-04-25T19:02:59.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:58 smithi194 ceph-mon[32740]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:59.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: [25/Apr/2024:19:02:57] ENGINE Bus STARTING 2024-04-25T19:02:59.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: [25/Apr/2024:19:02:57] ENGINE Serving on http://172.21.15.86:8765 2024-04-25T19:02:59.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: mgrmap e22: x(active, since 1.01997s), standbys: z, y 2024-04-25T19:02:59.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:59.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:02:59.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: [25/Apr/2024:19:02:57] ENGINE Serving on https://172.21.15.86:7150 2024-04-25T19:02:59.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: [25/Apr/2024:19:02:57] ENGINE Bus STARTED 2024-04-25T19:02:59.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:58 smithi086 ceph-mon[27783]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:02:59.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.438+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fbabc062390 con 0x7fbaa40a3ee0 2024-04-25T19:02:59.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.448+0000 7fbaba000640 1 -- 172.21.15.86:0/3490064314 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fbabc062390 con 0x7fbaa40a3ee0 2024-04-25T19:02:59.450 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T19:02:58", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T19:02:59.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbaa40a3ee0 msgr2=0x7fbaa40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbaa40a3ee0 0x7fbaa40a63a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbab00095f0 tx=0x7fbab0005f90 comp rx=0 tx=0).stop 2024-04-25T19:02:59.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 msgr2=0x7fbabc19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 0x7fbabc19e700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e9c0 tx=0x7fbaa800ee80 comp rx=0 tx=0).stop 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 shutdown_connections 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbaa40a3ee0 0x7fbaa40a63a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbabc0784f0 0x7fbabc1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc077b50 0x7fbabc19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 --2- 172.21.15.86:0/3490064314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc076910 0x7fbabc19e700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 >> 172.21.15.86:0/3490064314 conn(0x7fbabc0ff770 msgr2=0x7fbabc101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 shutdown_connections 2024-04-25T19:02:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.452+0000 7fbac4200640 1 -- 172.21.15.86:0/3490064314 wait complete. 2024-04-25T19:02:59.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T19:02:59.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.662+0000 7f0aa2800640 1 -- 172.21.15.86:0/1624371146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c0749b0 msgr2=0x7f0a9c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.662+0000 7f0aa2800640 1 --2- 172.21.15.86:0/1624371146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c074e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009900 tx=0x7f0a8802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 -- 172.21.15.86:0/1624371146 shutdown_connections 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 --2- 172.21.15.86:0/1624371146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a9c109e80 0x7f0a9c10c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 --2- 172.21.15.86:0/1624371146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c074e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 --2- 172.21.15.86:0/1624371146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c075f90 0x7f0a9c0743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 -- 172.21.15.86:0/1624371146 >> 172.21.15.86:0/1624371146 conn(0x7f0a9c0fd510 msgr2=0x7f0a9c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 -- 172.21.15.86:0/1624371146 shutdown_connections 2024-04-25T19:02:59.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.663+0000 7f0aa2800640 1 -- 172.21.15.86:0/1624371146 wait complete. 2024-04-25T19:02:59.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 Processor -- start 2024-04-25T19:02:59.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 -- start start 2024-04-25T19:02:59.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a9c075f90 0x7f0a9c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 0x7f0a9c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c10fd40 con 0x7f0a9c0749b0 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c10fbc0 con 0x7f0a9c109e80 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.664+0000 7f0aa2800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c10fec0 con 0x7f0a9c075f90 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 0x7f0a9c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39700/0 (socket says 172.21.15.86:39700) 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 0x7f0a9c1a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50776/0 (socket says 172.21.15.86:50776) 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9b400640 1 -- 172.21.15.86:0/820424746 learned_addr learned my addr 172.21.15.86:0/820424746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:02:59.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9aa00640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a9c075f90 0x7f0a9c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 -- 172.21.15.86:0/820424746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a9c075f90 msgr2=0x7f0a9c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a9c075f90 0x7f0a9c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 -- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c0749b0 msgr2=0x7f0a9c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 -- 172.21.15.86:0/820424746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c1a1f70 con 0x7f0a9c109e80 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.665+0000 7f0a9be00640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 0x7f0a9c1a1900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009fd0 tx=0x7f0a9400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.666+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400ce20 con 0x7f0a9c109e80 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.666+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a94005020 con 0x7f0a9c109e80 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.666+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94010690 con 0x7f0a9c109e80 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.666+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c1a2260 con 0x7f0a9c109e80 2024-04-25T19:02:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.666+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c1a2730 con 0x7f0a9c109e80 2024-04-25T19:02:59.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.667+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9c108bb0 con 0x7f0a9c109e80 2024-04-25T19:02:59.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.669+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f0a940054d0 con 0x7f0a9c109e80 2024-04-25T19:02:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.670+0000 7f0a93e00640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0a800a3ee0 0x7f0a800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.670+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f0a940d1150 con 0x7f0a9c109e80 2024-04-25T19:02:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.671+0000 7f0a9b400640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0a800a3ee0 0x7f0a800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.671+0000 7f0a9b400640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0a800a3ee0 0x7f0a800a63a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009c90 tx=0x7f0a8c005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:59.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.674+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a9409d5d0 con 0x7f0a9c109e80 2024-04-25T19:02:59.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.912+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0a9c062390 con 0x7f0a800a3ee0 2024-04-25T19:02:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.927+0000 7f0a93e00640 1 -- 172.21.15.86:0/820424746 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0a9c062390 con 0x7f0a800a3ee0 2024-04-25T19:02:59.929 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0a800a3ee0 msgr2=0x7f0a800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0a800a3ee0 0x7f0a800a63a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009c90 tx=0x7f0a8c005f90 comp rx=0 tx=0).stop 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 msgr2=0x7f0a9c1a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 0x7f0a9c1a1900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009fd0 tx=0x7f0a9400ef10 comp rx=0 tx=0).stop 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 shutdown_connections 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0a800a3ee0 0x7f0a800a63a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a9c109e80 0x7f0a9c1a1900 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a9c075f90 0x7f0a9c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 --2- 172.21.15.86:0/820424746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a9c0749b0 0x7f0a9c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.930+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 >> 172.21.15.86:0/820424746 conn(0x7f0a9c0fd510 msgr2=0x7f0a9c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.931+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 shutdown_connections 2024-04-25T19:02:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:02:59.931+0000 7f0aa2800640 1 -- 172.21.15.86:0/820424746 wait complete. 2024-04-25T19:02:59.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T19:03:00.022 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T19:03:00.033 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T19:03:00.043 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:03:00.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T19:03:00.047 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T19:03:00.060 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T19:03:00.060 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T19:03:00.097 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.098 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:02:59 smithi086 ceph-mon[27783]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:00.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:02:59 smithi183 ceph-mon[32344]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:00.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:00.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:02:59 smithi194 ceph-mon[32740]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi194 to 176.2M 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T19:03:01.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:00 smithi194 ceph-mon[32740]: Unable to set osd_memory_target on smithi194 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-25T19:03:01.320 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:03:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:03:01.014+0000 7feff3200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:03:01.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:03:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:03:01.014+0000 7feff3200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:03:01.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:03:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:03:01.014+0000 7feff3200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:03:01.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:03:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:03:01.014+0000 7feff3200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:03:01.321 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:03:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:03:01.014+0000 7feff3200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T19:03:01.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi194 to 176.2M 2024-04-25T19:03:01.324 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T19:03:01.324 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:00 smithi086 ceph-mon[27783]: Unable to set osd_memory_target on smithi194 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:01.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T19:03:01.419 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi194 to 176.2M 2024-04-25T19:03:01.419 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T19:03:01.419 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:00 smithi183 ceph-mon[32344]: Unable to set osd_memory_target on smithi194 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-25T19:03:02.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:02 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:02.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:02 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:03:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:02.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:02 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:03.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 -- 172.21.15.86:0/2081533250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c104eb0 msgr2=0x7f835c105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 --2- 172.21.15.86:0/2081533250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c104eb0 0x7f835c105290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f834c009990 tx=0x7f834c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 -- 172.21.15.86:0/2081533250 shutdown_connections 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 --2- 172.21.15.86:0/2081533250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c109f90 0x7f835c111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 --2- 172.21.15.86:0/2081533250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c104eb0 0x7f835c105290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 --2- 172.21.15.86:0/2081533250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c105860 0x7f835c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 -- 172.21.15.86:0/2081533250 >> 172.21.15.86:0/2081533250 conn(0x7f835c100740 msgr2=0x7f835c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 -- 172.21.15.86:0/2081533250 shutdown_connections 2024-04-25T19:03:03.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.005+0000 7f8361800700 1 -- 172.21.15.86:0/2081533250 wait complete. 2024-04-25T19:03:03.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.006+0000 7f8361800700 1 Processor -- start 2024-04-25T19:03:03.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.006+0000 7f8361800700 1 -- start start 2024-04-25T19:03:03.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.006+0000 7f8361800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 0x7f835c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.006+0000 7f8361800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c105860 0x7f835c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.006+0000 7f8361800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c109f90 0x7f835c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.006+0000 7f8361800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f835c114e70 con 0x7f835c109f90 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f8361800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f835c114cf0 con 0x7f835c105860 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f8361800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f835c114ff0 con 0x7f835c104eb0 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 0x7f835c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 0x7f835c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:42104/0 (socket says 172.21.15.86:42104) 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c105860 0x7f835c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835aa00700 1 -- 172.21.15.86:0/3204674214 learned_addr learned my addr 172.21.15.86:0/3204674214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:03.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835b400700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c109f90 0x7f835c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835a000700 1 -- 172.21.15.86:0/3204674214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 msgr2=0x7f835c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835a000700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 0x7f835c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835a000700 1 -- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c109f90 msgr2=0x7f835c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835a000700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c109f90 0x7f835c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835a000700 1 -- 172.21.15.86:0/3204674214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c1a0340 con 0x7f835c105860 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.007+0000 7f835aa00700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 0x7f835c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:03.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.008+0000 7f835a000700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c105860 0x7f835c19b8c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9c0 tx=0x7f834800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.008+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834800c9c0 con 0x7f835c105860 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.008+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f834800cb60 con 0x7f835c105860 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.008+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c1a0630 con 0x7f835c105860 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.008+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348018680 con 0x7f835c105860 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.008+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c1a7eb0 con 0x7f835c105860 2024-04-25T19:03:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.009+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c069730 con 0x7f835c105860 2024-04-25T19:03:03.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.011+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8348010430 con 0x7f835c105860 2024-04-25T19:03:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.012+0000 7f8353400700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f83400a3f50 0x7f83400a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.013+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f83480d0be0 con 0x7f835c105860 2024-04-25T19:03:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.013+0000 7f835aa00700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f83400a3f50 0x7f83400a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:03.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.014+0000 7f835aa00700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f83400a3f50 0x7f83400a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b790 tx=0x7f834c00b280 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.017+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f834809cfe0 con 0x7f835c105860 2024-04-25T19:03:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.238+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f8348098f50 con 0x7f835c105860 2024-04-25T19:03:03.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.256+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f835c062e80 con 0x7f83400a3f50 2024-04-25T19:03:03.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.265+0000 7f8353400700 1 -- 172.21.15.86:0/3204674214 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258091 (secure 0 0 0) 0x7f835c062e80 con 0x7f83400a3f50 2024-04-25T19:03:03.271 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.270+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f83400a3f50 msgr2=0x7f83400a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.270+0000 7f8361800700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f83400a3f50 0x7f83400a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b790 tx=0x7f834c00b280 comp rx=0 tx=0).stop 2024-04-25T19:03:03.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.270+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c105860 msgr2=0x7f835c19b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.270+0000 7f8361800700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c105860 0x7f835c19b8c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9c0 tx=0x7f834800ecd0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 shutdown_connections 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f835c109f90 0x7f835c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f83400a3f50 0x7f83400a6410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104eb0 0x7f835c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 --2- 172.21.15.86:0/3204674214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f835c105860 0x7f835c19b8c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 >> 172.21.15.86:0/3204674214 conn(0x7f835c100740 msgr2=0x7f835c102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 shutdown_connections 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:03.271+0000 7f8361800700 1 -- 172.21.15.86:0/3204674214 wait complete. 2024-04-25T19:03:03.274 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:03.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:03 smithi194 ceph-mon[32740]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:03.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:03 smithi086 ceph-mon[27783]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:03.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-25T19:03:03.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-25T19:03:03.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-25T19:03:03.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-25T19:03:03.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-25T19:03:03.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:03.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:03 smithi183 ceph-mon[32344]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:04.182 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-25T19:03:02.839531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192762,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2540,"num_read_kb":1437,"num_write":567,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362008,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579816,"statfs":{"total":1151940427776,"available":1151569731584,"internally_reserved":0,"allocated":13123584,"data_stored":9672083,"data_compressed":353111,"data_compressed_allocated":475136,"data_compressed_original":3801088,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981221"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.159028+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T19:02:57.159028+0000","last_peered":"2024-04-25T19:02:57.159028+0000","last_clean":"2024-04-25T19:02:57.159028+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T19:02:57.159028+0000","last_undegraded":"2024-04-25T19:02:57.159028+0000","last_fullsized":"2024-04-25T19:02:57.159028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888408+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T19:02:56.888408+0000","last_peered":"2024-04-25T19:02:56.888408+0000","last_clean":"2024-04-25T19:02:56.888408+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T19:02:56.888408+0000","last_undegraded":"2024-04-25T19:02:56.888408+0000","last_fullsized":"2024-04-25T19:02:56.888408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888162+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T19:02:56.888162+0000","last_peered":"2024-04-25T19:02:56.888162+0000","last_clean":"2024-04-25T19:02:56.888162+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T19:02:56.888162+0000","last_undegraded":"2024-04-25T19:02:56.888162+0000","last_fullsized":"2024-04-25T19:02:56.888162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827123+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T19:02:56.827123+0000","last_peered":"2024-04-25T19:02:56.827123+0000","last_clean":"2024-04-25T19:02:56.827123+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T19:02:56.827123+0000","last_undegraded":"2024-04-25T19:02:56.827123+0000","last_fullsized":"2024-04-25T19:02:56.827123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885598+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T19:02:56.885598+0000","last_peered":"2024-04-25T19:02:56.885598+0000","last_clean":"2024-04-25T19:02:56.885598+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T19:02:56.885598+0000","last_undegraded":"2024-04-25T19:02:56.885598+0000","last_fullsized":"2024-04-25T19:02:56.885598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838337+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T19:02:56.838337+0000","last_peered":"2024-04-25T19:02:56.838337+0000","last_clean":"2024-04-25T19:02:56.838337+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T19:02:56.838337+0000","last_undegraded":"2024-04-25T19:02:56.838337+0000","last_fullsized":"2024-04-25T19:02:56.838337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885734+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T19:02:56.885734+0000","last_peered":"2024-04-25T19:02:56.885734+0000","last_clean":"2024-04-25T19:02:56.885734+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T19:02:56.885734+0000","last_undegraded":"2024-04-25T19:02:56.885734+0000","last_fullsized":"2024-04-25T19:02:56.885734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885988+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T19:02:56.885988+0000","last_peered":"2024-04-25T19:02:56.885988+0000","last_clean":"2024-04-25T19:02:56.885988+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T19:02:56.885988+0000","last_undegraded":"2024-04-25T19:02:56.885988+0000","last_fullsized":"2024-04-25T19:02:56.885988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.823090+0000","last_change":"2024-04-25T19:00:38.543063+0000","last_active":"2024-04-25T19:02:56.823090+0000","last_peered":"2024-04-25T19:02:56.823090+0000","last_clean":"2024-04-25T19:02:56.823090+0000","last_became_active":"2024-04-25T19:00:38.541854+0000","last_became_peered":"2024-04-25T19:00:38.541854+0000","last_unstale":"2024-04-25T19:02:56.823090+0000","last_undegraded":"2024-04-25T19:02:56.823090+0000","last_fullsized":"2024-04-25T19:02:56.823090+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:10.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838538+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T19:02:56.838538+0000","last_peered":"2024-04-25T19:02:56.838538+0000","last_clean":"2024-04-25T19:02:56.838538+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T19:02:56.838538+0000","last_undegraded":"2024-04-25T19:02:56.838538+0000","last_fullsized":"2024-04-25T19:02:56.838538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886180+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T19:02:56.886180+0000","last_peered":"2024-04-25T19:02:56.886180+0000","last_clean":"2024-04-25T19:02:56.886180+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T19:02:56.886180+0000","last_undegraded":"2024-04-25T19:02:56.886180+0000","last_fullsized":"2024-04-25T19:02:56.886180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.826916+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T19:02:56.826916+0000","last_peered":"2024-04-25T19:02:56.826916+0000","last_clean":"2024-04-25T19:02:56.826916+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T19:02:56.826916+0000","last_undegraded":"2024-04-25T19:02:56.826916+0000","last_fullsized":"2024-04-25T19:02:56.826916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887124+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T19:02:56.887124+0000","last_peered":"2024-04-25T19:02:56.887124+0000","last_clean":"2024-04-25T19:02:56.887124+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T19:02:56.887124+0000","last_undegraded":"2024-04-25T19:02:56.887124+0000","last_fullsized":"2024-04-25T19:02:56.887124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887354+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T19:02:56.887354+0000","last_peered":"2024-04-25T19:02:56.887354+0000","last_clean":"2024-04-25T19:02:56.887354+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T19:02:56.887354+0000","last_undegraded":"2024-04-25T19:02:56.887354+0000","last_fullsized":"2024-04-25T19:02:56.887354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886787+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T19:02:56.886787+0000","last_peered":"2024-04-25T19:02:56.886787+0000","last_clean":"2024-04-25T19:02:56.886787+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T19:02:56.886787+0000","last_undegraded":"2024-04-25T19:02:56.886787+0000","last_fullsized":"2024-04-25T19:02:56.886787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885286+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T19:02:56.885286+0000","last_peered":"2024-04-25T19:02:56.885286+0000","last_clean":"2024-04-25T19:02:56.885286+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T19:02:56.885286+0000","last_undegraded":"2024-04-25T19:02:56.885286+0000","last_fullsized":"2024-04-25T19:02:56.885286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.159169+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T19:02:57.159169+0000","last_peered":"2024-04-25T19:02:57.159169+0000","last_clean":"2024-04-25T19:02:57.159169+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T19:02:57.159169+0000","last_undegraded":"2024-04-25T19:02:57.159169+0000","last_fullsized":"2024-04-25T19:02:57.159169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822913+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T19:02:56.822913+0000","last_peered":"2024-04-25T19:02:56.822913+0000","last_clean":"2024-04-25T19:02:56.822913+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T19:02:56.822913+0000","last_undegraded":"2024-04-25T19:02:56.822913+0000","last_fullsized":"2024-04-25T19:02:56.822913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824330+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T19:02:56.824330+0000","last_peered":"2024-04-25T19:02:56.824330+0000","last_clean":"2024-04-25T19:02:56.824330+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T19:02:56.824330+0000","last_undegraded":"2024-04-25T19:02:56.824330+0000","last_fullsized":"2024-04-25T19:02:56.824330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'15","reported_seq":276,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.156521+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T19:02:57.156521+0000","last_peered":"2024-04-25T19:02:57.156521+0000","last_clean":"2024-04-25T19:02:57.156521+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T19:02:57.156521+0000","last_undegraded":"2024-04-25T19:02:57.156521+0000","last_fullsized":"2024-04-25T19:02:57.156521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":227,"num_read_kb":227,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886573+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T19:02:56.886573+0000","last_peered":"2024-04-25T19:02:56.886573+0000","last_clean":"2024-04-25T19:02:56.886573+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T19:02:56.886573+0000","last_undegraded":"2024-04-25T19:02:56.886573+0000","last_fullsized":"2024-04-25T19:02:56.886573+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888176+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T19:02:56.888176+0000","last_peered":"2024-04-25T19:02:56.888176+0000","last_clean":"2024-04-25T19:02:56.888176+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T19:02:56.888176+0000","last_undegraded":"2024-04-25T19:02:56.888176+0000","last_fullsized":"2024-04-25T19:02:56.888176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888446+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T19:02:56.888446+0000","last_peered":"2024-04-25T19:02:56.888446+0000","last_clean":"2024-04-25T19:02:56.888446+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T19:02:56.888446+0000","last_undegraded":"2024-04-25T19:02:56.888446+0000","last_fullsized":"2024-04-25T19:02:56.888446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"94'13","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158963+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T19:02:57.158963+0000","last_peered":"2024-04-25T19:02:57.158963+0000","last_clean":"2024-04-25T19:02:57.158963+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T19:02:57.158963+0000","last_undegraded":"2024-04-25T19:02:57.158963+0000","last_fullsized":"2024-04-25T19:02:57.158963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885369+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T19:02:56.885369+0000","last_peered":"2024-04-25T19:02:56.885369+0000","last_clean":"2024-04-25T19:02:56.885369+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T19:02:56.885369+0000","last_undegraded":"2024-04-25T19:02:56.885369+0000","last_fullsized":"2024-04-25T19:02:56.885369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885839+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T19:02:56.885839+0000","last_peered":"2024-04-25T19:02:56.885839+0000","last_clean":"2024-04-25T19:02:56.885839+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T19:02:56.885839+0000","last_undegraded":"2024-04-25T19:02:56.885839+0000","last_fullsized":"2024-04-25T19:02:56.885839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841607+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T19:02:56.841607+0000","last_peered":"2024-04-25T19:02:56.841607+0000","last_clean":"2024-04-25T19:02:56.841607+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T19:02:56.841607+0000","last_undegraded":"2024-04-25T19:02:56.841607+0000","last_fullsized":"2024-04-25T19:02:56.841607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":271,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841798+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T19:02:56.841798+0000","last_peered":"2024-04-25T19:02:56.841798+0000","last_clean":"2024-04-25T19:02:56.841798+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T19:02:56.841798+0000","last_undegraded":"2024-04-25T19:02:56.841798+0000","last_fullsized":"2024-04-25T19:02:56.841798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885332+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T19:02:56.885332+0000","last_peered":"2024-04-25T19:02:56.885332+0000","last_clean":"2024-04-25T19:02:56.885332+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T19:02:56.885332+0000","last_undegraded":"2024-04-25T19:02:56.885332+0000","last_fullsized":"2024-04-25T19:02:56.885332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838196+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T19:02:56.838196+0000","last_peered":"2024-04-25T19:02:56.838196+0000","last_clean":"2024-04-25T19:02:56.838196+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T19:02:56.838196+0000","last_undegraded":"2024-04-25T19:02:56.838196+0000","last_fullsized":"2024-04-25T19:02:56.838196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841395+0000","last_change":"2024-04-25T19:00:38.695351+0000","last_active":"2024-04-25T19:02:56.841395+0000","last_peered":"2024-04-25T19:02:56.841395+0000","last_clean":"2024-04-25T19:02:56.841395+0000","last_became_active":"2024-04-25T19:00:38.694923+0000","last_became_peered":"2024-04-25T19:00:38.694923+0000","last_unstale":"2024-04-25T19:02:56.841395+0000","last_undegraded":"2024-04-25T19:02:56.841395+0000","last_fullsized":"2024-04-25T19:02:56.841395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:52.773252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886306+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T19:02:56.886306+0000","last_peered":"2024-04-25T19:02:56.886306+0000","last_clean":"2024-04-25T19:02:56.886306+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T19:02:56.886306+0000","last_undegraded":"2024-04-25T19:02:56.886306+0000","last_fullsized":"2024-04-25T19:02:56.886306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886498+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T19:02:56.886498+0000","last_peered":"2024-04-25T19:02:56.886498+0000","last_clean":"2024-04-25T19:02:56.886498+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T19:02:56.886498+0000","last_undegraded":"2024-04-25T19:02:56.886498+0000","last_fullsized":"2024-04-25T19:02:56.886498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841097+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T19:02:56.841097+0000","last_peered":"2024-04-25T19:02:56.841097+0000","last_clean":"2024-04-25T19:02:56.841097+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T19:02:56.841097+0000","last_undegraded":"2024-04-25T19:02:56.841097+0000","last_fullsized":"2024-04-25T19:02:56.841097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822598+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T19:02:56.822598+0000","last_peered":"2024-04-25T19:02:56.822598+0000","last_clean":"2024-04-25T19:02:56.822598+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T19:02:56.822598+0000","last_undegraded":"2024-04-25T19:02:56.822598+0000","last_fullsized":"2024-04-25T19:02:56.822598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":493,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.827012+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T19:02:57.827012+0000","last_peered":"2024-04-25T19:02:57.827012+0000","last_clean":"2024-04-25T19:02:57.827012+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T19:02:57.827012+0000","last_undegraded":"2024-04-25T19:02:57.827012+0000","last_fullsized":"2024-04-25T19:02:57.827012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":454,"num_read_kb":454,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885715+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T19:02:56.885715+0000","last_peered":"2024-04-25T19:02:56.885715+0000","last_clean":"2024-04-25T19:02:56.885715+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T19:02:56.885715+0000","last_undegraded":"2024-04-25T19:02:56.885715+0000","last_fullsized":"2024-04-25T19:02:56.885715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885708+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T19:02:56.885708+0000","last_peered":"2024-04-25T19:02:56.885708+0000","last_clean":"2024-04-25T19:02:56.885708+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T19:02:56.885708+0000","last_undegraded":"2024-04-25T19:02:56.885708+0000","last_fullsized":"2024-04-25T19:02:56.885708+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888378+0000","last_change":"2024-04-25T19:00:38.722645+0000","last_active":"2024-04-25T19:02:56.888378+0000","last_peered":"2024-04-25T19:02:56.888378+0000","last_clean":"2024-04-25T19:02:56.888378+0000","last_became_active":"2024-04-25T19:00:38.722223+0000","last_became_peered":"2024-04-25T19:00:38.722223+0000","last_unstale":"2024-04-25T19:02:56.888378+0000","last_undegraded":"2024-04-25T19:02:56.888378+0000","last_fullsized":"2024-04-25T19:02:56.888378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:51.266531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841719+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T19:02:56.841719+0000","last_peered":"2024-04-25T19:02:56.841719+0000","last_clean":"2024-04-25T19:02:56.841719+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T19:02:56.841719+0000","last_undegraded":"2024-04-25T19:02:56.841719+0000","last_fullsized":"2024-04-25T19:02:56.841719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885563+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T19:02:56.885563+0000","last_peered":"2024-04-25T19:02:56.885563+0000","last_clean":"2024-04-25T19:02:56.885563+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T19:02:56.885563+0000","last_undegraded":"2024-04-25T19:02:56.885563+0000","last_fullsized":"2024-04-25T19:02:56.885563+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'39","reported_seq":537,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.167627+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T19:02:57.167627+0000","last_peered":"2024-04-25T19:02:57.167627+0000","last_clean":"2024-04-25T19:02:57.167627+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T19:02:57.167627+0000","last_undegraded":"2024-04-25T19:02:57.167627+0000","last_fullsized":"2024-04-25T19:02:57.167627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67061,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":465,"num_read_kb":298,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885707+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T19:02:56.885707+0000","last_peered":"2024-04-25T19:02:56.885707+0000","last_clean":"2024-04-25T19:02:56.885707+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T19:02:56.885707+0000","last_undegraded":"2024-04-25T19:02:56.885707+0000","last_fullsized":"2024-04-25T19:02:56.885707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824643+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T19:02:56.824643+0000","last_peered":"2024-04-25T19:02:56.824643+0000","last_clean":"2024-04-25T19:02:56.824643+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T19:02:56.824643+0000","last_undegraded":"2024-04-25T19:02:56.824643+0000","last_fullsized":"2024-04-25T19:02:56.824643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886491+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T19:02:56.886491+0000","last_peered":"2024-04-25T19:02:56.886491+0000","last_clean":"2024-04-25T19:02:56.886491+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T19:02:56.886491+0000","last_undegraded":"2024-04-25T19:02:56.886491+0000","last_fullsized":"2024-04-25T19:02:56.886491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841183+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T19:02:56.841183+0000","last_peered":"2024-04-25T19:02:56.841183+0000","last_clean":"2024-04-25T19:02:56.841183+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T19:02:56.841183+0000","last_undegraded":"2024-04-25T19:02:56.841183+0000","last_fullsized":"2024-04-25T19:02:56.841183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886184+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T19:02:56.886184+0000","last_peered":"2024-04-25T19:02:56.886184+0000","last_clean":"2024-04-25T19:02:56.886184+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T19:02:56.886184+0000","last_undegraded":"2024-04-25T19:02:56.886184+0000","last_fullsized":"2024-04-25T19:02:56.886184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":273,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.042625+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T19:02:57.042625+0000","last_peered":"2024-04-25T19:02:57.042625+0000","last_clean":"2024-04-25T19:02:57.042625+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T19:02:57.042625+0000","last_undegraded":"2024-04-25T19:02:57.042625+0000","last_fullsized":"2024-04-25T19:02:57.042625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885362+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T19:02:56.885362+0000","last_peered":"2024-04-25T19:02:56.885362+0000","last_clean":"2024-04-25T19:02:56.885362+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T19:02:56.885362+0000","last_undegraded":"2024-04-25T19:02:56.885362+0000","last_fullsized":"2024-04-25T19:02:56.885362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885273+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T19:02:56.885273+0000","last_peered":"2024-04-25T19:02:56.885273+0000","last_clean":"2024-04-25T19:02:56.885273+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T19:02:56.885273+0000","last_undegraded":"2024-04-25T19:02:56.885273+0000","last_fullsized":"2024-04-25T19:02:56.885273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841117+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T19:02:56.841117+0000","last_peered":"2024-04-25T19:02:56.841117+0000","last_clean":"2024-04-25T19:02:56.841117+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T19:02:56.841117+0000","last_undegraded":"2024-04-25T19:02:56.841117+0000","last_fullsized":"2024-04-25T19:02:56.841117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885693+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T19:02:56.885693+0000","last_peered":"2024-04-25T19:02:56.885693+0000","last_clean":"2024-04-25T19:02:56.885693+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T19:02:56.885693+0000","last_undegraded":"2024-04-25T19:02:56.885693+0000","last_fullsized":"2024-04-25T19:02:56.885693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158836+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T19:02:57.158836+0000","last_peered":"2024-04-25T19:02:57.158836+0000","last_clean":"2024-04-25T19:02:57.158836+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T19:02:57.158836+0000","last_undegraded":"2024-04-25T19:02:57.158836+0000","last_fullsized":"2024-04-25T19:02:57.158836+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822980+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T19:02:56.822980+0000","last_peered":"2024-04-25T19:02:56.822980+0000","last_clean":"2024-04-25T19:02:56.822980+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T19:02:56.822980+0000","last_undegraded":"2024-04-25T19:02:56.822980+0000","last_fullsized":"2024-04-25T19:02:56.822980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838424+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T19:02:56.838424+0000","last_peered":"2024-04-25T19:02:56.838424+0000","last_clean":"2024-04-25T19:02:56.838424+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T19:02:56.838424+0000","last_undegraded":"2024-04-25T19:02:56.838424+0000","last_fullsized":"2024-04-25T19:02:56.838424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'9","reported_seq":272,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.167068+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T19:02:57.167068+0000","last_peered":"2024-04-25T19:02:57.167068+0000","last_clean":"2024-04-25T19:02:57.167068+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T19:02:57.167068+0000","last_undegraded":"2024-04-25T19:02:57.167068+0000","last_fullsized":"2024-04-25T19:02:57.167068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":229,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838424+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T19:02:56.838424+0000","last_peered":"2024-04-25T19:02:56.838424+0000","last_clean":"2024-04-25T19:02:56.838424+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T19:02:56.838424+0000","last_undegraded":"2024-04-25T19:02:56.838424+0000","last_fullsized":"2024-04-25T19:02:56.838424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822590+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T19:02:56.822590+0000","last_peered":"2024-04-25T19:02:56.822590+0000","last_clean":"2024-04-25T19:02:56.822590+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T19:02:56.822590+0000","last_undegraded":"2024-04-25T19:02:56.822590+0000","last_fullsized":"2024-04-25T19:02:56.822590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":268,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.041925+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T19:02:57.041925+0000","last_peered":"2024-04-25T19:02:57.041925+0000","last_clean":"2024-04-25T19:02:57.041925+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T19:02:57.041925+0000","last_undegraded":"2024-04-25T19:02:57.041925+0000","last_fullsized":"2024-04-25T19:02:57.041925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":227,"num_read_kb":227,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887165+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T19:02:56.887165+0000","last_peered":"2024-04-25T19:02:56.887165+0000","last_clean":"2024-04-25T19:02:56.887165+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T19:02:56.887165+0000","last_undegraded":"2024-04-25T19:02:56.887165+0000","last_fullsized":"2024-04-25T19:02:56.887165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827005+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T19:02:56.827005+0000","last_peered":"2024-04-25T19:02:56.827005+0000","last_clean":"2024-04-25T19:02:56.827005+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T19:02:56.827005+0000","last_undegraded":"2024-04-25T19:02:56.827005+0000","last_fullsized":"2024-04-25T19:02:56.827005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885680+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T19:02:56.885680+0000","last_peered":"2024-04-25T19:02:56.885680+0000","last_clean":"2024-04-25T19:02:56.885680+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T19:02:56.885680+0000","last_undegraded":"2024-04-25T19:02:56.885680+0000","last_fullsized":"2024-04-25T19:02:56.885680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886196+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T19:02:56.886196+0000","last_peered":"2024-04-25T19:02:56.886196+0000","last_clean":"2024-04-25T19:02:56.886196+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T19:02:56.886196+0000","last_undegraded":"2024-04-25T19:02:56.886196+0000","last_fullsized":"2024-04-25T19:02:56.886196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885873+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T19:02:56.885873+0000","last_peered":"2024-04-25T19:02:56.885873+0000","last_clean":"2024-04-25T19:02:56.885873+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T19:02:56.885873+0000","last_undegraded":"2024-04-25T19:02:56.885873+0000","last_fullsized":"2024-04-25T19:02:56.885873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158753+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T19:02:57.158753+0000","last_peered":"2024-04-25T19:02:57.158753+0000","last_clean":"2024-04-25T19:02:57.158753+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T19:02:57.158753+0000","last_undegraded":"2024-04-25T19:02:57.158753+0000","last_fullsized":"2024-04-25T19:02:57.158753+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887417+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T19:02:56.887417+0000","last_peered":"2024-04-25T19:02:56.887417+0000","last_clean":"2024-04-25T19:02:56.887417+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T19:02:56.887417+0000","last_undegraded":"2024-04-25T19:02:56.887417+0000","last_fullsized":"2024-04-25T19:02:56.887417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'52","reported_seq":108,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.093412+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T19:02:57.093412+0000","last_peered":"2024-04-25T19:02:57.093412+0000","last_clean":"2024-04-25T19:02:57.093412+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T19:02:57.093412+0000","last_undegraded":"2024-04-25T19:02:57.093412+0000","last_fullsized":"2024-04-25T19:02:57.093412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888097+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T19:02:56.888097+0000","last_peered":"2024-04-25T19:02:56.888097+0000","last_clean":"2024-04-25T19:02:56.888097+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T19:02:56.888097+0000","last_undegraded":"2024-04-25T19:02:56.888097+0000","last_fullsized":"2024-04-25T19:02:56.888097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827220+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T19:02:56.827220+0000","last_peered":"2024-04-25T19:02:56.827220+0000","last_clean":"2024-04-25T19:02:56.827220+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T19:02:56.827220+0000","last_undegraded":"2024-04-25T19:02:56.827220+0000","last_fullsized":"2024-04-25T19:02:56.827220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888502+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T19:02:56.888502+0000","last_peered":"2024-04-25T19:02:56.888502+0000","last_clean":"2024-04-25T19:02:56.888502+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T19:02:56.888502+0000","last_undegraded":"2024-04-25T19:02:56.888502+0000","last_fullsized":"2024-04-25T19:02:56.888502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885299+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T19:02:56.885299+0000","last_peered":"2024-04-25T19:02:56.885299+0000","last_clean":"2024-04-25T19:02:56.885299+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T19:02:56.885299+0000","last_undegraded":"2024-04-25T19:02:56.885299+0000","last_fullsized":"2024-04-25T19:02:56.885299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885296+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T19:02:56.885296+0000","last_peered":"2024-04-25T19:02:56.885296+0000","last_clean":"2024-04-25T19:02:56.885296+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T19:02:56.885296+0000","last_undegraded":"2024-04-25T19:02:56.885296+0000","last_fullsized":"2024-04-25T19:02:56.885296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886615+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T19:02:56.886615+0000","last_peered":"2024-04-25T19:02:56.886615+0000","last_clean":"2024-04-25T19:02:56.886615+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T19:02:56.886615+0000","last_undegraded":"2024-04-25T19:02:56.886615+0000","last_fullsized":"2024-04-25T19:02:56.886615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158513+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T19:02:57.158513+0000","last_peered":"2024-04-25T19:02:57.158513+0000","last_clean":"2024-04-25T19:02:57.158513+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T19:02:57.158513+0000","last_undegraded":"2024-04-25T19:02:57.158513+0000","last_fullsized":"2024-04-25T19:02:57.158513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824459+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T19:02:56.824459+0000","last_peered":"2024-04-25T19:02:56.824459+0000","last_clean":"2024-04-25T19:02:56.824459+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T19:02:56.824459+0000","last_undegraded":"2024-04-25T19:02:56.824459+0000","last_fullsized":"2024-04-25T19:02:56.824459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887352+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T19:02:56.887352+0000","last_peered":"2024-04-25T19:02:56.887352+0000","last_clean":"2024-04-25T19:02:56.887352+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T19:02:56.887352+0000","last_undegraded":"2024-04-25T19:02:56.887352+0000","last_fullsized":"2024-04-25T19:02:56.887352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885506+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T19:02:56.885506+0000","last_peered":"2024-04-25T19:02:56.885506+0000","last_clean":"2024-04-25T19:02:56.885506+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T19:02:56.885506+0000","last_undegraded":"2024-04-25T19:02:56.885506+0000","last_fullsized":"2024-04-25T19:02:56.885506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886172+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T19:02:56.886172+0000","last_peered":"2024-04-25T19:02:56.886172+0000","last_clean":"2024-04-25T19:02:56.886172+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T19:02:56.886172+0000","last_undegraded":"2024-04-25T19:02:56.886172+0000","last_fullsized":"2024-04-25T19:02:56.886172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'7","reported_seq":270,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.827417+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T19:02:57.827417+0000","last_peered":"2024-04-25T19:02:57.827417+0000","last_clean":"2024-04-25T19:02:57.827417+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T19:02:57.827417+0000","last_undegraded":"2024-04-25T19:02:57.827417+0000","last_fullsized":"2024-04-25T19:02:57.827417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886332+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T19:02:56.886332+0000","last_peered":"2024-04-25T19:02:56.886332+0000","last_clean":"2024-04-25T19:02:56.886332+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T19:02:56.886332+0000","last_undegraded":"2024-04-25T19:02:56.886332+0000","last_fullsized":"2024-04-25T19:02:56.886332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.090026+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T19:02:57.090026+0000","last_peered":"2024-04-25T19:02:57.090026+0000","last_clean":"2024-04-25T19:02:57.090026+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T19:02:57.090026+0000","last_undegraded":"2024-04-25T19:02:57.090026+0000","last_fullsized":"2024-04-25T19:02:57.090026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885921+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T19:02:56.885921+0000","last_peered":"2024-04-25T19:02:56.885921+0000","last_clean":"2024-04-25T19:02:56.885921+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T19:02:56.885921+0000","last_undegraded":"2024-04-25T19:02:56.885921+0000","last_fullsized":"2024-04-25T19:02:56.885921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887694+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T19:02:56.887694+0000","last_peered":"2024-04-25T19:02:56.887694+0000","last_clean":"2024-04-25T19:02:56.887694+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T19:02:56.887694+0000","last_undegraded":"2024-04-25T19:02:56.887694+0000","last_fullsized":"2024-04-25T19:02:56.887694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886022+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T19:02:56.886022+0000","last_peered":"2024-04-25T19:02:56.886022+0000","last_clean":"2024-04-25T19:02:56.886022+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T19:02:56.886022+0000","last_undegraded":"2024-04-25T19:02:56.886022+0000","last_fullsized":"2024-04-25T19:02:56.886022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158554+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T19:02:57.158554+0000","last_peered":"2024-04-25T19:02:57.158554+0000","last_clean":"2024-04-25T19:02:57.158554+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T19:02:57.158554+0000","last_undegraded":"2024-04-25T19:02:57.158554+0000","last_fullsized":"2024-04-25T19:02:57.158554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824344+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T19:02:56.824344+0000","last_peered":"2024-04-25T19:02:56.824344+0000","last_clean":"2024-04-25T19:02:56.824344+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T19:02:56.824344+0000","last_undegraded":"2024-04-25T19:02:56.824344+0000","last_fullsized":"2024-04-25T19:02:56.824344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824258+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T19:02:56.824258+0000","last_peered":"2024-04-25T19:02:56.824258+0000","last_clean":"2024-04-25T19:02:56.824258+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T19:02:56.824258+0000","last_undegraded":"2024-04-25T19:02:56.824258+0000","last_fullsized":"2024-04-25T19:02:56.824258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887702+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T19:02:56.887702+0000","last_peered":"2024-04-25T19:02:56.887702+0000","last_clean":"2024-04-25T19:02:56.887702+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T19:02:56.887702+0000","last_undegraded":"2024-04-25T19:02:56.887702+0000","last_fullsized":"2024-04-25T19:02:56.887702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887553+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T19:02:56.887553+0000","last_peered":"2024-04-25T19:02:56.887553+0000","last_clean":"2024-04-25T19:02:56.887553+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T19:02:56.887553+0000","last_undegraded":"2024-04-25T19:02:56.887553+0000","last_fullsized":"2024-04-25T19:02:56.887553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158936+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T19:02:57.158936+0000","last_peered":"2024-04-25T19:02:57.158936+0000","last_clean":"2024-04-25T19:02:57.158936+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T19:02:57.158936+0000","last_undegraded":"2024-04-25T19:02:57.158936+0000","last_fullsized":"2024-04-25T19:02:57.158936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824376+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T19:02:56.824376+0000","last_peered":"2024-04-25T19:02:56.824376+0000","last_clean":"2024-04-25T19:02:56.824376+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T19:02:56.824376+0000","last_undegraded":"2024-04-25T19:02:56.824376+0000","last_fullsized":"2024-04-25T19:02:56.824376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888047+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T19:02:56.888047+0000","last_peered":"2024-04-25T19:02:56.888047+0000","last_clean":"2024-04-25T19:02:56.888047+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T19:02:56.888047+0000","last_undegraded":"2024-04-25T19:02:56.888047+0000","last_fullsized":"2024-04-25T19:02:56.888047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841025+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T19:02:56.841025+0000","last_peered":"2024-04-25T19:02:56.841025+0000","last_clean":"2024-04-25T19:02:56.841025+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T19:02:56.841025+0000","last_undegraded":"2024-04-25T19:02:56.841025+0000","last_fullsized":"2024-04-25T19:02:56.841025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886443+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T19:02:56.886443+0000","last_peered":"2024-04-25T19:02:56.886443+0000","last_clean":"2024-04-25T19:02:56.886443+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T19:02:56.886443+0000","last_undegraded":"2024-04-25T19:02:56.886443+0000","last_fullsized":"2024-04-25T19:02:56.886443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885651+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T19:02:56.885651+0000","last_peered":"2024-04-25T19:02:56.885651+0000","last_clean":"2024-04-25T19:02:56.885651+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T19:02:56.885651+0000","last_undegraded":"2024-04-25T19:02:56.885651+0000","last_fullsized":"2024-04-25T19:02:56.885651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886966+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T19:02:56.886966+0000","last_peered":"2024-04-25T19:02:56.886966+0000","last_clean":"2024-04-25T19:02:56.886966+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T19:02:56.886966+0000","last_undegraded":"2024-04-25T19:02:56.886966+0000","last_fullsized":"2024-04-25T19:02:56.886966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158699+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T19:02:57.158699+0000","last_peered":"2024-04-25T19:02:57.158699+0000","last_clean":"2024-04-25T19:02:57.158699+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T19:02:57.158699+0000","last_undegraded":"2024-04-25T19:02:57.158699+0000","last_fullsized":"2024-04-25T19:02:57.158699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602160,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2404,"num_read_kb":1318,"num_write":384,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1806480,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":500954,"data_compressed":11542,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612680,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32368,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712784,"statfs":{"total":95995035648,"available":95961890816,"internally_reserved":0,"allocated":1216512,"data_stored":1300816,"data_compressed":56209,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":903039,"data_compressed":33584,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":903305,"data_compressed":33584,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1105407,"data_compressed":50053,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":510072,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":578828,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":511661,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1130496,"data_stored":579048,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149806,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1034181,"data_compressed":39740,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888169,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":510634,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724644,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1234138,"data_compressed":56209,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":201630,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:04.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T19:03:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: mgrmap e25: x(active, since 6s), standbys: z, y 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:04 smithi194 ceph-mon[32740]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: mgrmap e25: x(active, since 6s), standbys: z, y 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:04 smithi086 ceph-mon[27783]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: Updating smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: Updating smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: mgrmap e25: x(active, since 6s), standbys: z, y 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:04.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:04 smithi183 ceph-mon[32344]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-25T19:03:05.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:05.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:05 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:05.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:03:05.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:05.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:05.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:05 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: Updating smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:05.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:05 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:03:06.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:06 smithi086 ceph-mon[27783]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 170 KiB/s wr, 106 op/s 2024-04-25T19:03:06.568 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:06 smithi183 ceph-mon[32344]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 170 KiB/s wr, 106 op/s 2024-04-25T19:03:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:06 smithi194 ceph-mon[32740]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 170 KiB/s wr, 106 op/s 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 -- 172.21.15.86:0/1652975614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8290103b90 msgr2=0x7f8290103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 --2- 172.21.15.86:0/1652975614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8290103b90 0x7f8290103f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8280009990 tx=0x7f8280009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 -- 172.21.15.86:0/1652975614 shutdown_connections 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 --2- 172.21.15.86:0/1652975614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f829010cf70 0x7f829010f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 --2- 172.21.15.86:0/1652975614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8290103b90 0x7f8290103f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 --2- 172.21.15.86:0/1652975614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82901044b0 0x7f829010c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.849+0000 7f8295c00700 1 -- 172.21.15.86:0/1652975614 >> 172.21.15.86:0/1652975614 conn(0x7f82900fd360 msgr2=0x7f82900ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.850+0000 7f8295c00700 1 -- 172.21.15.86:0/1652975614 shutdown_connections 2024-04-25T19:03:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.850+0000 7f8295c00700 1 -- 172.21.15.86:0/1652975614 wait complete. 2024-04-25T19:03:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 Processor -- start 2024-04-25T19:03:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 -- start start 2024-04-25T19:03:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8290103b90 0x7f829019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82901044b0 0x7f829019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 0x7f829019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8290112b90 con 0x7f82901044b0 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8290112a10 con 0x7f829010cf70 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.851+0000 7f8295c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8290112d10 con 0x7f8290103b90 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 0x7f829019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f828fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8290103b90 0x7f829019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 0x7f829019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:47120/0 (socket says 172.21.15.86:47120) 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 -- 172.21.15.86:0/1232942720 learned_addr learned my addr 172.21.15.86:0/1232942720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f828f400700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82901044b0 0x7f829019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 -- 172.21.15.86:0/1232942720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8290103b90 msgr2=0x7f829019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8290103b90 0x7f829019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 -- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82901044b0 msgr2=0x7f829019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82901044b0 0x7f829019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.852+0000 7f8295200700 1 -- 172.21.15.86:0/1232942720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82901a02f0 con 0x7f829010cf70 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.853+0000 7f8295200700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 0x7f829019fbb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f828800ef30 tx=0x7f828800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.853+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880092e0 con 0x7f829010cf70 2024-04-25T19:03:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.853+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8288004bb0 con 0x7f829010cf70 2024-04-25T19:03:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.853+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82901a0580 con 0x7f829010cf70 2024-04-25T19:03:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.853+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288004020 con 0x7f829010cf70 2024-04-25T19:03:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.853+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82901a7e60 con 0x7f829010cf70 2024-04-25T19:03:06.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.854+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8290069730 con 0x7f829010cf70 2024-04-25T19:03:06.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.856+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f828801b030 con 0x7f829010cf70 2024-04-25T19:03:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.857+0000 7f828cc00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82740a3fe0 0x7f82740a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.858+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f82880d04c0 con 0x7f829010cf70 2024-04-25T19:03:06.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.858+0000 7f828fe00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82740a3fe0 0x7f82740a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.858+0000 7f828fe00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82740a3fe0 0x7f82740a64a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f828000b2e0 tx=0x7f8280005fd0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:06.862+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8288022020 con 0x7f829010cf70 2024-04-25T19:03:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.085+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f829019c640 con 0x7f82740a3fe0 2024-04-25T19:03:07.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.109+0000 7f828cc00700 1 -- 172.21.15.86:0/1232942720 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258127 (secure 0 0 0) 0x7f8280067060 con 0x7f82740a3fe0 2024-04-25T19:03:07.115 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82740a3fe0 msgr2=0x7f82740a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82740a3fe0 0x7f82740a64a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f828000b2e0 tx=0x7f8280005fd0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 msgr2=0x7f829019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 0x7f829019fbb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f828800ef30 tx=0x7f828800c3a0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 shutdown_connections 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f829010cf70 0x7f829019fbb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82740a3fe0 0x7f82740a64a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8290103b90 0x7f829019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 --2- 172.21.15.86:0/1232942720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82901044b0 0x7f829019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:07.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.115+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 >> 172.21.15.86:0/1232942720 conn(0x7f82900fd360 msgr2=0x7f82900fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:07.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.116+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 shutdown_connections 2024-04-25T19:03:07.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:07.116+0000 7f8295c00700 1 -- 172.21.15.86:0/1232942720 wait complete. 2024-04-25T19:03:07.118 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T19:03:07.990 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T19:03:06.840964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1395000,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3069,"num_read_kb":2022,"num_write":779,"num_write_kb":5601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1081,"ondisk_log_size":1081,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362096,"kb_used_data":12912,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579728,"statfs":{"total":1151940427776,"available":1151569641472,"internally_reserved":0,"allocated":13221888,"data_stored":10289412,"data_compressed":383598,"data_compressed_allocated":516096,"data_compressed_original":4362240,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":202238,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":529,"num_read_kb":585,"num_write":212,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.982655"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.159073+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T19:02:57.159073+0000","last_peered":"2024-04-25T19:02:57.159073+0000","last_clean":"2024-04-25T19:02:57.159073+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T19:02:57.159073+0000","last_undegraded":"2024-04-25T19:02:57.159073+0000","last_fullsized":"2024-04-25T19:02:57.159073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888408+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T19:02:56.888408+0000","last_peered":"2024-04-25T19:02:56.888408+0000","last_clean":"2024-04-25T19:02:56.888408+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T19:02:56.888408+0000","last_undegraded":"2024-04-25T19:02:56.888408+0000","last_fullsized":"2024-04-25T19:02:56.888408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888162+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T19:02:56.888162+0000","last_peered":"2024-04-25T19:02:56.888162+0000","last_clean":"2024-04-25T19:02:56.888162+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T19:02:56.888162+0000","last_undegraded":"2024-04-25T19:02:56.888162+0000","last_fullsized":"2024-04-25T19:02:56.888162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827123+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T19:02:56.827123+0000","last_peered":"2024-04-25T19:02:56.827123+0000","last_clean":"2024-04-25T19:02:56.827123+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T19:02:56.827123+0000","last_undegraded":"2024-04-25T19:02:56.827123+0000","last_fullsized":"2024-04-25T19:02:56.827123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885598+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T19:02:56.885598+0000","last_peered":"2024-04-25T19:02:56.885598+0000","last_clean":"2024-04-25T19:02:56.885598+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T19:02:56.885598+0000","last_undegraded":"2024-04-25T19:02:56.885598+0000","last_fullsized":"2024-04-25T19:02:56.885598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838337+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T19:02:56.838337+0000","last_peered":"2024-04-25T19:02:56.838337+0000","last_clean":"2024-04-25T19:02:56.838337+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T19:02:56.838337+0000","last_undegraded":"2024-04-25T19:02:56.838337+0000","last_fullsized":"2024-04-25T19:02:56.838337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885734+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T19:02:56.885734+0000","last_peered":"2024-04-25T19:02:56.885734+0000","last_clean":"2024-04-25T19:02:56.885734+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T19:02:56.885734+0000","last_undegraded":"2024-04-25T19:02:56.885734+0000","last_fullsized":"2024-04-25T19:02:56.885734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885988+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T19:02:56.885988+0000","last_peered":"2024-04-25T19:02:56.885988+0000","last_clean":"2024-04-25T19:02:56.885988+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T19:02:56.885988+0000","last_undegraded":"2024-04-25T19:02:56.885988+0000","last_fullsized":"2024-04-25T19:02:56.885988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.823143+0000","last_change":"2024-04-25T19:00:38.543063+0000","last_active":"2024-04-25T19:02:56.823143+0000","last_peered":"2024-04-25T19:02:56.823143+0000","last_clean":"2024-04-25T19:02:56.823143+0000","last_became_active":"2024-04-25T19:00:38.541854+0000","last_became_peered":"2024-04-25T19:00:38.541854+0000","last_unstale":"2024-04-25T19:02:56.823143+0000","last_undegraded":"2024-04-25T19:02:56.823143+0000","last_fullsized":"2024-04-25T19:02:56.823143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:10.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838538+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T19:02:56.838538+0000","last_peered":"2024-04-25T19:02:56.838538+0000","last_clean":"2024-04-25T19:02:56.838538+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T19:02:56.838538+0000","last_undegraded":"2024-04-25T19:02:56.838538+0000","last_fullsized":"2024-04-25T19:02:56.838538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886180+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T19:02:56.886180+0000","last_peered":"2024-04-25T19:02:56.886180+0000","last_clean":"2024-04-25T19:02:56.886180+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T19:02:56.886180+0000","last_undegraded":"2024-04-25T19:02:56.886180+0000","last_fullsized":"2024-04-25T19:02:56.886180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.826916+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T19:02:56.826916+0000","last_peered":"2024-04-25T19:02:56.826916+0000","last_clean":"2024-04-25T19:02:56.826916+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T19:02:56.826916+0000","last_undegraded":"2024-04-25T19:02:56.826916+0000","last_fullsized":"2024-04-25T19:02:56.826916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887124+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T19:02:56.887124+0000","last_peered":"2024-04-25T19:02:56.887124+0000","last_clean":"2024-04-25T19:02:56.887124+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T19:02:56.887124+0000","last_undegraded":"2024-04-25T19:02:56.887124+0000","last_fullsized":"2024-04-25T19:02:56.887124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887354+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T19:02:56.887354+0000","last_peered":"2024-04-25T19:02:56.887354+0000","last_clean":"2024-04-25T19:02:56.887354+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T19:02:56.887354+0000","last_undegraded":"2024-04-25T19:02:56.887354+0000","last_fullsized":"2024-04-25T19:02:56.887354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886787+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T19:02:56.886787+0000","last_peered":"2024-04-25T19:02:56.886787+0000","last_clean":"2024-04-25T19:02:56.886787+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T19:02:56.886787+0000","last_undegraded":"2024-04-25T19:02:56.886787+0000","last_fullsized":"2024-04-25T19:02:56.886787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885286+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T19:02:56.885286+0000","last_peered":"2024-04-25T19:02:56.885286+0000","last_clean":"2024-04-25T19:02:56.885286+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T19:02:56.885286+0000","last_undegraded":"2024-04-25T19:02:56.885286+0000","last_fullsized":"2024-04-25T19:02:56.885286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.159169+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T19:02:57.159169+0000","last_peered":"2024-04-25T19:02:57.159169+0000","last_clean":"2024-04-25T19:02:57.159169+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T19:02:57.159169+0000","last_undegraded":"2024-04-25T19:02:57.159169+0000","last_fullsized":"2024-04-25T19:02:57.159169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822913+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T19:02:56.822913+0000","last_peered":"2024-04-25T19:02:56.822913+0000","last_clean":"2024-04-25T19:02:56.822913+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T19:02:56.822913+0000","last_undegraded":"2024-04-25T19:02:56.822913+0000","last_fullsized":"2024-04-25T19:02:56.822913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824330+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T19:02:56.824330+0000","last_peered":"2024-04-25T19:02:56.824330+0000","last_clean":"2024-04-25T19:02:56.824330+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T19:02:56.824330+0000","last_undegraded":"2024-04-25T19:02:56.824330+0000","last_fullsized":"2024-04-25T19:02:56.824330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'15","reported_seq":281,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:00.987567+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T19:03:00.987567+0000","last_peered":"2024-04-25T19:03:00.987567+0000","last_clean":"2024-04-25T19:03:00.987567+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T19:03:00.987567+0000","last_undegraded":"2024-04-25T19:03:00.987567+0000","last_fullsized":"2024-04-25T19:03:00.987567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886573+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T19:02:56.886573+0000","last_peered":"2024-04-25T19:02:56.886573+0000","last_clean":"2024-04-25T19:02:56.886573+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T19:02:56.886573+0000","last_undegraded":"2024-04-25T19:02:56.886573+0000","last_fullsized":"2024-04-25T19:02:56.886573+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888176+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T19:02:56.888176+0000","last_peered":"2024-04-25T19:02:56.888176+0000","last_clean":"2024-04-25T19:02:56.888176+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T19:02:56.888176+0000","last_undegraded":"2024-04-25T19:02:56.888176+0000","last_fullsized":"2024-04-25T19:02:56.888176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888446+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T19:02:56.888446+0000","last_peered":"2024-04-25T19:02:56.888446+0000","last_clean":"2024-04-25T19:02:56.888446+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T19:02:56.888446+0000","last_undegraded":"2024-04-25T19:02:56.888446+0000","last_fullsized":"2024-04-25T19:02:56.888446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"94'13","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158963+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T19:02:57.158963+0000","last_peered":"2024-04-25T19:02:57.158963+0000","last_clean":"2024-04-25T19:02:57.158963+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T19:02:57.158963+0000","last_undegraded":"2024-04-25T19:02:57.158963+0000","last_fullsized":"2024-04-25T19:02:57.158963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885369+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T19:02:56.885369+0000","last_peered":"2024-04-25T19:02:56.885369+0000","last_clean":"2024-04-25T19:02:56.885369+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T19:02:56.885369+0000","last_undegraded":"2024-04-25T19:02:56.885369+0000","last_fullsized":"2024-04-25T19:02:56.885369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885839+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T19:02:56.885839+0000","last_peered":"2024-04-25T19:02:56.885839+0000","last_clean":"2024-04-25T19:02:56.885839+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T19:02:56.885839+0000","last_undegraded":"2024-04-25T19:02:56.885839+0000","last_fullsized":"2024-04-25T19:02:56.885839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841607+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T19:02:56.841607+0000","last_peered":"2024-04-25T19:02:56.841607+0000","last_clean":"2024-04-25T19:02:56.841607+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T19:02:56.841607+0000","last_undegraded":"2024-04-25T19:02:56.841607+0000","last_fullsized":"2024-04-25T19:02:56.841607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":275,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:00.850341+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T19:03:00.850341+0000","last_peered":"2024-04-25T19:03:00.850341+0000","last_clean":"2024-04-25T19:03:00.850341+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T19:03:00.850341+0000","last_undegraded":"2024-04-25T19:03:00.850341+0000","last_fullsized":"2024-04-25T19:03:00.850341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885332+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T19:02:56.885332+0000","last_peered":"2024-04-25T19:02:56.885332+0000","last_clean":"2024-04-25T19:02:56.885332+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T19:02:56.885332+0000","last_undegraded":"2024-04-25T19:02:56.885332+0000","last_fullsized":"2024-04-25T19:02:56.885332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838196+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T19:02:56.838196+0000","last_peered":"2024-04-25T19:02:56.838196+0000","last_clean":"2024-04-25T19:02:56.838196+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T19:02:56.838196+0000","last_undegraded":"2024-04-25T19:02:56.838196+0000","last_fullsized":"2024-04-25T19:02:56.838196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841395+0000","last_change":"2024-04-25T19:00:38.695351+0000","last_active":"2024-04-25T19:02:56.841395+0000","last_peered":"2024-04-25T19:02:56.841395+0000","last_clean":"2024-04-25T19:02:56.841395+0000","last_became_active":"2024-04-25T19:00:38.694923+0000","last_became_peered":"2024-04-25T19:00:38.694923+0000","last_unstale":"2024-04-25T19:02:56.841395+0000","last_undegraded":"2024-04-25T19:02:56.841395+0000","last_fullsized":"2024-04-25T19:02:56.841395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:52.773252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886306+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T19:02:56.886306+0000","last_peered":"2024-04-25T19:02:56.886306+0000","last_clean":"2024-04-25T19:02:56.886306+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T19:02:56.886306+0000","last_undegraded":"2024-04-25T19:02:56.886306+0000","last_fullsized":"2024-04-25T19:02:56.886306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886498+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T19:02:56.886498+0000","last_peered":"2024-04-25T19:02:56.886498+0000","last_clean":"2024-04-25T19:02:56.886498+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T19:02:56.886498+0000","last_undegraded":"2024-04-25T19:02:56.886498+0000","last_fullsized":"2024-04-25T19:02:56.886498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841097+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T19:02:56.841097+0000","last_peered":"2024-04-25T19:02:56.841097+0000","last_clean":"2024-04-25T19:02:56.841097+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T19:02:56.841097+0000","last_undegraded":"2024-04-25T19:02:56.841097+0000","last_fullsized":"2024-04-25T19:02:56.841097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822598+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T19:02:56.822598+0000","last_peered":"2024-04-25T19:02:56.822598+0000","last_clean":"2024-04-25T19:02:56.822598+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T19:02:56.822598+0000","last_undegraded":"2024-04-25T19:02:56.822598+0000","last_fullsized":"2024-04-25T19:02:56.822598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":502,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.851228+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T19:03:01.851228+0000","last_peered":"2024-04-25T19:03:01.851228+0000","last_clean":"2024-04-25T19:03:01.851228+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T19:03:01.851228+0000","last_undegraded":"2024-04-25T19:03:01.851228+0000","last_fullsized":"2024-04-25T19:03:01.851228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":463,"num_read_kb":463,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885715+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T19:02:56.885715+0000","last_peered":"2024-04-25T19:02:56.885715+0000","last_clean":"2024-04-25T19:02:56.885715+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T19:02:56.885715+0000","last_undegraded":"2024-04-25T19:02:56.885715+0000","last_fullsized":"2024-04-25T19:02:56.885715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885708+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T19:02:56.885708+0000","last_peered":"2024-04-25T19:02:56.885708+0000","last_clean":"2024-04-25T19:02:56.885708+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T19:02:56.885708+0000","last_undegraded":"2024-04-25T19:02:56.885708+0000","last_fullsized":"2024-04-25T19:02:56.885708+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888378+0000","last_change":"2024-04-25T19:00:38.722645+0000","last_active":"2024-04-25T19:02:56.888378+0000","last_peered":"2024-04-25T19:02:56.888378+0000","last_clean":"2024-04-25T19:02:56.888378+0000","last_became_active":"2024-04-25T19:00:38.722223+0000","last_became_peered":"2024-04-25T19:00:38.722223+0000","last_unstale":"2024-04-25T19:02:56.888378+0000","last_undegraded":"2024-04-25T19:02:56.888378+0000","last_fullsized":"2024-04-25T19:02:56.888378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:51.266531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841719+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T19:02:56.841719+0000","last_peered":"2024-04-25T19:02:56.841719+0000","last_clean":"2024-04-25T19:02:56.841719+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T19:02:56.841719+0000","last_undegraded":"2024-04-25T19:02:56.841719+0000","last_fullsized":"2024-04-25T19:02:56.841719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885563+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T19:02:56.885563+0000","last_peered":"2024-04-25T19:02:56.885563+0000","last_clean":"2024-04-25T19:02:56.885563+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T19:02:56.885563+0000","last_undegraded":"2024-04-25T19:02:56.885563+0000","last_fullsized":"2024-04-25T19:02:56.885563+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'44","reported_seq":554,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.851858+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T19:03:01.851858+0000","last_peered":"2024-04-25T19:03:01.851858+0000","last_clean":"2024-04-25T19:03:01.851858+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T19:03:01.851858+0000","last_undegraded":"2024-04-25T19:03:01.851858+0000","last_fullsized":"2024-04-25T19:03:01.851858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":477,"num_read_kb":308,"num_write":46,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885707+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T19:02:56.885707+0000","last_peered":"2024-04-25T19:02:56.885707+0000","last_clean":"2024-04-25T19:02:56.885707+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T19:02:56.885707+0000","last_undegraded":"2024-04-25T19:02:56.885707+0000","last_fullsized":"2024-04-25T19:02:56.885707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824643+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T19:02:56.824643+0000","last_peered":"2024-04-25T19:02:56.824643+0000","last_clean":"2024-04-25T19:02:56.824643+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T19:02:56.824643+0000","last_undegraded":"2024-04-25T19:02:56.824643+0000","last_fullsized":"2024-04-25T19:02:56.824643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886491+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T19:02:56.886491+0000","last_peered":"2024-04-25T19:02:56.886491+0000","last_clean":"2024-04-25T19:02:56.886491+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T19:02:56.886491+0000","last_undegraded":"2024-04-25T19:02:56.886491+0000","last_fullsized":"2024-04-25T19:02:56.886491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841183+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T19:02:56.841183+0000","last_peered":"2024-04-25T19:02:56.841183+0000","last_clean":"2024-04-25T19:02:56.841183+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T19:02:56.841183+0000","last_undegraded":"2024-04-25T19:02:56.841183+0000","last_fullsized":"2024-04-25T19:02:56.841183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886184+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T19:02:56.886184+0000","last_peered":"2024-04-25T19:02:56.886184+0000","last_clean":"2024-04-25T19:02:56.886184+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T19:02:56.886184+0000","last_undegraded":"2024-04-25T19:02:56.886184+0000","last_fullsized":"2024-04-25T19:02:56.886184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":279,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.048801+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T19:03:01.048801+0000","last_peered":"2024-04-25T19:03:01.048801+0000","last_clean":"2024-04-25T19:03:01.048801+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T19:03:01.048801+0000","last_undegraded":"2024-04-25T19:03:01.048801+0000","last_fullsized":"2024-04-25T19:03:01.048801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885362+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T19:02:56.885362+0000","last_peered":"2024-04-25T19:02:56.885362+0000","last_clean":"2024-04-25T19:02:56.885362+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T19:02:56.885362+0000","last_undegraded":"2024-04-25T19:02:56.885362+0000","last_fullsized":"2024-04-25T19:02:56.885362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885273+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T19:02:56.885273+0000","last_peered":"2024-04-25T19:02:56.885273+0000","last_clean":"2024-04-25T19:02:56.885273+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T19:02:56.885273+0000","last_undegraded":"2024-04-25T19:02:56.885273+0000","last_fullsized":"2024-04-25T19:02:56.885273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841117+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T19:02:56.841117+0000","last_peered":"2024-04-25T19:02:56.841117+0000","last_clean":"2024-04-25T19:02:56.841117+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T19:02:56.841117+0000","last_undegraded":"2024-04-25T19:02:56.841117+0000","last_fullsized":"2024-04-25T19:02:56.841117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885693+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T19:02:56.885693+0000","last_peered":"2024-04-25T19:02:56.885693+0000","last_clean":"2024-04-25T19:02:56.885693+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T19:02:56.885693+0000","last_undegraded":"2024-04-25T19:02:56.885693+0000","last_fullsized":"2024-04-25T19:02:56.885693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158836+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T19:02:57.158836+0000","last_peered":"2024-04-25T19:02:57.158836+0000","last_clean":"2024-04-25T19:02:57.158836+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T19:02:57.158836+0000","last_undegraded":"2024-04-25T19:02:57.158836+0000","last_fullsized":"2024-04-25T19:02:57.158836+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822980+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T19:02:56.822980+0000","last_peered":"2024-04-25T19:02:56.822980+0000","last_clean":"2024-04-25T19:02:56.822980+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T19:02:56.822980+0000","last_undegraded":"2024-04-25T19:02:56.822980+0000","last_fullsized":"2024-04-25T19:02:56.822980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838608+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T19:02:56.838608+0000","last_peered":"2024-04-25T19:02:56.838608+0000","last_clean":"2024-04-25T19:02:56.838608+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T19:02:56.838608+0000","last_undegraded":"2024-04-25T19:02:56.838608+0000","last_fullsized":"2024-04-25T19:02:56.838608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'11","reported_seq":281,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.990490+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T19:03:01.990490+0000","last_peered":"2024-04-25T19:03:01.990490+0000","last_clean":"2024-04-25T19:03:01.990490+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T19:03:01.990490+0000","last_undegraded":"2024-04-25T19:03:01.990490+0000","last_fullsized":"2024-04-25T19:03:01.990490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2684,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838659+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T19:02:56.838659+0000","last_peered":"2024-04-25T19:02:56.838659+0000","last_clean":"2024-04-25T19:02:56.838659+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T19:02:56.838659+0000","last_undegraded":"2024-04-25T19:02:56.838659+0000","last_fullsized":"2024-04-25T19:02:56.838659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822590+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T19:02:56.822590+0000","last_peered":"2024-04-25T19:02:56.822590+0000","last_clean":"2024-04-25T19:02:56.822590+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T19:02:56.822590+0000","last_undegraded":"2024-04-25T19:02:56.822590+0000","last_fullsized":"2024-04-25T19:02:56.822590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":273,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.048257+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T19:03:01.048257+0000","last_peered":"2024-04-25T19:03:01.048257+0000","last_clean":"2024-04-25T19:03:01.048257+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T19:03:01.048257+0000","last_undegraded":"2024-04-25T19:03:01.048257+0000","last_fullsized":"2024-04-25T19:03:01.048257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887165+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T19:02:56.887165+0000","last_peered":"2024-04-25T19:02:56.887165+0000","last_clean":"2024-04-25T19:02:56.887165+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T19:02:56.887165+0000","last_undegraded":"2024-04-25T19:02:56.887165+0000","last_fullsized":"2024-04-25T19:02:56.887165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827005+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T19:02:56.827005+0000","last_peered":"2024-04-25T19:02:56.827005+0000","last_clean":"2024-04-25T19:02:56.827005+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T19:02:56.827005+0000","last_undegraded":"2024-04-25T19:02:56.827005+0000","last_fullsized":"2024-04-25T19:02:56.827005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":411,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:58.970251+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T19:02:58.970251+0000","last_peered":"2024-04-25T19:02:58.970251+0000","last_clean":"2024-04-25T19:02:58.970251+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T19:02:58.970251+0000","last_undegraded":"2024-04-25T19:02:58.970251+0000","last_fullsized":"2024-04-25T19:02:58.970251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886196+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T19:02:56.886196+0000","last_peered":"2024-04-25T19:02:56.886196+0000","last_clean":"2024-04-25T19:02:56.886196+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T19:02:56.886196+0000","last_undegraded":"2024-04-25T19:02:56.886196+0000","last_fullsized":"2024-04-25T19:02:56.886196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885873+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T19:02:56.885873+0000","last_peered":"2024-04-25T19:02:56.885873+0000","last_clean":"2024-04-25T19:02:56.885873+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T19:02:56.885873+0000","last_undegraded":"2024-04-25T19:02:56.885873+0000","last_fullsized":"2024-04-25T19:02:56.885873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158753+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T19:02:57.158753+0000","last_peered":"2024-04-25T19:02:57.158753+0000","last_clean":"2024-04-25T19:02:57.158753+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T19:02:57.158753+0000","last_undegraded":"2024-04-25T19:02:57.158753+0000","last_fullsized":"2024-04-25T19:02:57.158753+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887459+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T19:02:56.887459+0000","last_peered":"2024-04-25T19:02:56.887459+0000","last_clean":"2024-04-25T19:02:56.887459+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T19:02:56.887459+0000","last_undegraded":"2024-04-25T19:02:56.887459+0000","last_fullsized":"2024-04-25T19:02:56.887459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'130","reported_seq":288,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.099498+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T19:03:01.099498+0000","last_peered":"2024-04-25T19:03:01.099498+0000","last_clean":"2024-04-25T19:03:01.099498+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T19:03:01.099498+0000","last_undegraded":"2024-04-25T19:03:01.099498+0000","last_fullsized":"2024-04-25T19:03:01.099498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":239,"num_write_kb":1415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888097+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T19:02:56.888097+0000","last_peered":"2024-04-25T19:02:56.888097+0000","last_clean":"2024-04-25T19:02:56.888097+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T19:02:56.888097+0000","last_undegraded":"2024-04-25T19:02:56.888097+0000","last_fullsized":"2024-04-25T19:02:56.888097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827274+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T19:02:56.827274+0000","last_peered":"2024-04-25T19:02:56.827274+0000","last_clean":"2024-04-25T19:02:56.827274+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T19:02:56.827274+0000","last_undegraded":"2024-04-25T19:02:56.827274+0000","last_fullsized":"2024-04-25T19:02:56.827274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888502+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T19:02:56.888502+0000","last_peered":"2024-04-25T19:02:56.888502+0000","last_clean":"2024-04-25T19:02:56.888502+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T19:02:56.888502+0000","last_undegraded":"2024-04-25T19:02:56.888502+0000","last_fullsized":"2024-04-25T19:02:56.888502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885299+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T19:02:56.885299+0000","last_peered":"2024-04-25T19:02:56.885299+0000","last_clean":"2024-04-25T19:02:56.885299+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T19:02:56.885299+0000","last_undegraded":"2024-04-25T19:02:56.885299+0000","last_fullsized":"2024-04-25T19:02:56.885299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885296+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T19:02:56.885296+0000","last_peered":"2024-04-25T19:02:56.885296+0000","last_clean":"2024-04-25T19:02:56.885296+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T19:02:56.885296+0000","last_undegraded":"2024-04-25T19:02:56.885296+0000","last_fullsized":"2024-04-25T19:02:56.885296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886615+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T19:02:56.886615+0000","last_peered":"2024-04-25T19:02:56.886615+0000","last_clean":"2024-04-25T19:02:56.886615+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T19:02:56.886615+0000","last_undegraded":"2024-04-25T19:02:56.886615+0000","last_fullsized":"2024-04-25T19:02:56.886615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158513+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T19:02:57.158513+0000","last_peered":"2024-04-25T19:02:57.158513+0000","last_clean":"2024-04-25T19:02:57.158513+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T19:02:57.158513+0000","last_undegraded":"2024-04-25T19:02:57.158513+0000","last_fullsized":"2024-04-25T19:02:57.158513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824536+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T19:02:56.824536+0000","last_peered":"2024-04-25T19:02:56.824536+0000","last_clean":"2024-04-25T19:02:56.824536+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T19:02:56.824536+0000","last_undegraded":"2024-04-25T19:02:56.824536+0000","last_fullsized":"2024-04-25T19:02:56.824536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887352+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T19:02:56.887352+0000","last_peered":"2024-04-25T19:02:56.887352+0000","last_clean":"2024-04-25T19:02:56.887352+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T19:02:56.887352+0000","last_undegraded":"2024-04-25T19:02:56.887352+0000","last_fullsized":"2024-04-25T19:02:56.887352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885506+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T19:02:56.885506+0000","last_peered":"2024-04-25T19:02:56.885506+0000","last_clean":"2024-04-25T19:02:56.885506+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T19:02:56.885506+0000","last_undegraded":"2024-04-25T19:02:56.885506+0000","last_fullsized":"2024-04-25T19:02:56.885506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886172+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T19:02:56.886172+0000","last_peered":"2024-04-25T19:02:56.886172+0000","last_clean":"2024-04-25T19:02:56.886172+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T19:02:56.886172+0000","last_undegraded":"2024-04-25T19:02:56.886172+0000","last_fullsized":"2024-04-25T19:02:56.886172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":277,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.833803+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T19:03:01.833803+0000","last_peered":"2024-04-25T19:03:01.833803+0000","last_clean":"2024-04-25T19:03:01.833803+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T19:03:01.833803+0000","last_undegraded":"2024-04-25T19:03:01.833803+0000","last_fullsized":"2024-04-25T19:03:01.833803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886332+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T19:02:56.886332+0000","last_peered":"2024-04-25T19:02:56.886332+0000","last_clean":"2024-04-25T19:02:56.886332+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T19:02:56.886332+0000","last_undegraded":"2024-04-25T19:02:56.886332+0000","last_fullsized":"2024-04-25T19:02:56.886332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'75","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.092751+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T19:03:01.092751+0000","last_peered":"2024-04-25T19:03:01.092751+0000","last_clean":"2024-04-25T19:03:01.092751+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T19:03:01.092751+0000","last_undegraded":"2024-04-25T19:03:01.092751+0000","last_fullsized":"2024-04-25T19:03:01.092751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885921+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T19:02:56.885921+0000","last_peered":"2024-04-25T19:02:56.885921+0000","last_clean":"2024-04-25T19:02:56.885921+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T19:02:56.885921+0000","last_undegraded":"2024-04-25T19:02:56.885921+0000","last_fullsized":"2024-04-25T19:02:56.885921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887694+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T19:02:56.887694+0000","last_peered":"2024-04-25T19:02:56.887694+0000","last_clean":"2024-04-25T19:02:56.887694+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T19:02:56.887694+0000","last_undegraded":"2024-04-25T19:02:56.887694+0000","last_fullsized":"2024-04-25T19:02:56.887694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886022+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T19:02:56.886022+0000","last_peered":"2024-04-25T19:02:56.886022+0000","last_clean":"2024-04-25T19:02:56.886022+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T19:02:56.886022+0000","last_undegraded":"2024-04-25T19:02:56.886022+0000","last_fullsized":"2024-04-25T19:02:56.886022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158586+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T19:02:57.158586+0000","last_peered":"2024-04-25T19:02:57.158586+0000","last_clean":"2024-04-25T19:02:57.158586+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T19:02:57.158586+0000","last_undegraded":"2024-04-25T19:02:57.158586+0000","last_fullsized":"2024-04-25T19:02:57.158586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824344+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T19:02:56.824344+0000","last_peered":"2024-04-25T19:02:56.824344+0000","last_clean":"2024-04-25T19:02:56.824344+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T19:02:56.824344+0000","last_undegraded":"2024-04-25T19:02:56.824344+0000","last_fullsized":"2024-04-25T19:02:56.824344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824258+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T19:02:56.824258+0000","last_peered":"2024-04-25T19:02:56.824258+0000","last_clean":"2024-04-25T19:02:56.824258+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T19:02:56.824258+0000","last_undegraded":"2024-04-25T19:02:56.824258+0000","last_fullsized":"2024-04-25T19:02:56.824258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887746+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T19:02:56.887746+0000","last_peered":"2024-04-25T19:02:56.887746+0000","last_clean":"2024-04-25T19:02:56.887746+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T19:02:56.887746+0000","last_undegraded":"2024-04-25T19:02:56.887746+0000","last_fullsized":"2024-04-25T19:02:56.887746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887553+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T19:02:56.887553+0000","last_peered":"2024-04-25T19:02:56.887553+0000","last_clean":"2024-04-25T19:02:56.887553+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T19:02:56.887553+0000","last_undegraded":"2024-04-25T19:02:56.887553+0000","last_fullsized":"2024-04-25T19:02:56.887553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158936+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T19:02:57.158936+0000","last_peered":"2024-04-25T19:02:57.158936+0000","last_clean":"2024-04-25T19:02:57.158936+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T19:02:57.158936+0000","last_undegraded":"2024-04-25T19:02:57.158936+0000","last_fullsized":"2024-04-25T19:02:57.158936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824376+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T19:02:56.824376+0000","last_peered":"2024-04-25T19:02:56.824376+0000","last_clean":"2024-04-25T19:02:56.824376+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T19:02:56.824376+0000","last_undegraded":"2024-04-25T19:02:56.824376+0000","last_fullsized":"2024-04-25T19:02:56.824376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888047+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T19:02:56.888047+0000","last_peered":"2024-04-25T19:02:56.888047+0000","last_clean":"2024-04-25T19:02:56.888047+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T19:02:56.888047+0000","last_undegraded":"2024-04-25T19:02:56.888047+0000","last_fullsized":"2024-04-25T19:02:56.888047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841025+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T19:02:56.841025+0000","last_peered":"2024-04-25T19:02:56.841025+0000","last_clean":"2024-04-25T19:02:56.841025+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T19:02:56.841025+0000","last_undegraded":"2024-04-25T19:02:56.841025+0000","last_fullsized":"2024-04-25T19:02:56.841025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886443+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T19:02:56.886443+0000","last_peered":"2024-04-25T19:02:56.886443+0000","last_clean":"2024-04-25T19:02:56.886443+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T19:02:56.886443+0000","last_undegraded":"2024-04-25T19:02:56.886443+0000","last_fullsized":"2024-04-25T19:02:56.886443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885651+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T19:02:56.885651+0000","last_peered":"2024-04-25T19:02:56.885651+0000","last_clean":"2024-04-25T19:02:56.885651+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T19:02:56.885651+0000","last_undegraded":"2024-04-25T19:02:56.885651+0000","last_fullsized":"2024-04-25T19:02:56.885651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886966+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T19:02:56.886966+0000","last_peered":"2024-04-25T19:02:56.886966+0000","last_clean":"2024-04-25T19:02:56.886966+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T19:02:56.886966+0000","last_undegraded":"2024-04-25T19:02:56.886966+0000","last_fullsized":"2024-04-25T19:02:56.886966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158699+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T19:02:57.158699+0000","last_peered":"2024-04-25T19:02:57.158699+0000","last_clean":"2024-04-25T19:02:57.158699+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T19:02:57.158699+0000","last_undegraded":"2024-04-25T19:02:57.158699+0000","last_fullsized":"2024-04-25T19:02:57.158699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":804398,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2832,"num_read_kb":1692,"num_write":576,"num_write_kb":2661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2413194,"data_compressed":103497,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":431,"ondisk_log_size":431,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351044,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":511178,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612681,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1245184,"data_stored":1501881,"data_compressed":65255,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":903039,"data_compressed":33735,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":903305,"data_compressed":33735,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1106189,"data_compressed":50053,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463007,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":511636,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":584043,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":512052,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":583872,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149807,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1230813,"data_compressed":48937,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888170,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":510634,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724645,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":1430770,"data_compressed":65255,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":75342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34499,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331584,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19297,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19297,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":402695,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:07.993 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T19:03:07.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T19:03:08.478 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:08 smithi086 ceph-mon[27783]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 132 KiB/s wr, 82 op/s 2024-04-25T19:03:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:08 smithi086 ceph-mon[27783]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:08 smithi183 ceph-mon[32344]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 132 KiB/s wr, 82 op/s 2024-04-25T19:03:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:08 smithi183 ceph-mon[32344]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:08 smithi194 ceph-mon[32740]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 132 KiB/s wr, 82 op/s 2024-04-25T19:03:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:08 smithi194 ceph-mon[32740]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:10.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:10 smithi086 ceph-mon[27783]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 72 op/s 2024-04-25T19:03:10.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:10 smithi183 ceph-mon[32344]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 72 op/s 2024-04-25T19:03:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 -- 172.21.15.86:0/3952004774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 msgr2=0x7fe37010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 --2- 172.21.15.86:0/3952004774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37010ef00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe358009990 tx=0x7fe358009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 -- 172.21.15.86:0/3952004774 shutdown_connections 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 --2- 172.21.15.86:0/3952004774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe37010f440 0x7fe370111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 --2- 172.21.15.86:0/3952004774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe370075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 --2- 172.21.15.86:0/3952004774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37010ef00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 -- 172.21.15.86:0/3952004774 >> 172.21.15.86:0/3952004774 conn(0x7fe370079ba0 msgr2=0x7fe370079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 -- 172.21.15.86:0/3952004774 shutdown_connections 2024-04-25T19:03:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.701+0000 7fe375000700 1 -- 172.21.15.86:0/3952004774 wait complete. 2024-04-25T19:03:10.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.702+0000 7fe375000700 1 Processor -- start 2024-04-25T19:03:10.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 -- start start 2024-04-25T19:03:10.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe37019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe37010f440 0x7fe37019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe370114e50 con 0x7fe370075590 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe370114cd0 con 0x7fe370074c70 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe375000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe370114fd0 con 0x7fe37010f440 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe367e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe36de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe37019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe36de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe37019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43538/0 (socket says 172.21.15.86:43538) 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe36de00700 1 -- 172.21.15.86:0/564618401 learned_addr learned my addr 172.21.15.86:0/564618401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.703+0000 7fe36e800700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe37010f440 0x7fe37019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe367e00700 1 -- 172.21.15.86:0/564618401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe37010f440 msgr2=0x7fe37019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe367e00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe37010f440 0x7fe37019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe367e00700 1 -- 172.21.15.86:0/564618401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 msgr2=0x7fe37019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe367e00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe37019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe367e00700 1 -- 172.21.15.86:0/564618401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3701a0370 con 0x7fe370075590 2024-04-25T19:03:10.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe36de00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe37019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:10.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe367e00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37019b8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3580098e0 tx=0x7fe3580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358026070 con 0x7fe370075590 2024-04-25T19:03:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe358003680 con 0x7fe370075590 2024-04-25T19:03:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3701a0600 con 0x7fe370075590 2024-04-25T19:03:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.704+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35800fb10 con 0x7fe370075590 2024-04-25T19:03:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.705+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3701a7ee0 con 0x7fe370075590 2024-04-25T19:03:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.706+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe370069730 con 0x7fe370075590 2024-04-25T19:03:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.708+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe358029070 con 0x7fe370075590 2024-04-25T19:03:10.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.710+0000 7fe366a00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe3540a3fe0 0x7fe3540a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:10.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.710+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fe3580d1270 con 0x7fe370075590 2024-04-25T19:03:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.710+0000 7fe36de00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe3540a3fe0 0x7fe3540a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.711+0000 7fe36de00700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe3540a3fe0 0x7fe3540a64a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe3600104e0 tx=0x7fe360007480 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:10.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.715+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe35805dff0 con 0x7fe370075590 2024-04-25T19:03:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:10 smithi194 ceph-mon[32740]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 72 op/s 2024-04-25T19:03:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.941+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe37019c6c0 con 0x7fe3540a3fe0 2024-04-25T19:03:10.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.964+0000 7fe366a00700 1 -- 172.21.15.86:0/564618401 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258129 (secure 0 0 0) 0x7fe37019c6c0 con 0x7fe3540a3fe0 2024-04-25T19:03:10.969 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe3540a3fe0 msgr2=0x7fe3540a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe3540a3fe0 0x7fe3540a64a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe3600104e0 tx=0x7fe360007480 comp rx=0 tx=0).stop 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 msgr2=0x7fe37019b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37019b8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3580098e0 tx=0x7fe3580057c0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 shutdown_connections 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe3540a3fe0 0x7fe3540a64a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe37010f440 0x7fe37019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe370074c70 0x7fe37019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 --2- 172.21.15.86:0/564618401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe370075590 0x7fe37019b8a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 >> 172.21.15.86:0/564618401 conn(0x7fe370079ba0 msgr2=0x7fe3700ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 shutdown_connections 2024-04-25T19:03:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:10.970+0000 7fe375000700 1 -- 172.21.15.86:0/564618401 wait complete. 2024-04-25T19:03:10.974 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T19:03:11.792 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-25T19:03:10.842449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1395060,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3120,"num_read_kb":2048,"num_write":780,"num_write_kb":5602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1082,"ondisk_log_size":1082,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362096,"kb_used_data":12912,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579728,"statfs":{"total":1151940427776,"available":1151569641472,"internally_reserved":0,"allocated":13221888,"data_stored":10289592,"data_compressed":383598,"data_compressed_allocated":516096,"data_compressed_original":4362240,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":202298,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":580,"num_read_kb":611,"num_write":213,"num_write_kb":1189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004041"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.159073+0000","last_change":"2024-04-25T18:58:48.576493+0000","last_active":"2024-04-25T19:02:57.159073+0000","last_peered":"2024-04-25T19:02:57.159073+0000","last_clean":"2024-04-25T19:02:57.159073+0000","last_became_active":"2024-04-25T18:58:48.576140+0000","last_became_peered":"2024-04-25T18:58:48.576140+0000","last_unstale":"2024-04-25T19:02:57.159073+0000","last_undegraded":"2024-04-25T19:02:57.159073+0000","last_fullsized":"2024-04-25T19:02:57.159073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:36.851037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888408+0000","last_change":"2024-04-25T18:58:48.575107+0000","last_active":"2024-04-25T19:02:56.888408+0000","last_peered":"2024-04-25T19:02:56.888408+0000","last_clean":"2024-04-25T19:02:56.888408+0000","last_became_active":"2024-04-25T18:58:48.574268+0000","last_became_peered":"2024-04-25T18:58:48.574268+0000","last_unstale":"2024-04-25T19:02:56.888408+0000","last_undegraded":"2024-04-25T19:02:56.888408+0000","last_fullsized":"2024-04-25T19:02:56.888408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:57.469001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888162+0000","last_change":"2024-04-25T18:58:48.566879+0000","last_active":"2024-04-25T19:02:56.888162+0000","last_peered":"2024-04-25T19:02:56.888162+0000","last_clean":"2024-04-25T19:02:56.888162+0000","last_became_active":"2024-04-25T18:58:48.566506+0000","last_became_peered":"2024-04-25T18:58:48.566506+0000","last_unstale":"2024-04-25T19:02:56.888162+0000","last_undegraded":"2024-04-25T19:02:56.888162+0000","last_fullsized":"2024-04-25T19:02:56.888162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:03.751960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827123+0000","last_change":"2024-04-25T18:58:48.563327+0000","last_active":"2024-04-25T19:02:56.827123+0000","last_peered":"2024-04-25T19:02:56.827123+0000","last_clean":"2024-04-25T19:02:56.827123+0000","last_became_active":"2024-04-25T18:58:48.562985+0000","last_became_peered":"2024-04-25T18:58:48.562985+0000","last_unstale":"2024-04-25T19:02:56.827123+0000","last_undegraded":"2024-04-25T19:02:56.827123+0000","last_fullsized":"2024-04-25T19:02:56.827123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:18.590907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885598+0000","last_change":"2024-04-25T18:58:48.574976+0000","last_active":"2024-04-25T19:02:56.885598+0000","last_peered":"2024-04-25T19:02:56.885598+0000","last_clean":"2024-04-25T19:02:56.885598+0000","last_became_active":"2024-04-25T18:58:48.574419+0000","last_became_peered":"2024-04-25T18:58:48.574419+0000","last_unstale":"2024-04-25T19:02:56.885598+0000","last_undegraded":"2024-04-25T19:02:56.885598+0000","last_fullsized":"2024-04-25T19:02:56.885598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:16.783072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838337+0000","last_change":"2024-04-25T18:59:38.317249+0000","last_active":"2024-04-25T19:02:56.838337+0000","last_peered":"2024-04-25T19:02:56.838337+0000","last_clean":"2024-04-25T19:02:56.838337+0000","last_became_active":"2024-04-25T18:59:38.316769+0000","last_became_peered":"2024-04-25T18:59:38.316769+0000","last_unstale":"2024-04-25T19:02:56.838337+0000","last_undegraded":"2024-04-25T19:02:56.838337+0000","last_fullsized":"2024-04-25T19:02:56.838337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:32.301235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885734+0000","last_change":"2024-04-25T18:58:48.570291+0000","last_active":"2024-04-25T19:02:56.885734+0000","last_peered":"2024-04-25T19:02:56.885734+0000","last_clean":"2024-04-25T19:02:56.885734+0000","last_became_active":"2024-04-25T18:58:48.569973+0000","last_became_peered":"2024-04-25T18:58:48.569973+0000","last_unstale":"2024-04-25T19:02:56.885734+0000","last_undegraded":"2024-04-25T19:02:56.885734+0000","last_fullsized":"2024-04-25T19:02:56.885734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.740558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885988+0000","last_change":"2024-04-25T18:58:48.585230+0000","last_active":"2024-04-25T19:02:56.885988+0000","last_peered":"2024-04-25T19:02:56.885988+0000","last_clean":"2024-04-25T19:02:56.885988+0000","last_became_active":"2024-04-25T18:58:48.582943+0000","last_became_peered":"2024-04-25T18:58:48.582943+0000","last_unstale":"2024-04-25T19:02:56.885988+0000","last_undegraded":"2024-04-25T19:02:56.885988+0000","last_fullsized":"2024-04-25T19:02:56.885988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.823143+0000","last_change":"2024-04-25T19:00:38.543063+0000","last_active":"2024-04-25T19:02:56.823143+0000","last_peered":"2024-04-25T19:02:56.823143+0000","last_clean":"2024-04-25T19:02:56.823143+0000","last_became_active":"2024-04-25T19:00:38.541854+0000","last_became_peered":"2024-04-25T19:00:38.541854+0000","last_unstale":"2024-04-25T19:02:56.823143+0000","last_undegraded":"2024-04-25T19:02:56.823143+0000","last_fullsized":"2024-04-25T19:02:56.823143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:10.276877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838538+0000","last_change":"2024-04-25T18:58:48.566975+0000","last_active":"2024-04-25T19:02:56.838538+0000","last_peered":"2024-04-25T19:02:56.838538+0000","last_clean":"2024-04-25T19:02:56.838538+0000","last_became_active":"2024-04-25T18:58:48.566653+0000","last_became_peered":"2024-04-25T18:58:48.566653+0000","last_unstale":"2024-04-25T19:02:56.838538+0000","last_undegraded":"2024-04-25T19:02:56.838538+0000","last_fullsized":"2024-04-25T19:02:56.838538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:03.510617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886180+0000","last_change":"2024-04-25T18:58:48.564042+0000","last_active":"2024-04-25T19:02:56.886180+0000","last_peered":"2024-04-25T19:02:56.886180+0000","last_clean":"2024-04-25T19:02:56.886180+0000","last_became_active":"2024-04-25T18:58:48.563712+0000","last_became_peered":"2024-04-25T18:58:48.563712+0000","last_unstale":"2024-04-25T19:02:56.886180+0000","last_undegraded":"2024-04-25T19:02:56.886180+0000","last_fullsized":"2024-04-25T19:02:56.886180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:40.758728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.826916+0000","last_change":"2024-04-25T18:58:48.561138+0000","last_active":"2024-04-25T19:02:56.826916+0000","last_peered":"2024-04-25T19:02:56.826916+0000","last_clean":"2024-04-25T19:02:56.826916+0000","last_became_active":"2024-04-25T18:58:48.560780+0000","last_became_peered":"2024-04-25T18:58:48.560780+0000","last_unstale":"2024-04-25T19:02:56.826916+0000","last_undegraded":"2024-04-25T19:02:56.826916+0000","last_fullsized":"2024-04-25T19:02:56.826916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:36.834394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887124+0000","last_change":"2024-04-25T18:58:48.571482+0000","last_active":"2024-04-25T19:02:56.887124+0000","last_peered":"2024-04-25T19:02:56.887124+0000","last_clean":"2024-04-25T19:02:56.887124+0000","last_became_active":"2024-04-25T18:58:48.571162+0000","last_became_peered":"2024-04-25T18:58:48.571162+0000","last_unstale":"2024-04-25T19:02:56.887124+0000","last_undegraded":"2024-04-25T19:02:56.887124+0000","last_fullsized":"2024-04-25T19:02:56.887124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:15.634542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887354+0000","last_change":"2024-04-25T18:58:48.573510+0000","last_active":"2024-04-25T19:02:56.887354+0000","last_peered":"2024-04-25T19:02:56.887354+0000","last_clean":"2024-04-25T19:02:56.887354+0000","last_became_active":"2024-04-25T18:58:48.570328+0000","last_became_peered":"2024-04-25T18:58:48.570328+0000","last_unstale":"2024-04-25T19:02:56.887354+0000","last_undegraded":"2024-04-25T19:02:56.887354+0000","last_fullsized":"2024-04-25T19:02:56.887354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:52.961657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886787+0000","last_change":"2024-04-25T18:58:48.572051+0000","last_active":"2024-04-25T19:02:56.886787+0000","last_peered":"2024-04-25T19:02:56.886787+0000","last_clean":"2024-04-25T19:02:56.886787+0000","last_became_active":"2024-04-25T18:58:48.571714+0000","last_became_peered":"2024-04-25T18:58:48.571714+0000","last_unstale":"2024-04-25T19:02:56.886787+0000","last_undegraded":"2024-04-25T19:02:56.886787+0000","last_fullsized":"2024-04-25T19:02:56.886787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.122308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885286+0000","last_change":"2024-04-25T18:58:48.565147+0000","last_active":"2024-04-25T19:02:56.885286+0000","last_peered":"2024-04-25T19:02:56.885286+0000","last_clean":"2024-04-25T19:02:56.885286+0000","last_became_active":"2024-04-25T18:58:48.564614+0000","last_became_peered":"2024-04-25T18:58:48.564614+0000","last_unstale":"2024-04-25T19:02:56.885286+0000","last_undegraded":"2024-04-25T19:02:56.885286+0000","last_fullsized":"2024-04-25T19:02:56.885286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:26.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.159169+0000","last_change":"2024-04-25T18:58:48.576417+0000","last_active":"2024-04-25T19:02:57.159169+0000","last_peered":"2024-04-25T19:02:57.159169+0000","last_clean":"2024-04-25T19:02:57.159169+0000","last_became_active":"2024-04-25T18:58:48.576059+0000","last_became_peered":"2024-04-25T18:58:48.576059+0000","last_unstale":"2024-04-25T19:02:57.159169+0000","last_undegraded":"2024-04-25T19:02:57.159169+0000","last_fullsized":"2024-04-25T19:02:57.159169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:00.311684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822913+0000","last_change":"2024-04-25T18:58:48.561821+0000","last_active":"2024-04-25T19:02:56.822913+0000","last_peered":"2024-04-25T19:02:56.822913+0000","last_clean":"2024-04-25T19:02:56.822913+0000","last_became_active":"2024-04-25T18:58:48.561400+0000","last_became_peered":"2024-04-25T18:58:48.561400+0000","last_unstale":"2024-04-25T19:02:56.822913+0000","last_undegraded":"2024-04-25T19:02:56.822913+0000","last_fullsized":"2024-04-25T19:02:56.822913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:15.631232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824330+0000","last_change":"2024-04-25T18:58:48.584727+0000","last_active":"2024-04-25T19:02:56.824330+0000","last_peered":"2024-04-25T19:02:56.824330+0000","last_clean":"2024-04-25T19:02:56.824330+0000","last_became_active":"2024-04-25T18:58:48.584466+0000","last_became_peered":"2024-04-25T18:58:48.584466+0000","last_unstale":"2024-04-25T19:02:56.824330+0000","last_undegraded":"2024-04-25T19:02:56.824330+0000","last_fullsized":"2024-04-25T19:02:56.824330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.192151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'15","reported_seq":286,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:05.998378+0000","last_change":"2024-04-25T18:58:44.509726+0000","last_active":"2024-04-25T19:03:05.998378+0000","last_peered":"2024-04-25T19:03:05.998378+0000","last_clean":"2024-04-25T19:03:05.998378+0000","last_became_active":"2024-04-25T18:58:44.508706+0000","last_became_peered":"2024-04-25T18:58:44.508706+0000","last_unstale":"2024-04-25T19:03:05.998378+0000","last_undegraded":"2024-04-25T19:03:05.998378+0000","last_fullsized":"2024-04-25T19:03:05.998378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.317624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886573+0000","last_change":"2024-04-25T18:58:48.587809+0000","last_active":"2024-04-25T19:02:56.886573+0000","last_peered":"2024-04-25T19:02:56.886573+0000","last_clean":"2024-04-25T19:02:56.886573+0000","last_became_active":"2024-04-25T18:58:48.587247+0000","last_became_peered":"2024-04-25T18:58:48.587247+0000","last_unstale":"2024-04-25T19:02:56.886573+0000","last_undegraded":"2024-04-25T19:02:56.886573+0000","last_fullsized":"2024-04-25T19:02:56.886573+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:53.026213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888176+0000","last_change":"2024-04-25T18:58:45.149938+0000","last_active":"2024-04-25T19:02:56.888176+0000","last_peered":"2024-04-25T19:02:56.888176+0000","last_clean":"2024-04-25T19:02:56.888176+0000","last_became_active":"2024-04-25T18:58:45.148987+0000","last_became_peered":"2024-04-25T18:58:45.148987+0000","last_unstale":"2024-04-25T19:02:56.888176+0000","last_undegraded":"2024-04-25T19:02:56.888176+0000","last_fullsized":"2024-04-25T19:02:56.888176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:20.989159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888446+0000","last_change":"2024-04-25T18:58:48.574620+0000","last_active":"2024-04-25T19:02:56.888446+0000","last_peered":"2024-04-25T19:02:56.888446+0000","last_clean":"2024-04-25T19:02:56.888446+0000","last_became_active":"2024-04-25T18:58:48.573660+0000","last_became_peered":"2024-04-25T18:58:48.573660+0000","last_unstale":"2024-04-25T19:02:56.888446+0000","last_undegraded":"2024-04-25T19:02:56.888446+0000","last_fullsized":"2024-04-25T19:02:56.888446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:09.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"94'13","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158963+0000","last_change":"2024-04-25T18:59:38.597378+0000","last_active":"2024-04-25T19:02:57.158963+0000","last_peered":"2024-04-25T19:02:57.158963+0000","last_clean":"2024-04-25T19:02:57.158963+0000","last_became_active":"2024-04-25T18:59:38.578414+0000","last_became_peered":"2024-04-25T18:59:38.578414+0000","last_unstale":"2024-04-25T19:02:57.158963+0000","last_undegraded":"2024-04-25T19:02:57.158963+0000","last_fullsized":"2024-04-25T19:02:57.158963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.790222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885369+0000","last_change":"2024-04-25T18:58:48.571765+0000","last_active":"2024-04-25T19:02:56.885369+0000","last_peered":"2024-04-25T19:02:56.885369+0000","last_clean":"2024-04-25T19:02:56.885369+0000","last_became_active":"2024-04-25T18:58:48.569318+0000","last_became_peered":"2024-04-25T18:58:48.569318+0000","last_unstale":"2024-04-25T19:02:56.885369+0000","last_undegraded":"2024-04-25T19:02:56.885369+0000","last_fullsized":"2024-04-25T19:02:56.885369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.359689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885839+0000","last_change":"2024-04-25T18:58:44.508750+0000","last_active":"2024-04-25T19:02:56.885839+0000","last_peered":"2024-04-25T19:02:56.885839+0000","last_clean":"2024-04-25T19:02:56.885839+0000","last_became_active":"2024-04-25T18:58:44.508384+0000","last_became_peered":"2024-04-25T18:58:44.508384+0000","last_unstale":"2024-04-25T19:02:56.885839+0000","last_undegraded":"2024-04-25T19:02:56.885839+0000","last_fullsized":"2024-04-25T19:02:56.885839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:02.807558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841607+0000","last_change":"2024-04-25T18:58:48.564674+0000","last_active":"2024-04-25T19:02:56.841607+0000","last_peered":"2024-04-25T19:02:56.841607+0000","last_clean":"2024-04-25T19:02:56.841607+0000","last_became_active":"2024-04-25T18:58:48.564320+0000","last_became_peered":"2024-04-25T18:58:48.564320+0000","last_unstale":"2024-04-25T19:02:56.841607+0000","last_undegraded":"2024-04-25T19:02:56.841607+0000","last_fullsized":"2024-04-25T19:02:56.841607+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:15.974054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":280,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:05.857713+0000","last_change":"2024-04-25T18:58:44.508742+0000","last_active":"2024-04-25T19:03:05.857713+0000","last_peered":"2024-04-25T19:03:05.857713+0000","last_clean":"2024-04-25T19:03:05.857713+0000","last_became_active":"2024-04-25T18:58:44.508408+0000","last_became_peered":"2024-04-25T18:58:44.508408+0000","last_unstale":"2024-04-25T19:03:05.857713+0000","last_undegraded":"2024-04-25T19:03:05.857713+0000","last_fullsized":"2024-04-25T19:03:05.857713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:37.529476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885332+0000","last_change":"2024-04-25T18:58:48.556956+0000","last_active":"2024-04-25T19:02:56.885332+0000","last_peered":"2024-04-25T19:02:56.885332+0000","last_clean":"2024-04-25T19:02:56.885332+0000","last_became_active":"2024-04-25T18:58:48.556551+0000","last_became_peered":"2024-04-25T18:58:48.556551+0000","last_unstale":"2024-04-25T19:02:56.885332+0000","last_undegraded":"2024-04-25T19:02:56.885332+0000","last_fullsized":"2024-04-25T19:02:56.885332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:40.464119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838196+0000","last_change":"2024-04-25T18:58:44.514073+0000","last_active":"2024-04-25T19:02:56.838196+0000","last_peered":"2024-04-25T19:02:56.838196+0000","last_clean":"2024-04-25T19:02:56.838196+0000","last_became_active":"2024-04-25T18:58:44.513800+0000","last_became_peered":"2024-04-25T18:58:44.513800+0000","last_unstale":"2024-04-25T19:02:56.838196+0000","last_undegraded":"2024-04-25T19:02:56.838196+0000","last_fullsized":"2024-04-25T19:02:56.838196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:50.422271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841395+0000","last_change":"2024-04-25T19:00:38.695351+0000","last_active":"2024-04-25T19:02:56.841395+0000","last_peered":"2024-04-25T19:02:56.841395+0000","last_clean":"2024-04-25T19:02:56.841395+0000","last_became_active":"2024-04-25T19:00:38.694923+0000","last_became_peered":"2024-04-25T19:00:38.694923+0000","last_unstale":"2024-04-25T19:02:56.841395+0000","last_undegraded":"2024-04-25T19:02:56.841395+0000","last_fullsized":"2024-04-25T19:02:56.841395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:52.773252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886306+0000","last_change":"2024-04-25T18:58:44.515385+0000","last_active":"2024-04-25T19:02:56.886306+0000","last_peered":"2024-04-25T19:02:56.886306+0000","last_clean":"2024-04-25T19:02:56.886306+0000","last_became_active":"2024-04-25T18:58:44.514895+0000","last_became_peered":"2024-04-25T18:58:44.514895+0000","last_unstale":"2024-04-25T19:02:56.886306+0000","last_undegraded":"2024-04-25T19:02:56.886306+0000","last_fullsized":"2024-04-25T19:02:56.886306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:55.372168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886498+0000","last_change":"2024-04-25T18:58:48.585270+0000","last_active":"2024-04-25T19:02:56.886498+0000","last_peered":"2024-04-25T19:02:56.886498+0000","last_clean":"2024-04-25T19:02:56.886498+0000","last_became_active":"2024-04-25T18:58:48.582518+0000","last_became_peered":"2024-04-25T18:58:48.582518+0000","last_unstale":"2024-04-25T19:02:56.886498+0000","last_undegraded":"2024-04-25T19:02:56.886498+0000","last_fullsized":"2024-04-25T19:02:56.886498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:19.564124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841097+0000","last_change":"2024-04-25T18:58:44.506938+0000","last_active":"2024-04-25T19:02:56.841097+0000","last_peered":"2024-04-25T19:02:56.841097+0000","last_clean":"2024-04-25T19:02:56.841097+0000","last_became_active":"2024-04-25T18:58:44.506591+0000","last_became_peered":"2024-04-25T18:58:44.506591+0000","last_unstale":"2024-04-25T19:02:56.841097+0000","last_undegraded":"2024-04-25T19:02:56.841097+0000","last_fullsized":"2024-04-25T19:02:56.841097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:47.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822598+0000","last_change":"2024-04-25T18:59:38.319372+0000","last_active":"2024-04-25T19:02:56.822598+0000","last_peered":"2024-04-25T19:02:56.822598+0000","last_clean":"2024-04-25T19:02:56.822598+0000","last_became_active":"2024-04-25T18:59:38.318164+0000","last_became_peered":"2024-04-25T18:59:38.318164+0000","last_unstale":"2024-04-25T19:02:56.822598+0000","last_undegraded":"2024-04-25T19:02:56.822598+0000","last_fullsized":"2024-04-25T19:02:56.822598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:43.718800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":512,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:06.858422+0000","last_change":"2024-04-25T18:58:44.512755+0000","last_active":"2024-04-25T19:03:06.858422+0000","last_peered":"2024-04-25T19:03:06.858422+0000","last_clean":"2024-04-25T19:03:06.858422+0000","last_became_active":"2024-04-25T18:58:44.512381+0000","last_became_peered":"2024-04-25T18:58:44.512381+0000","last_unstale":"2024-04-25T19:03:06.858422+0000","last_undegraded":"2024-04-25T19:03:06.858422+0000","last_fullsized":"2024-04-25T19:03:06.858422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:22.188423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":473,"num_read_kb":473,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885715+0000","last_change":"2024-04-25T18:58:48.562692+0000","last_active":"2024-04-25T19:02:56.885715+0000","last_peered":"2024-04-25T19:02:56.885715+0000","last_clean":"2024-04-25T19:02:56.885715+0000","last_became_active":"2024-04-25T18:58:48.561213+0000","last_became_peered":"2024-04-25T18:58:48.561213+0000","last_unstale":"2024-04-25T19:02:56.885715+0000","last_undegraded":"2024-04-25T19:02:56.885715+0000","last_fullsized":"2024-04-25T19:02:56.885715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:37.683160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885708+0000","last_change":"2024-04-25T18:59:38.600224+0000","last_active":"2024-04-25T19:02:56.885708+0000","last_peered":"2024-04-25T19:02:56.885708+0000","last_clean":"2024-04-25T19:02:56.885708+0000","last_became_active":"2024-04-25T18:59:38.578700+0000","last_became_peered":"2024-04-25T18:59:38.578700+0000","last_unstale":"2024-04-25T19:02:56.885708+0000","last_undegraded":"2024-04-25T19:02:56.885708+0000","last_fullsized":"2024-04-25T19:02:56.885708+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:10:29.701075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888378+0000","last_change":"2024-04-25T19:00:38.722645+0000","last_active":"2024-04-25T19:02:56.888378+0000","last_peered":"2024-04-25T19:02:56.888378+0000","last_clean":"2024-04-25T19:02:56.888378+0000","last_became_active":"2024-04-25T19:00:38.722223+0000","last_became_peered":"2024-04-25T19:00:38.722223+0000","last_unstale":"2024-04-25T19:02:56.888378+0000","last_undegraded":"2024-04-25T19:02:56.888378+0000","last_fullsized":"2024-04-25T19:02:56.888378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:51.266531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841719+0000","last_change":"2024-04-25T18:58:45.148631+0000","last_active":"2024-04-25T19:02:56.841719+0000","last_peered":"2024-04-25T19:02:56.841719+0000","last_clean":"2024-04-25T19:02:56.841719+0000","last_became_active":"2024-04-25T18:58:45.148242+0000","last_became_peered":"2024-04-25T18:58:45.148242+0000","last_unstale":"2024-04-25T19:02:56.841719+0000","last_undegraded":"2024-04-25T19:02:56.841719+0000","last_fullsized":"2024-04-25T19:02:56.841719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:52.342573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885563+0000","last_change":"2024-04-25T18:59:38.576644+0000","last_active":"2024-04-25T19:02:56.885563+0000","last_peered":"2024-04-25T19:02:56.885563+0000","last_clean":"2024-04-25T19:02:56.885563+0000","last_became_active":"2024-04-25T18:59:38.576212+0000","last_became_peered":"2024-04-25T18:59:38.576212+0000","last_unstale":"2024-04-25T19:02:56.885563+0000","last_undegraded":"2024-04-25T19:02:56.885563+0000","last_fullsized":"2024-04-25T19:02:56.885563+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:40.793917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'45","reported_seq":566,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:06.858574+0000","last_change":"2024-04-25T18:58:44.509215+0000","last_active":"2024-04-25T19:03:06.858574+0000","last_peered":"2024-04-25T19:03:06.858574+0000","last_clean":"2024-04-25T19:03:06.858574+0000","last_became_active":"2024-04-25T18:58:44.506711+0000","last_became_peered":"2024-04-25T18:58:44.506711+0000","last_unstale":"2024-04-25T19:03:06.858574+0000","last_undegraded":"2024-04-25T19:03:06.858574+0000","last_fullsized":"2024-04-25T19:03:06.858574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:14.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71554,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":314,"num_write":47,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885707+0000","last_change":"2024-04-25T18:58:48.560910+0000","last_active":"2024-04-25T19:02:56.885707+0000","last_peered":"2024-04-25T19:02:56.885707+0000","last_clean":"2024-04-25T19:02:56.885707+0000","last_became_active":"2024-04-25T18:58:48.560490+0000","last_became_peered":"2024-04-25T18:58:48.560490+0000","last_unstale":"2024-04-25T19:02:56.885707+0000","last_undegraded":"2024-04-25T19:02:56.885707+0000","last_fullsized":"2024-04-25T19:02:56.885707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:24.384407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824643+0000","last_change":"2024-04-25T18:58:44.510496+0000","last_active":"2024-04-25T19:02:56.824643+0000","last_peered":"2024-04-25T19:02:56.824643+0000","last_clean":"2024-04-25T19:02:56.824643+0000","last_became_active":"2024-04-25T18:58:44.509999+0000","last_became_peered":"2024-04-25T18:58:44.509999+0000","last_unstale":"2024-04-25T19:02:56.824643+0000","last_undegraded":"2024-04-25T19:02:56.824643+0000","last_fullsized":"2024-04-25T19:02:56.824643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:49.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886491+0000","last_change":"2024-04-25T18:58:48.575930+0000","last_active":"2024-04-25T19:02:56.886491+0000","last_peered":"2024-04-25T19:02:56.886491+0000","last_clean":"2024-04-25T19:02:56.886491+0000","last_became_active":"2024-04-25T18:58:48.575582+0000","last_became_peered":"2024-04-25T18:58:48.575582+0000","last_unstale":"2024-04-25T19:02:56.886491+0000","last_undegraded":"2024-04-25T19:02:56.886491+0000","last_fullsized":"2024-04-25T19:02:56.886491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:48.558372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841183+0000","last_change":"2024-04-25T18:58:44.512900+0000","last_active":"2024-04-25T19:02:56.841183+0000","last_peered":"2024-04-25T19:02:56.841183+0000","last_clean":"2024-04-25T19:02:56.841183+0000","last_became_active":"2024-04-25T18:58:44.512550+0000","last_became_peered":"2024-04-25T18:58:44.512550+0000","last_unstale":"2024-04-25T19:02:56.841183+0000","last_undegraded":"2024-04-25T19:02:56.841183+0000","last_fullsized":"2024-04-25T19:02:56.841183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:52.680266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886184+0000","last_change":"2024-04-25T18:58:48.573104+0000","last_active":"2024-04-25T19:02:56.886184+0000","last_peered":"2024-04-25T19:02:56.886184+0000","last_clean":"2024-04-25T19:02:56.886184+0000","last_became_active":"2024-04-25T18:58:48.572753+0000","last_became_peered":"2024-04-25T18:58:48.572753+0000","last_unstale":"2024-04-25T19:02:56.886184+0000","last_undegraded":"2024-04-25T19:02:56.886184+0000","last_fullsized":"2024-04-25T19:02:56.886184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:56.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":284,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:06.058575+0000","last_change":"2024-04-25T18:58:44.507436+0000","last_active":"2024-04-25T19:03:06.058575+0000","last_peered":"2024-04-25T19:03:06.058575+0000","last_clean":"2024-04-25T19:03:06.058575+0000","last_became_active":"2024-04-25T18:58:44.506756+0000","last_became_peered":"2024-04-25T18:58:44.506756+0000","last_unstale":"2024-04-25T19:03:06.058575+0000","last_undegraded":"2024-04-25T19:03:06.058575+0000","last_fullsized":"2024-04-25T19:03:06.058575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:46.635464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885362+0000","last_change":"2024-04-25T18:58:48.568845+0000","last_active":"2024-04-25T19:02:56.885362+0000","last_peered":"2024-04-25T19:02:56.885362+0000","last_clean":"2024-04-25T19:02:56.885362+0000","last_became_active":"2024-04-25T18:58:48.568498+0000","last_became_peered":"2024-04-25T18:58:48.568498+0000","last_unstale":"2024-04-25T19:02:56.885362+0000","last_undegraded":"2024-04-25T19:02:56.885362+0000","last_fullsized":"2024-04-25T19:02:56.885362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885273+0000","last_change":"2024-04-25T18:58:45.145792+0000","last_active":"2024-04-25T19:02:56.885273+0000","last_peered":"2024-04-25T19:02:56.885273+0000","last_clean":"2024-04-25T19:02:56.885273+0000","last_became_active":"2024-04-25T18:58:45.145447+0000","last_became_peered":"2024-04-25T18:58:45.145447+0000","last_unstale":"2024-04-25T19:02:56.885273+0000","last_undegraded":"2024-04-25T19:02:56.885273+0000","last_fullsized":"2024-04-25T19:02:56.885273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:15.056848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841117+0000","last_change":"2024-04-25T18:58:48.563802+0000","last_active":"2024-04-25T19:02:56.841117+0000","last_peered":"2024-04-25T19:02:56.841117+0000","last_clean":"2024-04-25T19:02:56.841117+0000","last_became_active":"2024-04-25T18:58:48.563456+0000","last_became_peered":"2024-04-25T18:58:48.563456+0000","last_unstale":"2024-04-25T19:02:56.841117+0000","last_undegraded":"2024-04-25T19:02:56.841117+0000","last_fullsized":"2024-04-25T19:02:56.841117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:20.698062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885693+0000","last_change":"2024-04-25T18:58:44.507609+0000","last_active":"2024-04-25T19:02:56.885693+0000","last_peered":"2024-04-25T19:02:56.885693+0000","last_clean":"2024-04-25T19:02:56.885693+0000","last_became_active":"2024-04-25T18:58:44.507260+0000","last_became_peered":"2024-04-25T18:58:44.507260+0000","last_unstale":"2024-04-25T19:02:56.885693+0000","last_undegraded":"2024-04-25T19:02:56.885693+0000","last_fullsized":"2024-04-25T19:02:56.885693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:00.190348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158836+0000","last_change":"2024-04-25T18:59:38.578855+0000","last_active":"2024-04-25T19:02:57.158836+0000","last_peered":"2024-04-25T19:02:57.158836+0000","last_clean":"2024-04-25T19:02:57.158836+0000","last_became_active":"2024-04-25T18:59:38.578524+0000","last_became_peered":"2024-04-25T18:59:38.578524+0000","last_unstale":"2024-04-25T19:02:57.158836+0000","last_undegraded":"2024-04-25T19:02:57.158836+0000","last_fullsized":"2024-04-25T19:02:57.158836+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:18.736146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822980+0000","last_change":"2024-04-25T18:58:45.150663+0000","last_active":"2024-04-25T19:02:56.822980+0000","last_peered":"2024-04-25T19:02:56.822980+0000","last_clean":"2024-04-25T19:02:56.822980+0000","last_became_active":"2024-04-25T18:58:45.149513+0000","last_became_peered":"2024-04-25T18:58:45.149513+0000","last_unstale":"2024-04-25T19:02:56.822980+0000","last_undegraded":"2024-04-25T19:02:56.822980+0000","last_fullsized":"2024-04-25T19:02:56.822980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:16.446946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838608+0000","last_change":"2024-04-25T18:58:48.585123+0000","last_active":"2024-04-25T19:02:56.838608+0000","last_peered":"2024-04-25T19:02:56.838608+0000","last_clean":"2024-04-25T19:02:56.838608+0000","last_became_active":"2024-04-25T18:58:48.584735+0000","last_became_peered":"2024-04-25T18:58:48.584735+0000","last_unstale":"2024-04-25T19:02:56.838608+0000","last_undegraded":"2024-04-25T19:02:56.838608+0000","last_fullsized":"2024-04-25T19:02:56.838608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:47.946374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'11","reported_seq":286,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:05.999403+0000","last_change":"2024-04-25T18:58:44.517233+0000","last_active":"2024-04-25T19:03:05.999403+0000","last_peered":"2024-04-25T19:03:05.999403+0000","last_clean":"2024-04-25T19:03:05.999403+0000","last_became_active":"2024-04-25T18:58:44.516828+0000","last_became_peered":"2024-04-25T18:58:44.516828+0000","last_unstale":"2024-04-25T19:03:05.999403+0000","last_undegraded":"2024-04-25T19:03:05.999403+0000","last_fullsized":"2024-04-25T19:03:05.999403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:16.893763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2684,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":241,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.838659+0000","last_change":"2024-04-25T18:58:48.580043+0000","last_active":"2024-04-25T19:02:56.838659+0000","last_peered":"2024-04-25T19:02:56.838659+0000","last_clean":"2024-04-25T19:02:56.838659+0000","last_became_active":"2024-04-25T18:58:48.579686+0000","last_became_peered":"2024-04-25T18:58:48.579686+0000","last_unstale":"2024-04-25T19:02:56.838659+0000","last_undegraded":"2024-04-25T19:02:56.838659+0000","last_fullsized":"2024-04-25T19:02:56.838659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:26.351983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.822590+0000","last_change":"2024-04-25T18:59:38.578321+0000","last_active":"2024-04-25T19:02:56.822590+0000","last_peered":"2024-04-25T19:02:56.822590+0000","last_clean":"2024-04-25T19:02:56.822590+0000","last_became_active":"2024-04-25T18:59:38.577519+0000","last_became_peered":"2024-04-25T18:59:38.577519+0000","last_unstale":"2024-04-25T19:02:56.822590+0000","last_undegraded":"2024-04-25T19:02:56.822590+0000","last_fullsized":"2024-04-25T19:02:56.822590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:11.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'7","reported_seq":278,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:06.057503+0000","last_change":"2024-04-25T18:58:45.145187+0000","last_active":"2024-04-25T19:03:06.057503+0000","last_peered":"2024-04-25T19:03:06.057503+0000","last_clean":"2024-04-25T19:03:06.057503+0000","last_became_active":"2024-04-25T18:58:45.143882+0000","last_became_peered":"2024-04-25T18:58:45.143882+0000","last_unstale":"2024-04-25T19:03:06.057503+0000","last_undegraded":"2024-04-25T19:03:06.057503+0000","last_fullsized":"2024-04-25T19:03:06.057503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:14.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887165+0000","last_change":"2024-04-25T18:58:48.582270+0000","last_active":"2024-04-25T19:02:56.887165+0000","last_peered":"2024-04-25T19:02:56.887165+0000","last_clean":"2024-04-25T19:02:56.887165+0000","last_became_active":"2024-04-25T18:58:48.581472+0000","last_became_peered":"2024-04-25T18:58:48.581472+0000","last_unstale":"2024-04-25T19:02:56.887165+0000","last_undegraded":"2024-04-25T19:02:56.887165+0000","last_fullsized":"2024-04-25T19:02:56.887165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:11:02.393248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827005+0000","last_change":"2024-04-25T18:58:44.515248+0000","last_active":"2024-04-25T19:02:56.827005+0000","last_peered":"2024-04-25T19:02:56.827005+0000","last_clean":"2024-04-25T19:02:56.827005+0000","last_became_active":"2024-04-25T18:58:44.514909+0000","last_became_peered":"2024-04-25T18:58:44.514909+0000","last_unstale":"2024-04-25T19:02:56.827005+0000","last_undegraded":"2024-04-25T19:02:56.827005+0000","last_fullsized":"2024-04-25T19:02:56.827005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:28.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":411,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:58.970251+0000","last_change":"2024-04-25T18:57:13.705288+0000","last_active":"2024-04-25T19:02:58.970251+0000","last_peered":"2024-04-25T19:02:58.970251+0000","last_clean":"2024-04-25T19:02:58.970251+0000","last_became_active":"2024-04-25T18:57:13.704791+0000","last_became_peered":"2024-04-25T18:57:13.704791+0000","last_unstale":"2024-04-25T19:02:58.970251+0000","last_undegraded":"2024-04-25T19:02:58.970251+0000","last_fullsized":"2024-04-25T19:02:58.970251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:53:20.176411+0000","last_clean_scrub_stamp":"2024-04-25T18:53:20.176411+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:06.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886196+0000","last_change":"2024-04-25T18:58:48.573855+0000","last_active":"2024-04-25T19:02:56.886196+0000","last_peered":"2024-04-25T19:02:56.886196+0000","last_clean":"2024-04-25T19:02:56.886196+0000","last_became_active":"2024-04-25T18:58:48.573320+0000","last_became_peered":"2024-04-25T18:58:48.573320+0000","last_unstale":"2024-04-25T19:02:56.886196+0000","last_undegraded":"2024-04-25T19:02:56.886196+0000","last_fullsized":"2024-04-25T19:02:56.886196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:06:15.019285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885873+0000","last_change":"2024-04-25T18:58:44.508251+0000","last_active":"2024-04-25T19:02:56.885873+0000","last_peered":"2024-04-25T19:02:56.885873+0000","last_clean":"2024-04-25T19:02:56.885873+0000","last_became_active":"2024-04-25T18:58:44.507907+0000","last_became_peered":"2024-04-25T18:58:44.507907+0000","last_unstale":"2024-04-25T19:02:56.885873+0000","last_undegraded":"2024-04-25T19:02:56.885873+0000","last_fullsized":"2024-04-25T19:02:56.885873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:41.565026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158753+0000","last_change":"2024-04-25T18:59:38.328989+0000","last_active":"2024-04-25T19:02:57.158753+0000","last_peered":"2024-04-25T19:02:57.158753+0000","last_clean":"2024-04-25T19:02:57.158753+0000","last_became_active":"2024-04-25T18:59:38.321250+0000","last_became_peered":"2024-04-25T18:59:38.321250+0000","last_unstale":"2024-04-25T19:02:57.158753+0000","last_undegraded":"2024-04-25T19:02:57.158753+0000","last_fullsized":"2024-04-25T19:02:57.158753+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:35.310346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887459+0000","last_change":"2024-04-25T18:58:48.567827+0000","last_active":"2024-04-25T19:02:56.887459+0000","last_peered":"2024-04-25T19:02:56.887459+0000","last_clean":"2024-04-25T19:02:56.887459+0000","last_became_active":"2024-04-25T18:58:48.567404+0000","last_became_peered":"2024-04-25T18:58:48.567404+0000","last_unstale":"2024-04-25T19:02:56.887459+0000","last_undegraded":"2024-04-25T19:02:56.887459+0000","last_fullsized":"2024-04-25T19:02:56.887459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.036754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'130","reported_seq":289,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.101426+0000","last_change":"2024-04-25T18:58:44.510666+0000","last_active":"2024-04-25T19:03:01.101426+0000","last_peered":"2024-04-25T19:03:01.101426+0000","last_clean":"2024-04-25T19:03:01.101426+0000","last_became_active":"2024-04-25T18:58:44.510277+0000","last_became_peered":"2024-04-25T18:58:44.510277+0000","last_unstale":"2024-04-25T19:03:01.101426+0000","last_undegraded":"2024-04-25T19:03:01.101426+0000","last_fullsized":"2024-04-25T19:03:01.101426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:04.387528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":239,"num_write_kb":1415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888152+0000","last_change":"2024-04-25T18:58:48.587831+0000","last_active":"2024-04-25T19:02:56.888152+0000","last_peered":"2024-04-25T19:02:56.888152+0000","last_clean":"2024-04-25T19:02:56.888152+0000","last_became_active":"2024-04-25T18:58:48.586019+0000","last_became_peered":"2024-04-25T18:58:48.586019+0000","last_unstale":"2024-04-25T19:02:56.888152+0000","last_undegraded":"2024-04-25T19:02:56.888152+0000","last_fullsized":"2024-04-25T19:02:56.888152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:15.559363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.827274+0000","last_change":"2024-04-25T18:58:44.623417+0000","last_active":"2024-04-25T19:02:56.827274+0000","last_peered":"2024-04-25T19:02:56.827274+0000","last_clean":"2024-04-25T19:02:56.827274+0000","last_became_active":"2024-04-25T18:58:44.623125+0000","last_became_peered":"2024-04-25T18:58:44.623125+0000","last_unstale":"2024-04-25T19:02:56.827274+0000","last_undegraded":"2024-04-25T19:02:56.827274+0000","last_fullsized":"2024-04-25T19:02:56.827274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:45.548860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888555+0000","last_change":"2024-04-25T18:58:48.576464+0000","last_active":"2024-04-25T19:02:56.888555+0000","last_peered":"2024-04-25T19:02:56.888555+0000","last_clean":"2024-04-25T19:02:56.888555+0000","last_became_active":"2024-04-25T18:58:48.575985+0000","last_became_peered":"2024-04-25T18:58:48.575985+0000","last_unstale":"2024-04-25T19:02:56.888555+0000","last_undegraded":"2024-04-25T19:02:56.888555+0000","last_fullsized":"2024-04-25T19:02:56.888555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:51.955945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885384+0000","last_change":"2024-04-25T18:58:45.146208+0000","last_active":"2024-04-25T19:02:56.885384+0000","last_peered":"2024-04-25T19:02:56.885384+0000","last_clean":"2024-04-25T19:02:56.885384+0000","last_became_active":"2024-04-25T18:58:45.145848+0000","last_became_peered":"2024-04-25T18:58:45.145848+0000","last_unstale":"2024-04-25T19:02:56.885384+0000","last_undegraded":"2024-04-25T19:02:56.885384+0000","last_fullsized":"2024-04-25T19:02:56.885384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:28.548584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885296+0000","last_change":"2024-04-25T18:58:48.561614+0000","last_active":"2024-04-25T19:02:56.885296+0000","last_peered":"2024-04-25T19:02:56.885296+0000","last_clean":"2024-04-25T19:02:56.885296+0000","last_became_active":"2024-04-25T18:58:48.561184+0000","last_became_peered":"2024-04-25T18:58:48.561184+0000","last_unstale":"2024-04-25T19:02:56.885296+0000","last_undegraded":"2024-04-25T19:02:56.885296+0000","last_fullsized":"2024-04-25T19:02:56.885296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:55.200250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886664+0000","last_change":"2024-04-25T18:58:44.516612+0000","last_active":"2024-04-25T19:02:56.886664+0000","last_peered":"2024-04-25T19:02:56.886664+0000","last_clean":"2024-04-25T19:02:56.886664+0000","last_became_active":"2024-04-25T18:58:44.516241+0000","last_became_peered":"2024-04-25T18:58:44.516241+0000","last_unstale":"2024-04-25T19:02:56.886664+0000","last_undegraded":"2024-04-25T19:02:56.886664+0000","last_fullsized":"2024-04-25T19:02:56.886664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:57.804830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158513+0000","last_change":"2024-04-25T18:58:48.570530+0000","last_active":"2024-04-25T19:02:57.158513+0000","last_peered":"2024-04-25T19:02:57.158513+0000","last_clean":"2024-04-25T19:02:57.158513+0000","last_became_active":"2024-04-25T18:58:48.570152+0000","last_became_peered":"2024-04-25T18:58:48.570152+0000","last_unstale":"2024-04-25T19:02:57.158513+0000","last_undegraded":"2024-04-25T19:02:57.158513+0000","last_fullsized":"2024-04-25T19:02:57.158513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:39.457445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824536+0000","last_change":"2024-04-25T18:58:44.514355+0000","last_active":"2024-04-25T19:02:56.824536+0000","last_peered":"2024-04-25T19:02:56.824536+0000","last_clean":"2024-04-25T19:02:56.824536+0000","last_became_active":"2024-04-25T18:58:44.513993+0000","last_became_peered":"2024-04-25T18:58:44.513993+0000","last_unstale":"2024-04-25T19:02:56.824536+0000","last_undegraded":"2024-04-25T19:02:56.824536+0000","last_fullsized":"2024-04-25T19:02:56.824536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.030303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887352+0000","last_change":"2024-04-25T18:58:48.584857+0000","last_active":"2024-04-25T19:02:56.887352+0000","last_peered":"2024-04-25T19:02:56.887352+0000","last_clean":"2024-04-25T19:02:56.887352+0000","last_became_active":"2024-04-25T18:58:48.582998+0000","last_became_peered":"2024-04-25T18:58:48.582998+0000","last_unstale":"2024-04-25T19:02:56.887352+0000","last_undegraded":"2024-04-25T19:02:56.887352+0000","last_fullsized":"2024-04-25T19:02:56.887352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.973305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885506+0000","last_change":"2024-04-25T18:58:44.512493+0000","last_active":"2024-04-25T19:02:56.885506+0000","last_peered":"2024-04-25T19:02:56.885506+0000","last_clean":"2024-04-25T19:02:56.885506+0000","last_became_active":"2024-04-25T18:58:44.512125+0000","last_became_peered":"2024-04-25T18:58:44.512125+0000","last_unstale":"2024-04-25T19:02:56.885506+0000","last_undegraded":"2024-04-25T19:02:56.885506+0000","last_fullsized":"2024-04-25T19:02:56.885506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.746478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886172+0000","last_change":"2024-04-25T18:58:48.569406+0000","last_active":"2024-04-25T19:02:56.886172+0000","last_peered":"2024-04-25T19:02:56.886172+0000","last_clean":"2024-04-25T19:02:56.886172+0000","last_became_active":"2024-04-25T18:58:48.569092+0000","last_became_peered":"2024-04-25T18:58:48.569092+0000","last_unstale":"2024-04-25T19:02:56.886172+0000","last_undegraded":"2024-04-25T19:02:56.886172+0000","last_fullsized":"2024-04-25T19:02:56.886172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:46.882061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":282,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:06.840639+0000","last_change":"2024-04-25T18:58:45.148128+0000","last_active":"2024-04-25T19:03:06.840639+0000","last_peered":"2024-04-25T19:03:06.840639+0000","last_clean":"2024-04-25T19:03:06.840639+0000","last_became_active":"2024-04-25T18:58:45.147716+0000","last_became_peered":"2024-04-25T18:58:45.147716+0000","last_unstale":"2024-04-25T19:03:06.840639+0000","last_undegraded":"2024-04-25T19:03:06.840639+0000","last_fullsized":"2024-04-25T19:03:06.840639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.500824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886332+0000","last_change":"2024-04-25T18:58:48.561535+0000","last_active":"2024-04-25T19:02:56.886332+0000","last_peered":"2024-04-25T19:02:56.886332+0000","last_clean":"2024-04-25T19:02:56.886332+0000","last_became_active":"2024-04-25T18:58:48.561204+0000","last_became_peered":"2024-04-25T18:58:48.561204+0000","last_unstale":"2024-04-25T19:02:56.886332+0000","last_undegraded":"2024-04-25T19:02:56.886332+0000","last_fullsized":"2024-04-25T19:02:56.886332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:29.222211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'75","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:03:01.092751+0000","last_change":"2024-04-25T18:58:44.510086+0000","last_active":"2024-04-25T19:03:01.092751+0000","last_peered":"2024-04-25T19:03:01.092751+0000","last_clean":"2024-04-25T19:03:01.092751+0000","last_became_active":"2024-04-25T18:58:44.509612+0000","last_became_peered":"2024-04-25T18:58:44.509612+0000","last_unstale":"2024-04-25T19:03:01.092751+0000","last_undegraded":"2024-04-25T19:03:01.092751+0000","last_fullsized":"2024-04-25T19:03:01.092751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:46.145390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885921+0000","last_change":"2024-04-25T18:58:48.582212+0000","last_active":"2024-04-25T19:02:56.885921+0000","last_peered":"2024-04-25T19:02:56.885921+0000","last_clean":"2024-04-25T19:02:56.885921+0000","last_became_active":"2024-04-25T18:58:48.581907+0000","last_became_peered":"2024-04-25T18:58:48.581907+0000","last_unstale":"2024-04-25T19:02:56.885921+0000","last_undegraded":"2024-04-25T19:02:56.885921+0000","last_fullsized":"2024-04-25T19:02:56.885921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.315271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887694+0000","last_change":"2024-04-25T18:58:44.512022+0000","last_active":"2024-04-25T19:02:56.887694+0000","last_peered":"2024-04-25T19:02:56.887694+0000","last_clean":"2024-04-25T19:02:56.887694+0000","last_became_active":"2024-04-25T18:58:44.509738+0000","last_became_peered":"2024-04-25T18:58:44.509738+0000","last_unstale":"2024-04-25T19:02:56.887694+0000","last_undegraded":"2024-04-25T19:02:56.887694+0000","last_fullsized":"2024-04-25T19:02:56.887694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:43.484987+0000","last_clean_scrub_stamp":"2024-04-25T18:58:43.484987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:47.978934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886022+0000","last_change":"2024-04-25T18:58:48.565190+0000","last_active":"2024-04-25T19:02:56.886022+0000","last_peered":"2024-04-25T19:02:56.886022+0000","last_clean":"2024-04-25T19:02:56.886022+0000","last_became_active":"2024-04-25T18:58:48.564816+0000","last_became_peered":"2024-04-25T18:58:48.564816+0000","last_unstale":"2024-04-25T19:02:56.886022+0000","last_undegraded":"2024-04-25T19:02:56.886022+0000","last_fullsized":"2024-04-25T19:02:56.886022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:12.440808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158586+0000","last_change":"2024-04-25T18:58:48.564617+0000","last_active":"2024-04-25T19:02:57.158586+0000","last_peered":"2024-04-25T19:02:57.158586+0000","last_clean":"2024-04-25T19:02:57.158586+0000","last_became_active":"2024-04-25T18:58:48.564306+0000","last_became_peered":"2024-04-25T18:58:48.564306+0000","last_unstale":"2024-04-25T19:02:57.158586+0000","last_undegraded":"2024-04-25T19:02:57.158586+0000","last_fullsized":"2024-04-25T19:02:57.158586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:59.991532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824344+0000","last_change":"2024-04-25T18:58:48.570390+0000","last_active":"2024-04-25T19:02:56.824344+0000","last_peered":"2024-04-25T19:02:56.824344+0000","last_clean":"2024-04-25T19:02:56.824344+0000","last_became_active":"2024-04-25T18:58:48.570041+0000","last_became_peered":"2024-04-25T18:58:48.570041+0000","last_unstale":"2024-04-25T19:02:56.824344+0000","last_undegraded":"2024-04-25T19:02:56.824344+0000","last_fullsized":"2024-04-25T19:02:56.824344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:10.677628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824258+0000","last_change":"2024-04-25T18:58:48.582573+0000","last_active":"2024-04-25T19:02:56.824258+0000","last_peered":"2024-04-25T19:02:56.824258+0000","last_clean":"2024-04-25T19:02:56.824258+0000","last_became_active":"2024-04-25T18:58:48.582123+0000","last_became_peered":"2024-04-25T18:58:48.582123+0000","last_unstale":"2024-04-25T19:02:56.824258+0000","last_undegraded":"2024-04-25T19:02:56.824258+0000","last_fullsized":"2024-04-25T19:02:56.824258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:33.309330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887746+0000","last_change":"2024-04-25T18:59:38.575702+0000","last_active":"2024-04-25T19:02:56.887746+0000","last_peered":"2024-04-25T19:02:56.887746+0000","last_clean":"2024-04-25T19:02:56.887746+0000","last_became_active":"2024-04-25T18:59:38.574983+0000","last_became_peered":"2024-04-25T18:59:38.574983+0000","last_unstale":"2024-04-25T19:02:56.887746+0000","last_undegraded":"2024-04-25T19:02:56.887746+0000","last_fullsized":"2024-04-25T19:02:56.887746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:30:52.532486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.887595+0000","last_change":"2024-04-25T18:58:48.579965+0000","last_active":"2024-04-25T19:02:56.887595+0000","last_peered":"2024-04-25T19:02:56.887595+0000","last_clean":"2024-04-25T19:02:56.887595+0000","last_became_active":"2024-04-25T18:58:48.578830+0000","last_became_peered":"2024-04-25T18:58:48.578830+0000","last_unstale":"2024-04-25T19:02:56.887595+0000","last_undegraded":"2024-04-25T19:02:56.887595+0000","last_fullsized":"2024-04-25T19:02:56.887595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:46.586234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158936+0000","last_change":"2024-04-25T18:58:48.565414+0000","last_active":"2024-04-25T19:02:57.158936+0000","last_peered":"2024-04-25T19:02:57.158936+0000","last_clean":"2024-04-25T19:02:57.158936+0000","last_became_active":"2024-04-25T18:58:48.565060+0000","last_became_peered":"2024-04-25T18:58:48.565060+0000","last_unstale":"2024-04-25T19:02:57.158936+0000","last_undegraded":"2024-04-25T19:02:57.158936+0000","last_fullsized":"2024-04-25T19:02:57.158936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:38.135441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.824430+0000","last_change":"2024-04-25T18:58:48.559687+0000","last_active":"2024-04-25T19:02:56.824430+0000","last_peered":"2024-04-25T19:02:56.824430+0000","last_clean":"2024-04-25T19:02:56.824430+0000","last_became_active":"2024-04-25T18:58:48.559324+0000","last_became_peered":"2024-04-25T18:58:48.559324+0000","last_unstale":"2024-04-25T19:02:56.824430+0000","last_undegraded":"2024-04-25T19:02:56.824430+0000","last_fullsized":"2024-04-25T19:02:56.824430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:54.829861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.888103+0000","last_change":"2024-04-25T18:58:48.585840+0000","last_active":"2024-04-25T19:02:56.888103+0000","last_peered":"2024-04-25T19:02:56.888103+0000","last_clean":"2024-04-25T19:02:56.888103+0000","last_became_active":"2024-04-25T18:58:48.585438+0000","last_became_peered":"2024-04-25T18:58:48.585438+0000","last_unstale":"2024-04-25T19:02:56.888103+0000","last_undegraded":"2024-04-25T19:02:56.888103+0000","last_fullsized":"2024-04-25T19:02:56.888103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:59.258417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.841025+0000","last_change":"2024-04-25T18:58:48.574788+0000","last_active":"2024-04-25T19:02:56.841025+0000","last_peered":"2024-04-25T19:02:56.841025+0000","last_clean":"2024-04-25T19:02:56.841025+0000","last_became_active":"2024-04-25T18:58:48.574433+0000","last_became_peered":"2024-04-25T18:58:48.574433+0000","last_unstale":"2024-04-25T19:02:56.841025+0000","last_undegraded":"2024-04-25T19:02:56.841025+0000","last_fullsized":"2024-04-25T19:02:56.841025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:15.366496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886443+0000","last_change":"2024-04-25T18:58:48.582241+0000","last_active":"2024-04-25T19:02:56.886443+0000","last_peered":"2024-04-25T19:02:56.886443+0000","last_clean":"2024-04-25T19:02:56.886443+0000","last_became_active":"2024-04-25T18:58:48.581874+0000","last_became_peered":"2024-04-25T18:58:48.581874+0000","last_unstale":"2024-04-25T19:02:56.886443+0000","last_undegraded":"2024-04-25T19:02:56.886443+0000","last_fullsized":"2024-04-25T19:02:56.886443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:19.632530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.885651+0000","last_change":"2024-04-25T18:58:48.574815+0000","last_active":"2024-04-25T19:02:56.885651+0000","last_peered":"2024-04-25T19:02:56.885651+0000","last_clean":"2024-04-25T19:02:56.885651+0000","last_became_active":"2024-04-25T18:58:48.574458+0000","last_became_peered":"2024-04-25T18:58:48.574458+0000","last_unstale":"2024-04-25T19:02:56.885651+0000","last_undegraded":"2024-04-25T19:02:56.885651+0000","last_fullsized":"2024-04-25T19:02:56.885651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:21.235891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:56.886966+0000","last_change":"2024-04-25T18:58:48.584362+0000","last_active":"2024-04-25T19:02:56.886966+0000","last_peered":"2024-04-25T19:02:56.886966+0000","last_clean":"2024-04-25T19:02:56.886966+0000","last_became_active":"2024-04-25T18:58:48.583996+0000","last_became_peered":"2024-04-25T18:58:48.583996+0000","last_unstale":"2024-04-25T19:02:56.886966+0000","last_undegraded":"2024-04-25T19:02:56.886966+0000","last_fullsized":"2024-04-25T19:02:56.886966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:57.292084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T19:02:57.158699+0000","last_change":"2024-04-25T18:58:48.563147+0000","last_active":"2024-04-25T19:02:57.158699+0000","last_peered":"2024-04-25T19:02:57.158699+0000","last_clean":"2024-04-25T19:02:57.158699+0000","last_became_active":"2024-04-25T18:58:48.562793+0000","last_became_peered":"2024-04-25T18:58:48.562793+0000","last_unstale":"2024-04-25T19:02:57.158699+0000","last_undegraded":"2024-04-25T19:02:57.158699+0000","last_fullsized":"2024-04-25T19:02:57.158699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:58:47.524034+0000","last_clean_scrub_stamp":"2024-04-25T18:58:47.524034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:24.031566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":804458,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2883,"num_read_kb":1718,"num_write":577,"num_write_kb":2662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2413374,"data_compressed":103497,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":432,"ondisk_log_size":432,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351045,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":511178,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612682,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1245184,"data_stored":1501941,"data_compressed":65255,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":903039,"data_compressed":33735,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037846,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":903305,"data_compressed":33735,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1106189,"data_compressed":50053,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":511636,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":584103,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738498,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":512052,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":583932,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149808,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1230813,"data_compressed":48937,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888171,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":510634,"data_compressed":14438,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724646,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":1430770,"data_compressed":65255,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":75402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34499,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331584,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19297,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19297,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":402755,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:03:11.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:03:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:12 smithi086 ceph-mon[27783]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-25T19:03:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:12 smithi086 ceph-mon[27783]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:12.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:12 smithi183 ceph-mon[32344]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-25T19:03:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:12 smithi183 ceph-mon[32344]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:12 smithi194 ceph-mon[32740]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-25T19:03:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:12 smithi194 ceph-mon[32740]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T19:03:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 -- 172.21.15.86:0/2415773655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0105aa0 msgr2=0x7f2bc0105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 --2- 172.21.15.86:0/2415773655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0105aa0 0x7f2bc0105ee0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009cd0 tx=0x7f2ba802a8f0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 -- 172.21.15.86:0/2415773655 shutdown_connections 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 --2- 172.21.15.86:0/2415773655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bc0106420 0x7f2bc010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 --2- 172.21.15.86:0/2415773655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0105aa0 0x7f2bc0105ee0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 --2- 172.21.15.86:0/2415773655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bc0101370 0x7f2bc0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 -- 172.21.15.86:0/2415773655 >> 172.21.15.86:0/2415773655 conn(0x7f2bc0077c50 msgr2=0x7f2bc0078060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 -- 172.21.15.86:0/2415773655 shutdown_connections 2024-04-25T19:03:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.274+0000 7f2bc7000640 1 -- 172.21.15.86:0/2415773655 wait complete. 2024-04-25T19:03:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.275+0000 7f2bc7000640 1 Processor -- start 2024-04-25T19:03:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.275+0000 7f2bc7000640 1 -- start start 2024-04-25T19:03:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.275+0000 7f2bc7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 0x7f2bc01a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.275+0000 7f2bc7000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bc0105aa0 0x7f2bc01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.275+0000 7f2bc7000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bc0106420 0x7f2bc019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bc7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01110e0 con 0x7f2bc0101370 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bc7000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0110f60 con 0x7f2bc0105aa0 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bc7000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0111260 con 0x7f2bc0106420 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 0x7f2bc01a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 0x7f2bc01a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60138/0 (socket says 172.21.15.86:60138) 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 -- 172.21.15.86:0/1341939984 learned_addr learned my addr 172.21.15.86:0/1341939984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbf400640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bc0105aa0 0x7f2bc01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bc5000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bc0106420 0x7f2bc019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 -- 172.21.15.86:0/1341939984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bc0106420 msgr2=0x7f2bc019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bc0106420 0x7f2bc019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 -- 172.21.15.86:0/1341939984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bc0105aa0 msgr2=0x7f2bc01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bc0105aa0 0x7f2bc01a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.276+0000 7f2bbfe00640 1 -- 172.21.15.86:0/1341939984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc019af40 con 0x7f2bc0101370 2024-04-25T19:03:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.277+0000 7f2bbfe00640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 0x7f2bc01a05c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9f0 tx=0x7f2bb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.277+0000 7f2bbcc00640 1 -- 172.21.15.86:0/1341939984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000cd80 con 0x7f2bc0101370 2024-04-25T19:03:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.277+0000 7f2bbcc00640 1 -- 172.21.15.86:0/1341939984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb00054d0 con 0x7f2bc0101370 2024-04-25T19:03:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.277+0000 7f2bbcc00640 1 -- 172.21.15.86:0/1341939984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0010690 con 0x7f2bc0101370 2024-04-25T19:03:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.277+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc019b230 con 0x7f2bc0101370 2024-04-25T19:03:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.277+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc01a7540 con 0x7f2bc0101370 2024-04-25T19:03:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.280+0000 7f2bc9d83740 1 Processor -- start 2024-04-25T19:03:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.280+0000 7f2bbcc00640 1 -- 172.21.15.86:0/1341939984 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2bb0004e90 con 0x7f2bc0101370 2024-04-25T19:03:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 -- start start 2024-04-25T19:03:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55584035b820 0x55584035bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555840366a00 0x555840368df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558402c1900 con 0x555840366a00 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555840267ef0 con 0x55584035c140 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5558402bd8c0 con 0x55584035b820 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555840366a00 0x555840368df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55584035b820 0x55584035bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43566/0 (socket says 172.21.15.86:43566) 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.281+0000 7f2bbf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555840366a00 0x555840368df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60154/0 (socket says 172.21.15.86:60154) 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bbfe00640 1 -- 172.21.15.86:0/3338177814 learned_addr learned my addr 172.21.15.86:0/3338177814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bc5000640 1 -- 172.21.15.86:0/3338177814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 msgr2=0x5558403664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bc5000640 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bc5000640 1 -- 172.21.15.86:0/3338177814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555840366a00 msgr2=0x555840368df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bc5000640 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555840366a00 0x555840368df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bc5000640 1 -- 172.21.15.86:0/3338177814 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583ffd4210 con 0x55584035b820 2024-04-25T19:03:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bc5000640 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55584035b820 0x55584035bc00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009b80 tx=0x7f2bb8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ec5c702ba0bfe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.282+0000 7f2bbcc00640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f2ba40a3ff0 0x7f2ba40a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bbcc00640 1 -- 172.21.15.86:0/1341939984 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f2bb00d0ea0 con 0x7f2bc0101370 2024-04-25T19:03:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bb5600640 1 -- 172.21.15.86:0/3338177814 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8004030 con 0x55584035b820 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bbfe00640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f2ba40a3ff0 0x7f2ba40a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bb5600640 1 -- 172.21.15.86:0/3338177814 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8005020 con 0x55584035b820 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bb5600640 1 -- 172.21.15.86:0/3338177814 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8004c60 con 0x55584035b820 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bc9d83740 1 -- 172.21.15.86:0/3338177814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55584035b820 msgr2=0x55584035bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55584035b820 0x55584035bc00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009b80 tx=0x7f2bb8011810 comp rx=0 tx=0).stop 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.283+0000 7f2bbfe00640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f2ba40a3ff0 0x7f2ba40a64b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2bb009c8b0 tx=0x7f2bb00d4970 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 -- 172.21.15.86:0/3338177814 shutdown_connections 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555840366a00 0x555840368df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/3338177814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55584035b820 0x55584035bc00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 -- 172.21.15.86:0/3338177814 >> 172.21.15.86:0/3338177814 conn(0x555840357d30 msgr2=0x555840358110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 -- 172.21.15.86:0/3338177814 shutdown_connections 2024-04-25T19:03:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 -- 172.21.15.86:0/3338177814 wait complete. 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 -- start start 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.284+0000 7f2bc9d83740 1 Processor -- start 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x5558403639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x555840363f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840364440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555840369630 con 0x55584035b820 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55584027b310 con 0x55584035c140 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5558403697b0 con 0x555840366a00 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc9d83740 10 client.0 _ll_register_callbacks cb 0x555840249cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x5558403639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x5558403639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60164/0 (socket says 172.21.15.86:60164) 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc5000640 1 -- 172.21.15.86:0/2535257048 learned_addr learned my addr 172.21.15.86:0/2535257048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bbf400640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x555840363f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.285+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840364440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc5000640 1 -- 172.21.15.86:0/2535257048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 msgr2=0x555840364440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840364440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc5000640 1 -- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 msgr2=0x555840363f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x555840363f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc5000640 1 -- 172.21.15.86:0/2535257048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558400a0620 con 0x55584035b820 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bbf400640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x555840363f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x5558403639c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005d00 tx=0x7f2bb8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12ce09a0dea6d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bae000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bae000640 1 -- 172.21.15.86:0/2535257048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb802d060 con 0x55584035b820 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bae000640 1 -- 172.21.15.86:0/2535257048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f2bb8004470 con 0x55584035b820 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb8005020 con 0x55584035b820 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bae000640 1 -- 172.21.15.86:0/2535257048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2bb801e040 con 0x55584035b820 2024-04-25T19:03:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc9d83740 10 client.15123 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.286+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558403758c0 con 0x55584035b820 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bae000640 1 -- 172.21.15.86:0/2535257048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8005020 con 0x55584035b820 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bae000640 1 -- 172.21.15.86:0/2535257048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2bb8018070 con 0x55584035b820 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bc9d83740 10 client.15123 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bc9d83740 10 client.15123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bc9d83740 20 client.15123 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555840376c70 0x555840379060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bc9d83740 4 client.15123 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.287+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558402d58b0 con 0x555840376c70 2024-04-25T19:03:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.288+0000 7f2bbf400640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555840376c70 0x555840379060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.288+0000 7f2bbf400640 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555840376c70 0x555840379060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca579b675210512 server_cookie=192720fccb315b96 in_seq=0 out_seq=0 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.288+0000 7f2bae000640 10 client.15123 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bae000640 1 -- 172.21.15.86:0/2535257048 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558402d58b0 con 0x555840376c70 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bae000640 10 client.15123 handle_command_reply: tid=0 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 client.15123 shutdown 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 msgr2=0x5558403639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x5558403639c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005d00 tx=0x7f2bb8005c50 comp rx=0 tx=0).stop 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 shutdown_connections 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555840376c70 0x555840379060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840364440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x555840363f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2535257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x5558403639c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 >> 172.21.15.86:0/2535257048 conn(0x555840357d30 msgr2=0x55584035da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bae000640 0 client.15123 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.289+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 shutdown_connections 2024-04-25T19:03:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.290+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2535257048 wait complete. 2024-04-25T19:03:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.290+0000 7f2bc9d83740 20 client.15123 trim_cache size 0 max 16384 2024-04-25T19:03:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 Processor -- start 2024-04-25T19:03:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 -- start start 2024-04-25T19:03:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x555840357610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840362b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01afe60 con 0x55584035b820 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b19a0 con 0x55584035c140 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b0530 con 0x555840366a00 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x555840357610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x555840357610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60168/0 (socket says 172.21.15.86:60168) 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.306+0000 7f2bc5000640 1 -- 172.21.15.86:0/2713765719 learned_addr learned my addr 172.21.15.86:0/2713765719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bbf400640 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840362b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 -- 172.21.15.86:0/2713765719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 msgr2=0x555840362b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840362b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 -- 172.21.15.86:0/2713765719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 msgr2=0x5558403724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 -- 172.21.15.86:0/2713765719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555840369510 con 0x55584035b820 2024-04-25T19:03:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bc5000640 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x555840357610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006010 tx=0x7f2bb8009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2cf88c92306fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bb4c00640 1 -- 172.21.15.86:0/2713765719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb802d070 con 0x55584035b820 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bb4c00640 1 -- 172.21.15.86:0/2713765719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8009e70 con 0x55584035b820 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.307+0000 7f2bb4c00640 1 -- 172.21.15.86:0/2713765719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8028dc0 con 0x55584035b820 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2713765719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 msgr2=0x555840357610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x555840357610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006010 tx=0x7f2bb8009b00 comp rx=0 tx=0).stop 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2713765719 shutdown_connections 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x555840362b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x5558403724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/2713765719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x555840357610 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2713765719 >> 172.21.15.86:0/2713765719 conn(0x555840357d30 msgr2=0x55584035d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2713765719 shutdown_connections 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.308+0000 7f2bc9d83740 1 -- 172.21.15.86:0/2713765719 wait complete. 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 -- start start 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 Processor -- start 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x55584036eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x55584036bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x55584036c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b12d0 con 0x55584035b820 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01b0c00 con 0x55584035c140 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01ade40 con 0x555840366a00 2024-04-25T19:03:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc9d83740 10 client.0 _ll_register_callbacks cb 0x7f2bb8009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.309+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x55584036eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x55584036eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60176/0 (socket says 172.21.15.86:60176) 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 -- 172.21.15.86:0/4151513399 learned_addr learned my addr 172.21.15.86:0/4151513399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bbf400640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x55584036bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 -- 172.21.15.86:0/4151513399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 msgr2=0x55584036c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x55584036c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 -- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 msgr2=0x55584036bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x55584036bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 -- 172.21.15.86:0/4151513399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584036cb20 con 0x55584035b820 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bbf400640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x55584036bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bc5000640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x55584036eba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8003a20 tx=0x7f2bb80054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dffef3ac367cf7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bae000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.310+0000 7f2bae000640 1 -- 172.21.15.86:0/4151513399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb802d070 con 0x55584035b820 2024-04-25T19:03:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bae000640 1 -- 172.21.15.86:0/4151513399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f2bb8028630 con 0x55584035b820 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb8009e70 con 0x55584035b820 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bae000640 1 -- 172.21.15.86:0/4151513399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2bb804a050 con 0x55584035b820 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 10 client.15129 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558403dcda0 con 0x55584035b820 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bae000640 1 -- 172.21.15.86:0/4151513399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8009e70 con 0x55584035b820 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bae000640 1 -- 172.21.15.86:0/4151513399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2bb8037020 con 0x55584035b820 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 10 client.15129 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 10 client.15129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 20 client.15129 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5558403dd4c0 0x5558403df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 4 client.15129 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.311+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558402cf320 con 0x5558403dd4c0 2024-04-25T19:03:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.312+0000 7f2bbf400640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5558403dd4c0 0x5558403df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.312+0000 7f2bbf400640 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5558403dd4c0 0x5558403df8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c2e76254c21312 server_cookie=326005cab2b90301 in_seq=0 out_seq=0 2024-04-25T19:03:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.312+0000 7f2bae000640 10 client.15129 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bae000640 1 -- 172.21.15.86:0/4151513399 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558402cf320 con 0x5558403dd4c0 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bae000640 10 client.15129 handle_command_reply: tid=0 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 client.15129 shutdown 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 msgr2=0x55584036eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x55584036eba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8003a20 tx=0x7f2bb80054e0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 shutdown_connections 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5558403dd4c0 0x5558403df8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555840366a00 0x55584036c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55584035c140 0x55584036bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bae000640 0 client.15129 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 --2- 172.21.15.86:0/4151513399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55584035b820 0x55584036eba0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 >> 172.21.15.86:0/4151513399 conn(0x555840357d30 msgr2=0x555840363980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 shutdown_connections 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.326+0000 7f2bc9d83740 1 -- 172.21.15.86:0/4151513399 wait complete. 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc9d83740 20 client.15129 trim_cache size 0 max 16384 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:03:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f2ba40a3ff0 msgr2=0x7f2ba40a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f2ba40a3ff0 0x7f2ba40a64b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2bb009c8b0 tx=0x7f2bb00d4970 comp rx=0 tx=0).stop 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 msgr2=0x7f2bc01a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 0x7f2bc01a05c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9f0 tx=0x7f2bb000edf0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 shutdown_connections 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f2ba40a3ff0 0x7f2ba40a64b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2bc0106420 0x7f2bc019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bc0105aa0 0x7f2bc01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 --2- 172.21.15.86:0/1341939984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc0101370 0x7f2bc01a05c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.327+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 >> 172.21.15.86:0/1341939984 conn(0x7f2bc0077c50 msgr2=0x7f2bc006a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.328+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 shutdown_connections 2024-04-25T19:03:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:13.328+0000 7f2bc7000640 1 -- 172.21.15.86:0/1341939984 wait complete. 2024-04-25T19:03:14.472 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:14 smithi086 ceph-mon[27783]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.633+0000 7fe060a00700 1 -- 172.21.15.86:0/2912933712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c06a750 msgr2=0x7fe05c06ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.633+0000 7fe060a00700 1 --2- 172.21.15.86:0/2912933712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c06a750 0x7fe05c06ab30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009990 tx=0x7fe04c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 -- 172.21.15.86:0/2912933712 shutdown_connections 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 --2- 172.21.15.86:0/2912933712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c10f4f0 0x7fe05c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 --2- 172.21.15.86:0/2912933712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c06a750 0x7fe05c06ab30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 --2- 172.21.15.86:0/2912933712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06b070 0x7fe05c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 -- 172.21.15.86:0/2912933712 >> 172.21.15.86:0/2912933712 conn(0x7fe05c0788f0 msgr2=0x7fe05c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 -- 172.21.15.86:0/2912933712 shutdown_connections 2024-04-25T19:03:14.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.634+0000 7fe060a00700 1 -- 172.21.15.86:0/2912933712 wait complete. 2024-04-25T19:03:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.635+0000 7fe060a00700 1 Processor -- start 2024-04-25T19:03:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 -- start start 2024-04-25T19:03:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06a750 0x7fe05c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c06b070 0x7fe05c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c10f4f0 0x7fe05c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c114e70 con 0x7fe05c10f4f0 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c114cf0 con 0x7fe05c06a750 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.636+0000 7fe060a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c114ff0 con 0x7fe05c06b070 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe051400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c06b070 0x7fe05c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe05a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c10f4f0 0x7fe05c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe051400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c06b070 0x7fe05c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:54090/0 (socket says 172.21.15.86:54090) 2024-04-25T19:03:14.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe051400700 1 -- 172.21.15.86:0/2673353999 learned_addr learned my addr 172.21.15.86:0/2673353999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06a750 0x7fe05c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 -- 172.21.15.86:0/2673353999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c06b070 msgr2=0x7fe05c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c06b070 0x7fe05c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 -- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c10f4f0 msgr2=0x7fe05c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c10f4f0 0x7fe05c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 -- 172.21.15.86:0/2673353999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c1a0380 con 0x7fe05c06a750 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe05a800700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c10f4f0 0x7fe05c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.637+0000 7fe059e00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06a750 0x7fe05c19b3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0098e0 tx=0x7fe04c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.638+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c027070 con 0x7fe05c06a750 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.638+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04c022470 con 0x7fe05c06a750 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.638+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c1a0610 con 0x7fe05c06a750 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.638+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c017650 con 0x7fe05c06a750 2024-04-25T19:03:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.638+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c1a7ef0 con 0x7fe05c06a750 2024-04-25T19:03:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.640+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05c101f60 con 0x7fe05c06a750 2024-04-25T19:03:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.641+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe04c02a070 con 0x7fe05c06a750 2024-04-25T19:03:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.644+0000 7fe053400700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0400a3f60 0x7fe0400a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.644+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fe04c0d1790 con 0x7fe05c06a750 2024-04-25T19:03:14.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.644+0000 7fe051400700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0400a3f60 0x7fe0400a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:14.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.645+0000 7fe051400700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0400a3f60 0x7fe0400a6420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe05c102f70 tx=0x7fe044009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:14.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.645+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe04c013070 con 0x7fe05c06a750 2024-04-25T19:03:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:14 smithi183 ceph-mon[32344]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-25T19:03:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:14 smithi194 ceph-mon[32740]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-25T19:03:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.941+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe05c02df80 con 0x7fe05c06a750 2024-04-25T19:03:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.942+0000 7fe053400700 1 -- 172.21.15.86:0/2673353999 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fe04c09dc30 con 0x7fe05c06a750 2024-04-25T19:03:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:14.945 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:03:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.945+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0400a3f60 msgr2=0x7fe0400a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.945+0000 7fe060a00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0400a3f60 0x7fe0400a6420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe05c102f70 tx=0x7fe044009260 comp rx=0 tx=0).stop 2024-04-25T19:03:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06a750 msgr2=0x7fe05c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06a750 0x7fe05c19b3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0098e0 tx=0x7fe04c0057c0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 shutdown_connections 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05c10f4f0 0x7fe05c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c06a750 0x7fe05c19b3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0400a3f60 0x7fe0400a6420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 --2- 172.21.15.86:0/2673353999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe05c06b070 0x7fe05c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 >> 172.21.15.86:0/2673353999 conn(0x7fe05c0788f0 msgr2=0x7fe05c1109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 shutdown_connections 2024-04-25T19:03:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:14.946+0000 7fe060a00700 1 -- 172.21.15.86:0/2673353999 wait complete. 2024-04-25T19:03:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:03:15.515 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:15 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2673353999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:15.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:15 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2673353999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:15.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:15 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2673353999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:15.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:03:16.299 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:16.444 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:16 smithi086 ceph-mon[27783]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 70 op/s 2024-04-25T19:03:16.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:16 smithi183 ceph-mon[32344]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 70 op/s 2024-04-25T19:03:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:16 smithi194 ceph-mon[32740]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 70 op/s 2024-04-25T19:03:18.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 -- 172.21.15.86:0/2039444064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d01092d0 msgr2=0x7f22d01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 --2- 172.21.15.86:0/2039444064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d01092d0 0x7f22d01096b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22c00099c0 tx=0x7f22c0009cd0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 -- 172.21.15.86:0/2039444064 shutdown_connections 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 --2- 172.21.15.86:0/2039444064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d010e3b0 0x7f22d0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 --2- 172.21.15.86:0/2039444064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d01092d0 0x7f22d01096b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 --2- 172.21.15.86:0/2039444064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d0109c80 0x7f22d010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.459+0000 7f22d6800700 1 -- 172.21.15.86:0/2039444064 >> 172.21.15.86:0/2039444064 conn(0x7f22d0076210 msgr2=0x7f22d0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.460+0000 7f22d6800700 1 -- 172.21.15.86:0/2039444064 shutdown_connections 2024-04-25T19:03:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.460+0000 7f22d6800700 1 -- 172.21.15.86:0/2039444064 wait complete. 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 Processor -- start 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 -- start start 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d01092d0 0x7f22d01a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 0x7f22d01a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d010e3b0 0x7f22d019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01192b0 con 0x7f22d0109c80 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0119130 con 0x7f22d010e3b0 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.461+0000 7f22d6800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0119430 con 0x7f22d01092d0 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 0x7f22d01a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 0x7f22d01a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60228/0 (socket says 172.21.15.86:60228) 2024-04-25T19:03:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 -- 172.21.15.86:0/3797594556 learned_addr learned my addr 172.21.15.86:0/3797594556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cf400700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d01092d0 0x7f22d01a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cfe00700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d010e3b0 0x7f22d019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 -- 172.21.15.86:0/3797594556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d01092d0 msgr2=0x7f22d01a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d01092d0 0x7f22d01a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 -- 172.21.15.86:0/3797594556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d010e3b0 msgr2=0x7f22d019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d010e3b0 0x7f22d019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cea00700 1 -- 172.21.15.86:0/3797594556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d019fd90 con 0x7f22d0109c80 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.462+0000 7f22cf400700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d01092d0 0x7f22d01a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:18.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.463+0000 7f22cea00700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 0x7f22d01a5960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00e9f0 tx=0x7f22bc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.463+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc004a30 con 0x7f22d0109c80 2024-04-25T19:03:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.463+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22bc00cc70 con 0x7f22d0109c80 2024-04-25T19:03:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.463+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc005530 con 0x7f22d0109c80 2024-04-25T19:03:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.463+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01a0080 con 0x7f22d0109c80 2024-04-25T19:03:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.463+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01ac2f0 con 0x7f22d0109c80 2024-04-25T19:03:18.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.464+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d0069730 con 0x7f22d0109c80 2024-04-25T19:03:18.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.466+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f22bc004bd0 con 0x7f22d0109c80 2024-04-25T19:03:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.467+0000 7f22c7e00700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22b40a3fe0 0x7f22b40a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.468+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f22bc0d0e90 con 0x7f22d0109c80 2024-04-25T19:03:18.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.468+0000 7f22cf400700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22b40a3fe0 0x7f22b40a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:18.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.468+0000 7f22cf400700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22b40a3fe0 0x7f22b40a64a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f22c0003760 tx=0x7f22c0005fb0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:18.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.472+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22bc00ce20 con 0x7f22d0109c80 2024-04-25T19:03:18.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:18 smithi086 ceph-mon[27783]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T19:03:18.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:18 smithi183 ceph-mon[32344]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T19:03:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:18 smithi194 ceph-mon[32740]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T19:03:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.753+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22d002df80 con 0x7f22d0109c80 2024-04-25T19:03:18.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.754+0000 7f22c7e00700 1 -- 172.21.15.86:0/3797594556 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f22bc09d2a0 con 0x7f22d0109c80 2024-04-25T19:03:18.756 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:18.756 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22b40a3fe0 msgr2=0x7f22b40a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22b40a3fe0 0x7f22b40a64a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f22c0003760 tx=0x7f22c0005fb0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 msgr2=0x7f22d01a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 0x7f22d01a5960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00e9f0 tx=0x7f22bc00ed00 comp rx=0 tx=0).stop 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 shutdown_connections 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22d010e3b0 0x7f22d019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22b40a3fe0 0x7f22b40a64a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d01092d0 0x7f22d01a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 --2- 172.21.15.86:0/3797594556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0109c80 0x7f22d01a5960 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 >> 172.21.15.86:0/3797594556 conn(0x7f22d0076210 msgr2=0x7f22d0076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.757+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 shutdown_connections 2024-04-25T19:03:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:18.758+0000 7f22d6800700 1 -- 172.21.15.86:0/3797594556 wait complete. 2024-04-25T19:03:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:03:19.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:19 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3797594556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:19.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:19 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3797594556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:19.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T19:03:19.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:19 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3797594556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:20.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:20 smithi086 ceph-mon[27783]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-25T19:03:20.633 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:20 smithi194 ceph-mon[32740]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-25T19:03:20.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:20 smithi183 ceph-mon[32344]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-25T19:03:22.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 -- 172.21.15.86:0/1951929615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c102af0 msgr2=0x7f4c1c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 --2- 172.21.15.86:0/1951929615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c102af0 0x7f4c1c102f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c080099c0 tx=0x7f4c08009cd0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 -- 172.21.15.86:0/1951929615 shutdown_connections 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 --2- 172.21.15.86:0/1951929615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 --2- 172.21.15.86:0/1951929615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c107940 0x7f4c1c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 --2- 172.21.15.86:0/1951929615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c102af0 0x7f4c1c102f70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.414+0000 7f4c20e00700 1 -- 172.21.15.86:0/1951929615 >> 172.21.15.86:0/1951929615 conn(0x7f4c1c079ba0 msgr2=0x7f4c1c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.415+0000 7f4c20e00700 1 -- 172.21.15.86:0/1951929615 shutdown_connections 2024-04-25T19:03:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.415+0000 7f4c20e00700 1 -- 172.21.15.86:0/1951929615 wait complete. 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 Processor -- start 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 -- start start 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c102af0 0x7f4c1c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c107940 0x7f4c1c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c112ca0 con 0x7f4c1c1034b0 2024-04-25T19:03:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c112b20 con 0x7f4c1c107940 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.416+0000 7f4c20e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c112e20 con 0x7f4c1c102af0 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48424/0 (socket says 172.21.15.86:48424) 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19400700 1 -- 172.21.15.86:0/2813251540 learned_addr learned my addr 172.21.15.86:0/2813251540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c102af0 0x7f4c1c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c1a800700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c107940 0x7f4c1c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 -- 172.21.15.86:0/2813251540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c107940 msgr2=0x7f4c1c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c107940 0x7f4c1c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 -- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 msgr2=0x7f4c1c19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 -- 172.21.15.86:0/2813251540 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c1a03b0 con 0x7f4c1c102af0 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19400700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.417+0000 7f4c19e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c102af0 0x7f4c1c19b3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00e9f0 tx=0x7f4c0c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.418+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00cb70 con 0x7f4c1c102af0 2024-04-25T19:03:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.418+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c00ee90 con 0x7f4c1c102af0 2024-04-25T19:03:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.418+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1c1a06a0 con 0x7f4c1c102af0 2024-04-25T19:03:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.418+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c003d40 con 0x7f4c1c102af0 2024-04-25T19:03:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.418+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1c1a7f20 con 0x7f4c1c102af0 2024-04-25T19:03:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.419+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1c069730 con 0x7f4c1c102af0 2024-04-25T19:03:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.421+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f4c0c003ee0 con 0x7f4c1c102af0 2024-04-25T19:03:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.422+0000 7f4c12a00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c000a3f60 0x7f4c000a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.422+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f4c0c0d0b70 con 0x7f4c1c102af0 2024-04-25T19:03:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.422+0000 7f4c19400700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c000a3f60 0x7f4c000a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.423+0000 7f4c19400700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c000a3f60 0x7f4c000a6420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c079fc0 tx=0x7f4c08005ab0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.426+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c0c09cf80 con 0x7f4c1c102af0 2024-04-25T19:03:22.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:22 smithi086 ceph-mon[27783]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T19:03:22.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.653+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c1c1a80f0 con 0x7f4c1c102af0 2024-04-25T19:03:22.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.655+0000 7f4c12a00700 1 -- 172.21.15.86:0/2813251540 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23242 (secure 0 0 0) 0x7f4c0c09c880 con 0x7f4c1c102af0 2024-04-25T19:03:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:22.657 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T19:02:56.815567+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2093097457":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1310908896":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1288701391":"2024-04-26T19:02:56.815518+0000","172.21.15.86:6800/237697797":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/324236586":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/237697797":"2024-04-26T19:02:56.815518+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:03:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c000a3f60 msgr2=0x7f4c000a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c000a3f60 0x7f4c000a6420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c079fc0 tx=0x7f4c08005ab0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c102af0 msgr2=0x7f4c1c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c102af0 0x7f4c1c19b3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00e9f0 tx=0x7f4c0c00ed00 comp rx=0 tx=0).stop 2024-04-25T19:03:22.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 shutdown_connections 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c107940 0x7f4c1c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c000a3f60 0x7f4c000a6420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1c102af0 0x7f4c1c19b3f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 --2- 172.21.15.86:0/2813251540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c1c1034b0 0x7f4c1c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.658+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 >> 172.21.15.86:0/2813251540 conn(0x7f4c1c079ba0 msgr2=0x7f4c1c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.659+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 shutdown_connections 2024-04-25T19:03:22.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:22.659+0000 7f4c20e00700 1 -- 172.21.15.86:0/2813251540 wait complete. 2024-04-25T19:03:22.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:22 smithi183 ceph-mon[32344]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T19:03:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:22 smithi194 ceph-mon[32740]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T19:03:23.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-25T19:03:23.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:23 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2813251540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:03:23.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:23 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2813251540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:03:23.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:23 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2813251540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:03:23.903 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:24.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:24 smithi086 ceph-mon[27783]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-04-25T19:03:24.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:24 smithi183 ceph-mon[32344]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-04-25T19:03:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:24 smithi194 ceph-mon[32740]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 -- 172.21.15.86:0/3100057175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860101620 msgr2=0x7f2860105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 --2- 172.21.15.86:0/3100057175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860101620 0x7f2860105ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f284c009860 tx=0x7f284c02a3d0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 -- 172.21.15.86:0/3100057175 shutdown_connections 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 --2- 172.21.15.86:0/3100057175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2860106100 0x7f2860106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 --2- 172.21.15.86:0/3100057175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860101620 0x7f2860105ae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 --2- 172.21.15.86:0/3100057175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860100d00 0x7f28601010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 -- 172.21.15.86:0/3100057175 >> 172.21.15.86:0/3100057175 conn(0x7f28600fc770 msgr2=0x7f28600feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 -- 172.21.15.86:0/3100057175 shutdown_connections 2024-04-25T19:03:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.471+0000 7f2866200640 1 -- 172.21.15.86:0/3100057175 wait complete. 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.472+0000 7f2866200640 1 Processor -- start 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.472+0000 7f2866200640 1 -- start start 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2866200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860100d00 0x7f28601a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2866200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2860101620 0x7f28601a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2866200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 0x7f286019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2866200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2860118520 con 0x7f2860106100 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2866200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28601183a0 con 0x7f2860100d00 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2866200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28601186a0 con 0x7f2860101620 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2865800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 0x7f286019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2865800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 0x7f286019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48448/0 (socket says 172.21.15.86:48448) 2024-04-25T19:03:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2864e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860100d00 0x7f28601a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2865800640 1 -- 172.21.15.86:0/11312031 learned_addr learned my addr 172.21.15.86:0/11312031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f285fe00640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2860101620 0x7f28601a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2865800640 1 -- 172.21.15.86:0/11312031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2860101620 msgr2=0x7f28601a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2865800640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2860101620 0x7f28601a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.473+0000 7f2865800640 1 -- 172.21.15.86:0/11312031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860100d00 msgr2=0x7f28601a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f2865800640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860100d00 0x7f28601a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f2865800640 1 -- 172.21.15.86:0/11312031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286019f140 con 0x7f2860106100 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f2864e00640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860100d00 0x7f28601a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f2865800640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 0x7f286019e9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f285800eb10 tx=0x7f285800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f285d600640 1 -- 172.21.15.86:0/11312031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285800ce20 con 0x7f2860106100 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f285d600640 1 -- 172.21.15.86:0/11312031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2858005020 con 0x7f2860106100 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286019f430 con 0x7f2860106100 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f285d600640 1 -- 172.21.15.86:0/11312031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858010690 con 0x7f2860106100 2024-04-25T19:03:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.474+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601ab6d0 con 0x7f2860106100 2024-04-25T19:03:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.477+0000 7f286a65e740 1 Processor -- start 2024-04-25T19:03:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.477+0000 7f285d600640 1 -- 172.21.15.86:0/11312031 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2858005510 con 0x7f2860106100 2024-04-25T19:03:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 -- start start 2024-04-25T19:03:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b74d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560688ada900 con 0x560688b749b0 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560688a80ef0 con 0x560688b7fb90 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f286a65e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560688ad68c0 con 0x560688b752d0 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f285fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b74d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f285fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b74d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48450/0 (socket says 172.21.15.86:48450) 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.478+0000 7f285fe00640 1 -- 172.21.15.86:0/1316336145 learned_addr learned my addr 172.21.15.86:0/1316336145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f2864e00640 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 -- 172.21.15.86:0/1316336145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 msgr2=0x560688b7f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 -- 172.21.15.86:0/1316336145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 msgr2=0x560688b81f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b81f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 -- 172.21.15.86:0/1316336145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606887ed210 con 0x560688b749b0 2024-04-25T19:03:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285d600640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f28480a3ec0 0x7f28480a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285fe00640 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b74d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f284c005db0 tx=0x7f284c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7506b6b1c36fa5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f285d600640 1 -- 172.21.15.86:0/11312031 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f28580d1140 con 0x7f2860106100 2024-04-25T19:03:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.479+0000 7f2864e00640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f28480a3ec0 0x7f28480a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.480+0000 7f2852000640 1 -- 172.21.15.86:0/1316336145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c040070 con 0x560688b749b0 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.480+0000 7f2852000640 1 -- 172.21.15.86:0/1316336145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284c0054f0 con 0x560688b749b0 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.480+0000 7f2852000640 1 -- 172.21.15.86:0/1316336145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c004c80 con 0x560688b749b0 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.480+0000 7f286a65e740 1 -- 172.21.15.86:0/1316336145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 msgr2=0x560688b74d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.480+0000 7f2864e00640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f28480a3ec0 0x7f28480a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x560688b75f50 tx=0x7f2854009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.480+0000 7f286a65e740 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b74d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f284c005db0 tx=0x7f284c005a90 comp rx=0 tx=0).stop 2024-04-25T19:03:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 -- 172.21.15.86:0/1316336145 shutdown_connections 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b81f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 --2- 172.21.15.86:0/1316336145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b74d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 -- 172.21.15.86:0/1316336145 >> 172.21.15.86:0/1316336145 conn(0x560688b70ec0 msgr2=0x560688b712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 -- 172.21.15.86:0/1316336145 shutdown_connections 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 -- 172.21.15.86:0/1316336145 wait complete. 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 -- start start 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.481+0000 7f286a65e740 1 Processor -- start 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b7cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560688b82760 con 0x560688b749b0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560688a94310 con 0x560688b7fb90 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560688b828e0 con 0x560688b752d0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f286a65e740 10 client.0 _ll_register_callbacks cb 0x560688a62cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f2864e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b7cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f2864e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b7cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48452/0 (socket says 172.21.15.86:48452) 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f2864e00640 1 -- 172.21.15.86:0/3110592133 learned_addr learned my addr 172.21.15.86:0/3110592133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f2865800640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f285fe00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f2864e00640 1 -- 172.21.15.86:0/3110592133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 msgr2=0x560688b7d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.482+0000 7f2864e00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2864e00640 1 -- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 msgr2=0x560688b7d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2864e00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2864e00640 1 -- 172.21.15.86:0/3110592133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606888b9620 con 0x560688b749b0 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f285fe00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2864e00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b7cb10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f285400fbd0 tx=0x7f2854004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2accb9d1226f6d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2846a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2846a00640 1 -- 172.21.15.86:0/3110592133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285402e060 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2846a00640 1 -- 172.21.15.86:0/3110592133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f2854018030 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284c0054f0 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f2846a00640 1 -- 172.21.15.86:0/3110592133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f285401e560 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f286a65e740 10 client.15159 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.483+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560688b8ea00 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f2846a00640 1 -- 172.21.15.86:0/3110592133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284c0054f0 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f2846a00640 1 -- 172.21.15.86:0/3110592133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f285401c8c0 con 0x560688b749b0 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f286a65e740 10 client.15159 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f286a65e740 10 client.15159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f286a65e740 20 client.15159 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f286a65e740 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688b8fe70 0x560688b922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f286a65e740 4 client.15159 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560688aee8b0 con 0x560688b8fe70 2024-04-25T19:03:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.484+0000 7f285fe00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688b8fe70 0x560688b922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.485+0000 7f285fe00640 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688b8fe70 0x560688b922e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e69ba834509367 server_cookie=11d2712656bbbc90 in_seq=0 out_seq=0 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.485+0000 7f2846a00640 10 client.15159 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f2846a00640 1 -- 172.21.15.86:0/3110592133 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560688aee8b0 con 0x560688b8fe70 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f2846a00640 10 client.15159 handle_command_reply: tid=0 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 client.15159 shutdown 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 msgr2=0x560688b7cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b7cb10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f285400fbd0 tx=0x7f2854004730 comp rx=0 tx=0).stop 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 shutdown_connections 2024-04-25T19:03:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688b8fe70 0x560688b922e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f2846a00640 0 client.15159 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 --2- 172.21.15.86:0/3110592133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b749b0 0x560688b7cb10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 >> 172.21.15.86:0/3110592133 conn(0x560688b70ec0 msgr2=0x560688b76c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 shutdown_connections 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.486+0000 7f286a65e740 1 -- 172.21.15.86:0/3110592133 wait complete. 2024-04-25T19:03:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.487+0000 7f286a65e740 20 client.15159 trim_cache size 0 max 16384 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.502+0000 7f286a65e740 1 Processor -- start 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.502+0000 7f286a65e740 1 -- start start 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f286a65e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b749b0 0x560688b8b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f286a65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 0x560688b77370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f286a65e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f286a65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28601b4170 con 0x560688b752d0 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f286a65e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28601b5cb0 con 0x560688b7fb90 2024-04-25T19:03:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f286a65e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28601b4840 con 0x560688b749b0 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 0x560688b77370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 0x560688b77370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48468/0 (socket says 172.21.15.86:48468) 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 -- 172.21.15.86:0/2193074751 learned_addr learned my addr 172.21.15.86:0/2193074751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f2864e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b749b0 0x560688b8b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f2865800640 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 -- 172.21.15.86:0/2193074751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b749b0 msgr2=0x560688b8b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b749b0 0x560688b8b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 -- 172.21.15.86:0/2193074751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 msgr2=0x560688b7d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.503+0000 7f285fe00640 1 -- 172.21.15.86:0/2193074751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560688b6c820 con 0x560688b752d0 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.504+0000 7f285fe00640 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 0x560688b77370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f284c0048d0 tx=0x7f284c02a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55be1c0d118808b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.504+0000 7f2851600640 1 -- 172.21.15.86:0/2193074751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c040040 con 0x560688b752d0 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.504+0000 7f2851600640 1 -- 172.21.15.86:0/2193074751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284c05a020 con 0x560688b752d0 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.504+0000 7f2851600640 1 -- 172.21.15.86:0/2193074751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c005040 con 0x560688b752d0 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 -- 172.21.15.86:0/2193074751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 msgr2=0x560688b77370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 0x560688b77370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f284c0048d0 tx=0x7f284c02a3d0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 -- 172.21.15.86:0/2193074751 shutdown_connections 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b7fb90 0x560688b7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b752d0 0x560688b77370 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 --2- 172.21.15.86:0/2193074751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b749b0 0x560688b8b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 -- 172.21.15.86:0/2193074751 >> 172.21.15.86:0/2193074751 conn(0x560688b70ec0 msgr2=0x560688b7cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 -- 172.21.15.86:0/2193074751 shutdown_connections 2024-04-25T19:03:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 -- 172.21.15.86:0/2193074751 wait complete. 2024-04-25T19:03:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 -- start start 2024-04-25T19:03:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.505+0000 7f286a65e740 1 Processor -- start 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b749b0 0x560688b87eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b85240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 0x560688b85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28601b55e0 con 0x560688b7fb90 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28601b4f10 con 0x560688b749b0 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28601b2150 con 0x560688b752d0 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f286a65e740 10 client.0 _ll_register_callbacks cb 0x7f285400fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2865800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 0x560688b85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2865800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 0x560688b85780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48476/0 (socket says 172.21.15.86:48476) 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2865800640 1 -- 172.21.15.86:0/3254487260 learned_addr learned my addr 172.21.15.86:0/3254487260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2864e00640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b749b0 0x560688b87eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f285fe00640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b85240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2865800640 1 -- 172.21.15.86:0/3254487260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 msgr2=0x560688b85240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2865800640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b85240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.506+0000 7f2865800640 1 -- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b749b0 msgr2=0x560688b87eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2865800640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b749b0 0x560688b87eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2865800640 1 -- 172.21.15.86:0/3254487260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560688b85f80 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2864e00640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b749b0 0x560688b87eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2865800640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 0x560688b85780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f28580ce740 tx=0x7f285809ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4677a09b95f7d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2846a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2846a00640 1 -- 172.21.15.86:0/3254487260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28580b6180 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2846a00640 1 -- 172.21.15.86:0/3254487260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f28580b6320 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284c05a020 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f2846a00640 1 -- 172.21.15.86:0/3254487260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f285809c2a0 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f286a65e740 10 client.15165 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.507+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560688bf6450 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f2846a00640 1 -- 172.21.15.86:0/3254487260 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284c05a020 con 0x560688b7fb90 2024-04-25T19:03:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f2846a00640 1 -- 172.21.15.86:0/3254487260 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f285809bd10 con 0x560688b7fb90 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f286a65e740 10 client.15165 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f286a65e740 10 client.15165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f286a65e740 20 client.15165 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f286a65e740 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688bf6bf0 0x560688bf9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f286a65e740 4 client.15165 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560688ae8320 con 0x560688bf6bf0 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.508+0000 7f2864e00640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688bf6bf0 0x560688bf9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.509+0000 7f2864e00640 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688bf6bf0 0x560688bf9010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84dc303c1456f92b server_cookie=c9b60f007850f70a in_seq=0 out_seq=0 2024-04-25T19:03:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.509+0000 7f2846a00640 10 client.15165 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.516+0000 7f2846a00640 1 -- 172.21.15.86:0/3254487260 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560688ae8320 con 0x560688bf6bf0 2024-04-25T19:03:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.516+0000 7f2846a00640 10 client.15165 handle_command_reply: tid=0 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 client.15165 shutdown 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 msgr2=0x560688b85780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 0x560688b85780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f28580ce740 tx=0x7f285809ce50 comp rx=0 tx=0).stop 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 shutdown_connections 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560688bf6bf0 0x560688bf9010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560688b7fb90 0x560688b85780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560688b752d0 0x560688b85240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 --2- 172.21.15.86:0/3254487260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560688b749b0 0x560688b87eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f2846a00640 0 client.15165 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 >> 172.21.15.86:0/3254487260 conn(0x560688b70ec0 msgr2=0x560688b70020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 shutdown_connections 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 1 -- 172.21.15.86:0/3254487260 wait complete. 2024-04-25T19:03:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.517+0000 7f286a65e740 20 client.15165 trim_cache size 0 max 16384 2024-04-25T19:03:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:03:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:03:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:03:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f28480a3ec0 msgr2=0x7f28480a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f28480a3ec0 0x7f28480a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x560688b75f50 tx=0x7f2854009260 comp rx=0 tx=0).stop 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 msgr2=0x7f286019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 0x7f286019e9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f285800eb10 tx=0x7f285800ef10 comp rx=0 tx=0).stop 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 shutdown_connections 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f28480a3ec0 0x7f28480a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860106100 0x7f286019e9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2860101620 0x7f28601a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 --2- 172.21.15.86:0/11312031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2860100d00 0x7f28601a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 >> 172.21.15.86:0/11312031 conn(0x7f28600fc770 msgr2=0x7f28600fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 shutdown_connections 2024-04-25T19:03:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:26.518+0000 7f2866200640 1 -- 172.21.15.86:0/11312031 wait complete. 2024-04-25T19:03:26.560 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:26 smithi086 ceph-mon[27783]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-25T19:03:26.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:26 smithi183 ceph-mon[32344]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-25T19:03:26.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:26 smithi194 ceph-mon[32740]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-25T19:03:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:{"run_dir":"/var/run/ceph"} 2024-04-25T19:03:27.085 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-25T19:03:27.086 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 336149 2024-04-25T19:03:27.086 DEBUG:tasks.quiescer:Yielding 2024-04-25T19:03:27.086 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T19:03:27.099 INFO:tasks.workunit:Pulling workunits from ref 7797745e6d8b3ac036504dc61491d368615024da 2024-04-25T19:03:27.099 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T19:03:27.099 INFO:tasks.workunit:timeout=3h 2024-04-25T19:03:27.099 INFO:tasks.workunit:cleanup=True 2024-04-25T19:03:27.100 DEBUG:teuthology.orchestra.run.smithi086:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:27.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:03:27.152 INFO:teuthology.orchestra.run.smithi086.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 67h/103d Inode: 1099511628283 Links: 2 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 18:59:30.473339607 +0000 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 18:59:30.473339607 +0000 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 19:02:55.598778690 +0000 2024-04-25T19:03:27.153 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 18:59:30.473339607 +0000 2024-04-25T19:03:27.153 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T19:03:27.153 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T19:03:27.280 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7797745e6d8b3ac036504dc61491d368615024da 2024-04-25T19:03:27.331 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T19:03:27.531 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:28.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:28 smithi086 ceph-mon[27783]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T19:03:28.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:28 smithi183 ceph-mon[32344]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T19:03:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:28 smithi194 ceph-mon[32740]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 -- 172.21.15.86:0/143039475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 msgr2=0x7f9ee0102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/143039475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee0102f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009990 tx=0x7f9ecc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 -- 172.21.15.86:0/143039475 shutdown_connections 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/143039475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/143039475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee0108a70 0x7f9ee0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/143039475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee0102f20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 -- 172.21.15.86:0/143039475 >> 172.21.15.86:0/143039475 conn(0x7f9ee0079bf0 msgr2=0x7f9ee007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:29.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 -- 172.21.15.86:0/143039475 shutdown_connections 2024-04-25T19:03:29.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.738+0000 7f9ee7a00700 1 -- 172.21.15.86:0/143039475 wait complete. 2024-04-25T19:03:29.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.739+0000 7f9ee7a00700 1 Processor -- start 2024-04-25T19:03:29.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.739+0000 7f9ee7a00700 1 -- start start 2024-04-25T19:03:29.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee7a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:29.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee7a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:29.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee7a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee0108a70 0x7f9ee019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:29.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee7a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0112c80 con 0x7f9ee0102ac0 2024-04-25T19:03:29.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee7a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0112b00 con 0x7f9ee0103460 2024-04-25T19:03:29.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee7a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0112e00 con 0x7f9ee0108a70 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee5a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee0108a70 0x7f9ee019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9edfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9ee5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9edfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59944/0 (socket says 172.21.15.86:59944) 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.740+0000 7f9edfe00700 1 -- 172.21.15.86:0/2277087451 learned_addr learned my addr 172.21.15.86:0/2277087451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edfe00700 1 -- 172.21.15.86:0/2277087451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee0108a70 msgr2=0x7f9ee019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edfe00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee0108a70 0x7f9ee019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edfe00700 1 -- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 msgr2=0x7f9ee019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edfe00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edfe00700 1 -- 172.21.15.86:0/2277087451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee01a0400 con 0x7f9ee0103460 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9ee5000700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:29.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edfe00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee019b930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc005910 tx=0x7f9ecc00b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:29.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc027070 con 0x7f9ee0103460 2024-04-25T19:03:29.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ecc022470 con 0x7f9ee0103460 2024-04-25T19:03:29.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc017650 con 0x7f9ee0103460 2024-04-25T19:03:29.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.741+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee01a0690 con 0x7f9ee0103460 2024-04-25T19:03:29.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.742+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee01a7f70 con 0x7f9ee0103460 2024-04-25T19:03:29.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.743+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee01041b0 con 0x7f9ee0103460 2024-04-25T19:03:29.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.745+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9ecc02a070 con 0x7f9ee0103460 2024-04-25T19:03:29.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.746+0000 7f9edd600700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9ec80a3f60 0x7f9ec80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.746+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f9ecc0d19e0 con 0x7f9ee0103460 2024-04-25T19:03:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.747+0000 7f9ee5000700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9ec80a3f60 0x7f9ec80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.747+0000 7f9ee5000700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9ec80a3f60 0x7f9ec80a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009fd0 tx=0x7f9ed4009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:29.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:29.749+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ecc013070 con 0x7f9ee0103460 2024-04-25T19:03:30.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.066+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ee0069730 con 0x7f9ee0103460 2024-04-25T19:03:30.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.067+0000 7f9edd600700 1 -- 172.21.15.86:0/2277087451 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f9ee0069730 con 0x7f9ee0103460 2024-04-25T19:03:30.069 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:03:30.069 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9ec80a3f60 msgr2=0x7f9ec80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9ec80a3f60 0x7f9ec80a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009fd0 tx=0x7f9ed4009260 comp rx=0 tx=0).stop 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 msgr2=0x7f9ee019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee019b930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc005910 tx=0x7f9ecc00b670 comp rx=0 tx=0).stop 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 shutdown_connections 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9ec80a3f60 0x7f9ec80a6420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.070+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee0108a70 0x7f9ee019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.071+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee0102ac0 0x7f9ee019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.071+0000 7f9ee7a00700 1 --2- 172.21.15.86:0/2277087451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee0103460 0x7f9ee019b930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.071+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 >> 172.21.15.86:0/2277087451 conn(0x7f9ee0079bf0 msgr2=0x7f9ee0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.071+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 shutdown_connections 2024-04-25T19:03:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:30.071+0000 7f9ee7a00700 1 -- 172.21.15.86:0/2277087451 wait complete. 2024-04-25T19:03:30.074 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:03:30.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:30 smithi086 ceph-mon[27783]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.9 KiB/s wr, 18 op/s 2024-04-25T19:03:30.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:30 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2277087451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:30 smithi183 ceph-mon[32344]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.9 KiB/s wr, 18 op/s 2024-04-25T19:03:30.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:30 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2277087451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:30 smithi194 ceph-mon[32740]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.9 KiB/s wr, 18 op/s 2024-04-25T19:03:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:30 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2277087451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:03:30.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:58:51.446663+0000', 'modified': '2024-04-25T19:02:56.674071+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24509, 'mds_2': 24494, 'mds_3': 24493, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/45783241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 45783241}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 45783241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2135214296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2135214296}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2135214296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/3883478130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3883478130}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3883478130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/425079996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 425079996}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 425079996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/4174239962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 4174239962}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 4174239962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.194:6837/3328875881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3328875881}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3328875881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.194:6839/2085744336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 2085744336}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 2085744336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.183:6837/3273542947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3273542947}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3273542947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.86:6839/3676043902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3676043902}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3676043902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.183:6839/703487979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 703487979}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 703487979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24509, 24494, 24493, 24478, 24527]} 2024-04-25T19:03:30.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T19:03:30.936 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-25T19:03:31.376 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:32.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:32 smithi183 ceph-mon[32344]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:03:32.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:32 smithi194 ceph-mon[32740]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:03:32.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:32 smithi086 ceph-mon[27783]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:03:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 -- 172.21.15.86:0/1283690575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a810f2d0 msgr2=0x7f49a81116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 --2- 172.21.15.86:0/1283690575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a810f2d0 0x7f49a81116c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f49a400b0a0 tx=0x7f49a400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 -- 172.21.15.86:0/1283690575 shutdown_connections 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 --2- 172.21.15.86:0/1283690575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a810f2d0 0x7f49a81116c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 --2- 172.21.15.86:0/1283690575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a8101720 0x7f49a8101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 --2- 172.21.15.86:0/1283690575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a81020d0 0x7f49a810ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.731+0000 7f49af200700 1 -- 172.21.15.86:0/1283690575 >> 172.21.15.86:0/1283690575 conn(0x7f49a80fd360 msgr2=0x7f49a80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.732+0000 7f49af200700 1 -- 172.21.15.86:0/1283690575 shutdown_connections 2024-04-25T19:03:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.732+0000 7f49af200700 1 -- 172.21.15.86:0/1283690575 wait complete. 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 Processor -- start 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 -- start start 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 0x7f49a819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a81020d0 0x7f49a819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a810f2d0 0x7f49a819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8114d90 con 0x7f49a8101720 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8114c10 con 0x7f49a81020d0 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49af200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8114f10 con 0x7f49a810f2d0 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 0x7f49a819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 0x7f49a819b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34506/0 (socket says 172.21.15.86:34506) 2024-04-25T19:03:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49ade00700 1 -- 172.21.15.86:0/2884672830 learned_addr learned my addr 172.21.15.86:0/2884672830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49ad400700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a81020d0 0x7f49a819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.733+0000 7f49ae800700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a810f2d0 0x7f49a819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ade00700 1 -- 172.21.15.86:0/2884672830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a810f2d0 msgr2=0x7f49a819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ade00700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a810f2d0 0x7f49a819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ade00700 1 -- 172.21.15.86:0/2884672830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a81020d0 msgr2=0x7f49a819b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ade00700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a81020d0 0x7f49a819b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ade00700 1 -- 172.21.15.86:0/2884672830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a81a0230 con 0x7f49a8101720 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ae800700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a810f2d0 0x7f49a819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.734+0000 7f49ade00700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 0x7f49a819b220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d950 tx=0x7f499c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.735+0000 7f49a2a00700 1 -- 172.21.15.86:0/2884672830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c016070 con 0x7f49a8101720 2024-04-25T19:03:33.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.735+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a81a0520 con 0x7f49a8101720 2024-04-25T19:03:33.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.735+0000 7f49a2a00700 1 -- 172.21.15.86:0/2884672830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c00fd40 con 0x7f49a8101720 2024-04-25T19:03:33.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.735+0000 7f49a2a00700 1 -- 172.21.15.86:0/2884672830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c00e3b0 con 0x7f49a8101720 2024-04-25T19:03:33.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.735+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a7da0 con 0x7f49a8101720 2024-04-25T19:03:33.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.738+0000 7f49a2a00700 1 -- 172.21.15.86:0/2884672830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f499c019070 con 0x7f49a8101720 2024-04-25T19:03:33.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.739+0000 7f49a2a00700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f49900a3fe0 0x7f49900a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.739+0000 7f49a2a00700 1 -- 172.21.15.86:0/2884672830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f499c0d0160 con 0x7f49a8101720 2024-04-25T19:03:33.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.740+0000 7f49ad400700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f49900a3fe0 0x7f49900a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.740+0000 7f49ad400700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f49900a3fe0 0x7f49900a64a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4998009680 tx=0x7f4998009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:33.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.741+0000 7f49bb1c1b80 1 Processor -- start 2024-04-25T19:03:33.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.741+0000 7f49bb1c1b80 1 -- start start 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863aabb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863aae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8639f3e40 con 0x55a863aa08b0 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a86399a6e0 con 0x55a863aa11d0 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a8639efe00 con 0x55a863aac080 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34512/0 (socket says 172.21.15.86:34512) 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49ade00700 1 -- 172.21.15.86:0/3935923057 learned_addr learned my addr 172.21.15.86:0/3935923057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:33.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49ad400700 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863aabb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.742+0000 7f49ae800700 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863aae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f49ad400700 1 -- 172.21.15.86:0/3935923057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 msgr2=0x55a863aae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f49ad400700 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863aae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f49ad400700 1 -- 172.21.15.86:0/3935923057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 msgr2=0x55a863aa0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f49ad400700 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f49ad400700 1 -- 172.21.15.86:0/3935923057 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a863990630 con 0x55a863aa11d0 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f49ad400700 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863aabb40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49a80ff830 tx=0x7f4998014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775a19561b041787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f4997400700 1 -- 172.21.15.86:0/3935923057 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998022070 con 0x55a863aa11d0 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f4997400700 1 -- 172.21.15.86:0/3935923057 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49980044e0 con 0x55a863aa11d0 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.743+0000 7f4997400700 1 -- 172.21.15.86:0/3935923057 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998003970 con 0x55a863aa11d0 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/3935923057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 msgr2=0x55a863aabb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863aabb40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49a80ff830 tx=0x7f4998014000 comp rx=0 tx=0).stop 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/3935923057 shutdown_connections 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863aae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/3935923057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863aabb40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/3935923057 >> 172.21.15.86:0/3935923057 conn(0x55a8639cff00 msgr2=0x55a863a9cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.744+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/3935923057 shutdown_connections 2024-04-25T19:03:33.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.745+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/3935923057 wait complete. 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.745+0000 7f49bb1c1b80 1 -- start start 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.745+0000 7f49bb1c1b80 1 Processor -- start 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 0x55a863aab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863aa8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aac080 0x55a863aa8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a863aaec00 con 0x55a863aac080 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a8639ad810 con 0x55a863aa08b0 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a863aaed80 con 0x55a863aa11d0 2024-04-25T19:03:33.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.746+0000 7f49bb1c1b80 10 client.0 _ll_register_callbacks cb 0x55a86398c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 0x55a863aab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aac080 0x55a863aa8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 0x55a863aab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43018/0 (socket says 172.21.15.86:43018) 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aac080 0x55a863aa8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34528/0 (socket says 172.21.15.86:34528) 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 -- 172.21.15.86:0/708901264 learned_addr learned my addr 172.21.15.86:0/708901264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ad400700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863aa8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 -- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 msgr2=0x55a863aa8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863aa8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 -- 172.21.15.86:0/708901264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aac080 msgr2=0x55a863aa8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aac080 0x55a863aa8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ade00700 1 -- 172.21.15.86:0/708901264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a863941880 con 0x55a863aa08b0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.747+0000 7f49ad400700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863aa8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f49ade00700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 0x55a863aab760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f499c0cd7b0 tx=0x7f499c09b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e7ad54ecab5530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f498be00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f498be00700 1 -- 172.21.15.86:0/708901264 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c020040 con 0x55a863aa08b0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f498be00700 1 -- 172.21.15.86:0/708901264 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f499c09c380 con 0x55a863aa08b0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49980044e0 con 0x55a863aa08b0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f498be00700 1 -- 172.21.15.86:0/708901264 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f499c0a02c0 con 0x55a863aa08b0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f49bb1c1b80 10 client.25310 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.748+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a86398aaa0 con 0x55a863aa08b0 2024-04-25T19:03:33.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f498be00700 1 -- 172.21.15.86:0/708901264 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49980044e0 con 0x55a863aa08b0 2024-04-25T19:03:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f498be00700 1 -- 172.21.15.86:0/708901264 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f499c0a4780 con 0x55a863aa08b0 2024-04-25T19:03:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f49bb1c1b80 10 client.25310 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f49bb1c1b80 10 client.25310 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:03:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f49bb1c1b80 20 client.25310 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863abb810 0x55a863abcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f49bb1c1b80 4 client.25310 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.749+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 --> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a863a01630 con 0x55a863abb810 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.750+0000 7f49ad400700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863abb810 0x55a863abcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.750+0000 7f49ad400700 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863abb810 0x55a863abcc80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=982e43e3aa1fb38d server_cookie=96f704528a9fddbe in_seq=0 out_seq=0 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.750+0000 7f498be00700 10 client.25310 ms_handle_connect on v2:172.21.15.194:6836/3328875881 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f498be00700 1 -- 172.21.15.86:0/708901264 <== mds.4 v2:172.21.15.194:6836/3328875881 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a863a01630 con 0x55a863abb810 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f498be00700 10 client.25310 handle_command_reply: tid=0 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 client.25310 shutdown 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 msgr2=0x55a863aab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 0x55a863aab760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f499c0cd7b0 tx=0x7f499c09b960 comp rx=0 tx=0).stop 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 shutdown_connections 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863abb810 0x55a863abcc80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aac080 0x55a863aa8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa08b0 0x55a863aab760 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f498be00700 0 client.25310 ms_handle_reset on v2:172.21.15.194:6836/3328875881 2024-04-25T19:03:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/708901264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863aa8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.751+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 >> 172.21.15.86:0/708901264 conn(0x55a8639cff00 msgr2=0x55a863a9c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.752+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 shutdown_connections 2024-04-25T19:03:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.752+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/708901264 wait complete. 2024-04-25T19:03:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.752+0000 7f49bb1c1b80 20 client.25310 trim_cache size 0 max 16384 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 Processor -- start 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 -- start start 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863a9c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863a9cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a863abafc0 con 0x55a863aa08b0 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a863abb690 con 0x55a863aa11d0 2024-04-25T19:03:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a863abbd60 con 0x55a863aac080 2024-04-25T19:03:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.774+0000 7f49ae800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863a9cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34538/0 (socket says 172.21.15.86:34538) 2024-04-25T19:03:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ae800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863a9cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50458/0 (socket says 172.21.15.86:50458) 2024-04-25T19:03:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 -- 172.21.15.86:0/1026444933 learned_addr learned my addr 172.21.15.86:0/1026444933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ad400700 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863a9c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 -- 172.21.15.86:0/1026444933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 msgr2=0x55a863a9cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863a9cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 -- 172.21.15.86:0/1026444933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 msgr2=0x55a863a9c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863a9c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 -- 172.21.15.86:0/1026444933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a863aab3a0 con 0x55a863aa08b0 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.775+0000 7f49ade00700 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa8000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b2f0 tx=0x7f499c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b9753d0e9b68c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.776+0000 7f4996a00700 1 -- 172.21.15.86:0/1026444933 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c020070 con 0x55a863aa08b0 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.776+0000 7f4996a00700 1 -- 172.21.15.86:0/1026444933 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c09c380 con 0x55a863aa08b0 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.776+0000 7f4996a00700 1 -- 172.21.15.86:0/1026444933 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0b5240 con 0x55a863aa08b0 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.776+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1026444933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 msgr2=0x55a863aa8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.776+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa8000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b2f0 tx=0x7f499c010000 comp rx=0 tx=0).stop 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1026444933 shutdown_connections 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aac080 0x55a863a9cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863aa8000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1026444933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aa11d0 0x55a863a9c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1026444933 >> 172.21.15.86:0/1026444933 conn(0x55a8639cff00 msgr2=0x55a863aa7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1026444933 shutdown_connections 2024-04-25T19:03:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1026444933 wait complete. 2024-04-25T19:03:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.777+0000 7f49bb1c1b80 1 -- start start 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 Processor -- start 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863ac6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863ac6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aac080 0x55a863ac1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a863abcb00 con 0x55a863aa08b0 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a863abc430 con 0x55a863aac080 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a863abd1d0 con 0x55a863aa11d0 2024-04-25T19:03:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49bb1c1b80 10 client.0 _ll_register_callbacks cb 0x55a86398c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863ac6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863ac6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34540/0 (socket says 172.21.15.86:34540) 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.778+0000 7f49ade00700 1 -- 172.21.15.86:0/1104829657 learned_addr learned my addr 172.21.15.86:0/1104829657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ad400700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863ac6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ae800700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aac080 0x55a863ac1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ade00700 1 -- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 msgr2=0x55a863ac6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ade00700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863ac6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ade00700 1 -- 172.21.15.86:0/1104829657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aac080 msgr2=0x55a863ac1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ade00700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aac080 0x55a863ac1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ade00700 1 -- 172.21.15.86:0/1104829657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a863ac1aa0 con 0x55a863aa08b0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f49ade00700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863ac6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f499c09a1c0 tx=0x7f499c0d00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59798cb99026fc7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f498be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f498be00700 1 -- 172.21.15.86:0/1104829657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c020070 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.779+0000 7f498be00700 1 -- 172.21.15.86:0/1104829657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f499c0b5240 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499c09c380 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f498be00700 1 -- 172.21.15.86:0/1104829657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f499c0a4900 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f49bb1c1b80 10 client.15189 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a863b23620 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f498be00700 1 -- 172.21.15.86:0/1104829657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c09c380 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f498be00700 1 -- 172.21.15.86:0/1104829657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f499c097c60 con 0x55a863aa08b0 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f49bb1c1b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f49bb1c1b80 10 client.15189 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.780+0000 7f49bb1c1b80 20 client.15189 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.781+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863b27e50 0x55a863b26b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:33.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.781+0000 7f49bb1c1b80 4 client.15189 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:03:33.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.781+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 --> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] -- command(tid 0: {"prefix": "status"}) -- 0x55a8639fb6d0 con 0x55a863b27e50 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.781+0000 7f49ad400700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863b27e50 0x55a863b26b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49ad400700 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863b27e50 0x55a863b26b60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4b774f7dde6225f server_cookie=f9d85923ee2b02d2 in_seq=0 out_seq=0 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f498be00700 10 client.15189 ms_handle_connect on v2:172.21.15.194:6836/3328875881 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f498be00700 1 -- 172.21.15.86:0/1104829657 <== mds.4 v2:172.21.15.194:6836/3328875881 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a8639fb6d0 con 0x55a863b27e50 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f498be00700 10 client.15189 handle_command_reply: tid=0 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 client.15189 shutdown 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 msgr2=0x55a863ac6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863ac6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f499c09a1c0 tx=0x7f499c0d00d0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 shutdown_connections 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:6836/3328875881,v1:172.21.15.194:6837/3328875881] conn(0x55a863b27e50 0x55a863b26b60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a863aac080 0x55a863ac1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a863aa08b0 0x55a863ac6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 --2- 172.21.15.86:0/1104829657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a863aa11d0 0x55a863ac6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f498be00700 0 client.15189 ms_handle_reset on v2:172.21.15.194:6836/3328875881 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.782+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 >> 172.21.15.86:0/1104829657 conn(0x55a8639cff00 msgr2=0x55a863aa38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 shutdown_connections 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49bb1c1b80 1 -- 172.21.15.86:0/1104829657 wait complete. 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49bb1c1b80 20 client.15189 trim_cache size 0 max 16384 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:03:33.788 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24478, 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 263.30377744499998, 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 321.888165829 2024-04-25T19:03:33.789 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f49900a3fe0 msgr2=0x7f49900a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49af200700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f49900a3fe0 0x7f49900a64a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4998009680 tx=0x7f4998009260 comp rx=0 tx=0).stop 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 msgr2=0x7f49a819b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.783+0000 7f49af200700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 0x7f49a819b220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d950 tx=0x7f499c00dc60 comp rx=0 tx=0).stop 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 shutdown_connections 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f49900a3fe0 0x7f49900a64a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a810f2d0 0x7f49a819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49a8101720 0x7f49a819b220 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 --2- 172.21.15.86:0/2884672830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49a81020d0 0x7f49a819b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 >> 172.21.15.86:0/2884672830 conn(0x7f49a80fd360 msgr2=0x7f49a8110500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 shutdown_connections 2024-04-25T19:03:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:33.784+0000 7f49af200700 1 -- 172.21.15.86:0/2884672830 wait complete. 2024-04-25T19:03:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:34 smithi086 ceph-mon[27783]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T19:03:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:34 smithi183 ceph-mon[32344]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T19:03:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:34 smithi194 ceph-mon[32740]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T19:03:34.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T19:03:35.210 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:36.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:36 smithi086 ceph-mon[27783]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T19:03:36.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:36 smithi183 ceph-mon[32344]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T19:03:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:36 smithi194 ceph-mon[32740]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T19:03:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 -- 172.21.15.86:0/1586506395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 msgr2=0x7f9870075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 --2- 172.21.15.86:0/1586506395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f9870075050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9860009990 tx=0x7f9860009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 -- 172.21.15.86:0/1586506395 shutdown_connections 2024-04-25T19:03:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 --2- 172.21.15.86:0/1586506395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 0x7f9870111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 --2- 172.21.15.86:0/1586506395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f9870075050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 --2- 172.21.15.86:0/1586506395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870075590 0x7f987010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 -- 172.21.15.86:0/1586506395 >> 172.21.15.86:0/1586506395 conn(0x7f9870079ba0 msgr2=0x7f9870079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 -- 172.21.15.86:0/1586506395 shutdown_connections 2024-04-25T19:03:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.530+0000 7f9876200700 1 -- 172.21.15.86:0/1586506395 wait complete. 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.531+0000 7f9876200700 1 Processor -- start 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.531+0000 7f9876200700 1 -- start start 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9876200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f987019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9876200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870075590 0x7f987019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9876200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 0x7f987019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9876200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9870114e60 con 0x7f9870075590 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9876200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9870114ce0 con 0x7f9870074c70 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9876200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9870114fe0 con 0x7f987010f440 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f987019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f987019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43062/0 (socket says 172.21.15.86:43062) 2024-04-25T19:03:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f9867e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870075590 0x7f987019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 0x7f987019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 -- 172.21.15.86:0/3203271838 learned_addr learned my addr 172.21.15.86:0/3203271838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 -- 172.21.15.86:0/3203271838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 msgr2=0x7f987019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 0x7f987019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 -- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870075590 msgr2=0x7f987019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.532+0000 7f986f400700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870075590 0x7f987019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f986f400700 1 -- 172.21.15.86:0/3203271838 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701a0300 con 0x7f9870074c70 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f986fe00700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 0x7f987019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f986f400700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f987019b2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9860005910 tx=0x7f986000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f986cc00700 1 -- 172.21.15.86:0/3203271838 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860027070 con 0x7f9870074c70 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f986cc00700 1 -- 172.21.15.86:0/3203271838 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860022470 con 0x7f9870074c70 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f986cc00700 1 -- 172.21.15.86:0/3203271838 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017650 con 0x7f9870074c70 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98701a0590 con 0x7f9870074c70 2024-04-25T19:03:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.533+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701a7e70 con 0x7f9870074c70 2024-04-25T19:03:37.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.538+0000 7f986cc00700 1 -- 172.21.15.86:0/3203271838 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f986002a070 con 0x7f9870074c70 2024-04-25T19:03:37.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.539+0000 7f986cc00700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98540a3fe0 0x7f98540a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.539+0000 7f9880ae1b80 1 Processor -- start 2024-04-25T19:03:37.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.539+0000 7f9880ae1b80 1 -- start start 2024-04-25T19:03:37.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.539+0000 7f986cc00700 1 -- 172.21.15.86:0/3203271838 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f98600d1790 con 0x7f9870074c70 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9867e00700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98540a3fe0 0x7f98540a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d5bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 0x5578b9d559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578b9ca7e40 con 0x5578b9d55ef0 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578b9c4e6e0 con 0x5578b9d5c0a0 2024-04-25T19:03:37.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5578b9ca3e00 con 0x5578b9d5b780 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.540+0000 7f9867e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 0x5578b9d559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f986fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f986f400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d5bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f986f400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d5bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50500/0 (socket says 172.21.15.86:50500) 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f986fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d56330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34572/0 (socket says 172.21.15.86:34572) 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f986fe00700 1 -- 172.21.15.86:0/3328417348 learned_addr learned my addr 172.21.15.86:0/3328417348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:37.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 0x5578b9d559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43064/0 (socket says 172.21.15.86:43064) 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98540a3fe0 0x7f98540a64a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9858009680 tx=0x7f9858009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 -- 172.21.15.86:0/3328417348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 msgr2=0x5578b9d5bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d5bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 -- 172.21.15.86:0/3328417348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 msgr2=0x5578b9d56330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d56330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 -- 172.21.15.86:0/3328417348 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b9c44630 con 0x5578b9d5c0a0 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.541+0000 7f9867e00700 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 0x5578b9d559b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9870075f70 tx=0x7f9858019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e599d8e39878d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.542+0000 7f9864c00700 1 -- 172.21.15.86:0/3328417348 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858004230 con 0x5578b9d5c0a0 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.542+0000 7f9864c00700 1 -- 172.21.15.86:0/3328417348 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985801cdc0 con 0x5578b9d5c0a0 2024-04-25T19:03:37.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.542+0000 7f9864c00700 1 -- 172.21.15.86:0/3328417348 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858003e30 con 0x5578b9d5c0a0 2024-04-25T19:03:37.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.542+0000 7f9880ae1b80 1 -- 172.21.15.86:0/3328417348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 msgr2=0x5578b9d559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.542+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 0x5578b9d559b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9870075f70 tx=0x7f9858019000 comp rx=0 tx=0).stop 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 -- 172.21.15.86:0/3328417348 shutdown_connections 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d56330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d5bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/3328417348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5c0a0 0x5578b9d559b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 -- 172.21.15.86:0/3328417348 >> 172.21.15.86:0/3328417348 conn(0x5578b9c83f00 msgr2=0x5578b9d508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 -- 172.21.15.86:0/3328417348 shutdown_connections 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 -- 172.21.15.86:0/3328417348 wait complete. 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 -- start start 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.543+0000 7f9880ae1b80 1 Processor -- start 2024-04-25T19:03:37.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 0x5578b9d61cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5b780 0x5578b9d5efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5f4f0 0x5578b9d5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d62920 con 0x5578b9d5b780 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578b9c61810 con 0x5578b9d55ef0 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d62aa0 con 0x5578b9d5f4f0 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f9880ae1b80 10 client.0 _ll_register_callbacks cb 0x5578b9c406d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f986f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 0x5578b9d61cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f986f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 0x5578b9d61cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43080/0 (socket says 172.21.15.86:43080) 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f986f400700 1 -- 172.21.15.86:0/2413486841 learned_addr learned my addr 172.21.15.86:0/2413486841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.544+0000 7f986fe00700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5f4f0 0x5578b9d5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f9867e00700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5b780 0x5578b9d5efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986f400700 1 -- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5f4f0 msgr2=0x5578b9d5f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986f400700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5f4f0 0x5578b9d5f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986f400700 1 -- 172.21.15.86:0/2413486841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5b780 msgr2=0x5578b9d5efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986f400700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5b780 0x5578b9d5efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986f400700 1 -- 172.21.15.86:0/2413486841 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b9bf5880 con 0x5578b9d55ef0 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986fe00700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5f4f0 0x5578b9d5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f9867e00700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5b780 0x5578b9d5efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f986f400700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 0x5578b9d61cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9860000c00 tx=0x7f986009d120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b1973d6dbbe4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f985d600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f985d600700 1 -- 172.21.15.86:0/2413486841 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986009d530 con 0x5578b9d55ef0 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f985d600700 1 -- 172.21.15.86:0/2413486841 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f986009d9a0 con 0x5578b9d55ef0 2024-04-25T19:03:37.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.545+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985801cdc0 con 0x5578b9d55ef0 2024-04-25T19:03:37.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f985d600700 1 -- 172.21.15.86:0/2413486841 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f98600bb050 con 0x5578b9d55ef0 2024-04-25T19:03:37.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 10 client.25328 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:37.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578b9c3eaa0 con 0x5578b9d55ef0 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f985d600700 1 -- 172.21.15.86:0/2413486841 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985801cdc0 con 0x5578b9d55ef0 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f985d600700 1 -- 172.21.15.86:0/2413486841 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f98600cef60 con 0x5578b9d55ef0 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 10 client.25328 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 10 client.25328 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 20 client.25328 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9d6fa30 0x5578b9d70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 4 client.25328 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.546+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 --> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b9cb5630 con 0x5578b9d6fa30 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.547+0000 7f9867e00700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9d6fa30 0x5578b9d70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.547+0000 7f9867e00700 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9d6fa30 0x5578b9d70ea0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=193144307e22e303 server_cookie=9f850efbb323b804 in_seq=0 out_seq=0 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.547+0000 7f985d600700 10 client.25328 ms_handle_connect on v2:172.21.15.194:6838/2085744336 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f985d600700 1 -- 172.21.15.86:0/2413486841 <== mds.3 v2:172.21.15.194:6838/2085744336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578b9cb5630 con 0x5578b9d6fa30 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f985d600700 10 client.25328 handle_command_reply: tid=0 2024-04-25T19:03:37.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 client.25328 shutdown 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 msgr2=0x5578b9d61cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 0x5578b9d61cb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9860000c00 tx=0x7f986009d120 comp rx=0 tx=0).stop 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 shutdown_connections 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5f4f0 0x5578b9d5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9d6fa30 0x5578b9d70ea0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d55ef0 0x5578b9d61cb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2413486841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5b780 0x5578b9d5efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f985d600700 0 client.25328 ms_handle_reset on v2:172.21.15.194:6838/2085744336 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.548+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 >> 172.21.15.86:0/2413486841 conn(0x5578b9c83f00 msgr2=0x5578b9d4fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.549+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 shutdown_connections 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.549+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2413486841 wait complete. 2024-04-25T19:03:37.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.549+0000 7f9880ae1b80 20 client.25328 trim_cache size 0 max 16384 2024-04-25T19:03:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 Processor -- start 2024-04-25T19:03:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 -- start start 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 0x5578b9d559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d7a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d6f1e0 con 0x5578b9d55ef0 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d6f8b0 con 0x5578b9c83f00 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.571+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d6ff80 con 0x5578b9d5b780 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 0x5578b9d559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 0x5578b9d559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43086/0 (socket says 172.21.15.86:43086) 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 -- 172.21.15.86:0/2602517393 learned_addr learned my addr 172.21.15.86:0/2602517393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f9867e00700 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986fe00700 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d7a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 -- 172.21.15.86:0/2602517393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 msgr2=0x5578b9d7a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d7a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 -- 172.21.15.86:0/2602517393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 msgr2=0x5578b9d505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 -- 172.21.15.86:0/2602517393 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b9d6f010 con 0x5578b9c83f00 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986fe00700 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.572+0000 7f986f400700 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 0x5578b9d559b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f98600ced40 tx=0x7f9860022cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f085c310f97fdec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f985fe00700 1 -- 172.21.15.86:0/2602517393 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986009d530 con 0x5578b9c83f00 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f985fe00700 1 -- 172.21.15.86:0/2602517393 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98600a18e0 con 0x5578b9c83f00 2024-04-25T19:03:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f985fe00700 1 -- 172.21.15.86:0/2602517393 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860099300 con 0x5578b9c83f00 2024-04-25T19:03:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2602517393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 msgr2=0x5578b9d559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 0x5578b9d559b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f98600ced40 tx=0x7f9860022cc0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2602517393 shutdown_connections 2024-04-25T19:03:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d5b780 0x5578b9d7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9c83f00 0x5578b9d559b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2602517393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d55ef0 0x5578b9d505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.573+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2602517393 >> 172.21.15.86:0/2602517393 conn(0x5578b9d4ce30 msgr2=0x5578b9d5d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.574+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2602517393 shutdown_connections 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.574+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2602517393 wait complete. 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.574+0000 7f9880ae1b80 1 -- start start 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.574+0000 7f9880ae1b80 1 Processor -- start 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d55ef0 0x5578b9d752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 0x5578b9d75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5f4f0 0x5578b9d75d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d70d20 con 0x5578b9d5f4f0 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d70650 con 0x5578b9d5b780 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d713f0 con 0x5578b9d55ef0 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9880ae1b80 10 client.0 _ll_register_callbacks cb 0x5578b9c406d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 0x5578b9d75810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f986fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5f4f0 0x5578b9d75d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 0x5578b9d75810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43088/0 (socket says 172.21.15.86:43088) 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 -- 172.21.15.86:0/2027835240 learned_addr learned my addr 172.21.15.86:0/2027835240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f986f400700 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d55ef0 0x5578b9d752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 -- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d55ef0 msgr2=0x5578b9d752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d55ef0 0x5578b9d752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 -- 172.21.15.86:0/2027835240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5f4f0 msgr2=0x5578b9d75d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5f4f0 0x5578b9d75d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.575+0000 7f9867e00700 1 -- 172.21.15.86:0/2027835240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b9dd6b60 con 0x5578b9d5b780 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f9867e00700 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 0x5578b9d75810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f985801c720 tx=0x7f9858009ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9ce3ab4dde6754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f985d600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f985d600700 1 -- 172.21.15.86:0/2027835240 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858039060 con 0x5578b9d5b780 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f985d600700 1 -- 172.21.15.86:0/2027835240 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f985801cb30 con 0x5578b9d5b780 2024-04-25T19:03:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98600a18e0 con 0x5578b9d5b780 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f985d600700 1 -- 172.21.15.86:0/2027835240 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f985802a040 con 0x5578b9d5b780 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f9880ae1b80 10 client.25334 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.576+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578b9dd7d50 con 0x5578b9d5b780 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f985d600700 1 -- 172.21.15.86:0/2027835240 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98600a18e0 con 0x5578b9d5b780 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f985d600700 1 -- 172.21.15.86:0/2027835240 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9858023070 con 0x5578b9d5b780 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f9880ae1b80 10 client.25334 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f9880ae1b80 10 client.25334 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f9880ae1b80 20 client.25334 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9ddc4c0 0x5578b9ddb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f9880ae1b80 4 client.25334 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:03:37.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 --> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] -- command(tid 0: {"prefix": "status"}) -- 0x5578b9caf6d0 con 0x5578b9ddc4c0 2024-04-25T19:03:37.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.577+0000 7f986f400700 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9ddc4c0 0x5578b9ddb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f986f400700 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9ddc4c0 0x5578b9ddb1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b48726a757ef684 server_cookie=df0a9371e2856a3a in_seq=0 out_seq=0 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f985d600700 10 client.25334 ms_handle_connect on v2:172.21.15.194:6838/2085744336 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f985d600700 1 -- 172.21.15.86:0/2027835240 <== mds.3 v2:172.21.15.194:6838/2085744336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5578b9caf6d0 con 0x5578b9ddc4c0 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f985d600700 10 client.25334 handle_command_reply: tid=0 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 client.25334 shutdown 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 msgr2=0x5578b9d75810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 0x5578b9d75810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f985801c720 tx=0x7f9858009ad0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 shutdown_connections 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578b9d5f4f0 0x5578b9d75d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5578b9d55ef0 0x5578b9d752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.194:6838/2085744336,v1:172.21.15.194:6839/2085744336] conn(0x5578b9ddc4c0 0x5578b9ddb1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 --2- 172.21.15.86:0/2027835240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578b9d5b780 0x5578b9d75810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f985d600700 0 client.25334 ms_handle_reset on v2:172.21.15.194:6838/2085744336 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.578+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 >> 172.21.15.86:0/2027835240 conn(0x5578b9d4ce30 msgr2=0x5578b9d5c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 shutdown_connections 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9880ae1b80 1 -- 172.21.15.86:0/2027835240 wait complete. 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9880ae1b80 20 client.25334 trim_cache size 0 max 16384 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24493, 2024-04-25T19:03:37.585 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 268.11404025899998, 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 319.61530860099998 2024-04-25T19:03:37.586 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98540a3fe0 msgr2=0x7f98540a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9876200700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98540a3fe0 0x7f98540a64a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9858009680 tx=0x7f9858009260 comp rx=0 tx=0).stop 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 msgr2=0x7f987019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.579+0000 7f9876200700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f987019b2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9860005910 tx=0x7f986000b670 comp rx=0 tx=0).stop 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 shutdown_connections 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98540a3fe0 0x7f98540a64a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f987010f440 0x7f987019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9870074c70 0x7f987019b2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 --2- 172.21.15.86:0/3203271838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9870075590 0x7f987019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:37.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 >> 172.21.15.86:0/3203271838 conn(0x7f9870079ba0 msgr2=0x7f98700ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:37.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 shutdown_connections 2024-04-25T19:03:37.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:37.580+0000 7f9876200700 1 -- 172.21.15.86:0/3203271838 wait complete. 2024-04-25T19:03:38.637 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:38 smithi086 ceph-mon[27783]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:03:38.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:38 smithi183 ceph-mon[32344]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:03:38.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T19:03:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:38 smithi194 ceph-mon[32740]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:03:39.119 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 -- 172.21.15.86:0/2345899467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c10e6a0 msgr2=0x7f436c110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 --2- 172.21.15.86:0/2345899467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c10e6a0 0x7f436c110a90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4364009ca0 tx=0x7f436402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 -- 172.21.15.86:0/2345899467 shutdown_connections 2024-04-25T19:03:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 --2- 172.21.15.86:0/2345899467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c10e6a0 0x7f436c110a90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 --2- 172.21.15.86:0/2345899467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 0x7f436c10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 --2- 172.21.15.86:0/2345899467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c100af0 0x7f436c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 -- 172.21.15.86:0/2345899467 >> 172.21.15.86:0/2345899467 conn(0x7f436c0fc770 msgr2=0x7f436c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 -- 172.21.15.86:0/2345899467 shutdown_connections 2024-04-25T19:03:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.681+0000 7f436b200640 1 -- 172.21.15.86:0/2345899467 wait complete. 2024-04-25T19:03:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.683+0000 7f436b200640 1 Processor -- start 2024-04-25T19:03:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.683+0000 7f436b200640 1 -- start start 2024-04-25T19:03:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 0x7f436c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 0x7f436c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c10e6a0 0x7f436c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1141a0 con 0x7f436c1014a0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f436c114020 con 0x7f436c10e6a0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f436c114320 con 0x7f436c100af0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f4369e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 0x7f436c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f4369e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 0x7f436c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50538/0 (socket says 172.21.15.86:50538) 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f4369e00640 1 -- 172.21.15.86:0/2814179543 learned_addr learned my addr 172.21.15.86:0/2814179543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f436a800640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c10e6a0 0x7f436c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.684+0000 7f4369400640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 0x7f436c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369e00640 1 -- 172.21.15.86:0/2814179543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c10e6a0 msgr2=0x7f436c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369e00640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c10e6a0 0x7f436c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369e00640 1 -- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 msgr2=0x7f436c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369e00640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 0x7f436c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369e00640 1 -- 172.21.15.86:0/2814179543 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c19f710 con 0x7f436c100af0 2024-04-25T19:03:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369400640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 0x7f436c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4369e00640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 0x7f436c19a720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f435c00e9f0 tx=0x7f435c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4362a00640 1 -- 172.21.15.86:0/2814179543 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c00cd80 con 0x7f436c100af0 2024-04-25T19:03:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c19fa00 con 0x7f436c100af0 2024-04-25T19:03:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f4362a00640 1 -- 172.21.15.86:0/2814179543 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435c0108e0 con 0x7f436c100af0 2024-04-25T19:03:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.685+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c1a7290 con 0x7f436c100af0 2024-04-25T19:03:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.686+0000 7f4362a00640 1 -- 172.21.15.86:0/2814179543 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c018980 con 0x7f436c100af0 2024-04-25T19:03:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 Processor -- start 2024-04-25T19:03:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 -- start start 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993889b0 0x55b899388d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993892d0 0x55b899393650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899395f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8992ee900 con 0x55b899393b90 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b899294ef0 con 0x55b8993892d0 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.689+0000 7f4373bd2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b8992ea8c0 con 0x55b8993889b0 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f4369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899395f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f4369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899395f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34628/0 (socket says 172.21.15.86:34628) 2024-04-25T19:03:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f4369400640 1 -- 172.21.15.86:0/622388775 learned_addr learned my addr 172.21.15.86:0/622388775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f436a800640 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993889b0 0x55b899388d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f436a800640 1 -- 172.21.15.86:0/622388775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993892d0 msgr2=0x55b899393650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:03:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f436a800640 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993892d0 0x55b899393650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f436a800640 1 -- 172.21.15.86:0/622388775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 msgr2=0x55b899395f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f436a800640 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899395f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.690+0000 7f436a800640 1 -- 172.21.15.86:0/622388775 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b899001210 con 0x55b8993889b0 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.691+0000 7f436a800640 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993889b0 0x55b899388d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4364005e30 tx=0x7f4364005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=329b561f9df9bad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.691+0000 7f4357000640 1 -- 172.21.15.86:0/622388775 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364031cf0 con 0x55b8993889b0 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.691+0000 7f4357000640 1 -- 172.21.15.86:0/622388775 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4364036030 con 0x55b8993889b0 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.691+0000 7f4357000640 1 -- 172.21.15.86:0/622388775 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364004040 con 0x55b8993889b0 2024-04-25T19:03:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.691+0000 7f4362a00640 1 -- 172.21.15.86:0/2814179543 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f435c010430 con 0x7f436c100af0 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 -- 172.21.15.86:0/622388775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993889b0 msgr2=0x55b899388d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993889b0 0x55b899388d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4364005e30 tx=0x7f4364005ef0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 -- 172.21.15.86:0/622388775 shutdown_connections 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899395f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993892d0 0x55b899393650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 --2- 172.21.15.86:0/622388775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993889b0 0x55b899388d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 -- 172.21.15.86:0/622388775 >> 172.21.15.86:0/622388775 conn(0x55b899384ec0 msgr2=0x55b8993852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 -- 172.21.15.86:0/622388775 shutdown_connections 2024-04-25T19:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.692+0000 7f4373bd2740 1 -- 172.21.15.86:0/622388775 wait complete. 2024-04-25T19:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 -- start start 2024-04-25T19:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 Processor -- start 2024-04-25T19:03:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899390b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993892d0 0x55b8993910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899391600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8993965e0 con 0x55b899393b90 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8992a8310 con 0x55b8993889b0 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b899396760 con 0x55b8993892d0 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.693+0000 7f4373bd2740 10 client.0 _ll_register_callbacks cb 0x55b899276cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4362a00640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43500a3dd0 0x7f43500a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899391600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899391600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34638/0 (socket says 172.21.15.86:34638) 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 -- 172.21.15.86:0/2857903689 learned_addr learned my addr 172.21.15.86:0/2857903689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f436a800640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899390b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4362a00640 1 -- 172.21.15.86:0/2814179543 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f435c0d0ec0 con 0x7f436c100af0 2024-04-25T19:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f436a800640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43500a3dd0 0x7f43500a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 -- 172.21.15.86:0/2857903689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993892d0 msgr2=0x55b8993910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369e00640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993892d0 0x55b8993910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993892d0 0x55b8993910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 -- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 msgr2=0x55b899390b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899390b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369400640 1 -- 172.21.15.86:0/2857903689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8990cd620 con 0x55b899393b90 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.694+0000 7f4369e00640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993892d0 0x55b8993910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.695+0000 7f4369400640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899391600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f43580126b0 tx=0x7f4358012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa7aab220d329b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.695+0000 7f436a800640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43500a3dd0 0x7f43500a6290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4364009a60 tx=0x7f436402a8c0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.695+0000 7f434b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.695+0000 7f434b400640 1 -- 172.21.15.86:0/2857903689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358011bb0 con 0x55b899393b90 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.695+0000 7f434b400640 1 -- 172.21.15.86:0/2857903689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f4358011d50 con 0x55b899393b90 2024-04-25T19:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.695+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4364036030 con 0x55b899393b90 2024-04-25T19:03:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.696+0000 7f434b400640 1 -- 172.21.15.86:0/2857903689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4358004020 con 0x55b899393b90 2024-04-25T19:03:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.696+0000 7f4373bd2740 10 client.15219 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.696+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b8993a2900 con 0x55b899393b90 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.696+0000 7f434b400640 1 -- 172.21.15.86:0/2857903689 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4364036030 con 0x55b899393b90 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.696+0000 7f434b400640 1 -- 172.21.15.86:0/2857903689 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f435801c470 con 0x55b899393b90 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.696+0000 7f4373bd2740 10 client.15219 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.697+0000 7f4373bd2740 10 client.15219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.697+0000 7f4373bd2740 20 client.15219 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.697+0000 7f4373bd2740 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b8993a3dc0 0x55b8993a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.697+0000 7f4373bd2740 4 client.15219 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.697+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8993028b0 con 0x55b8993a3dc0 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.697+0000 7f4369e00640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b8993a3dc0 0x55b8993a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.698+0000 7f434b400640 10 client.15219 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.698+0000 7f4369e00640 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b8993a3dc0 0x55b8993a61d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d597582a3120f3 server_cookie=b2c2eb956ae85e45 in_seq=0 out_seq=0 2024-04-25T19:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f434b400640 1 -- 172.21.15.86:0/2857903689 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8993028b0 con 0x55b8993a3dc0 2024-04-25T19:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f434b400640 10 client.15219 handle_command_reply: tid=0 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 client.15219 shutdown 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 msgr2=0x55b899391600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899391600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f43580126b0 tx=0x7f4358012b70 comp rx=0 tx=0).stop 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 shutdown_connections 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b8993a3dc0 0x55b8993a61d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b899393b90 0x55b899391600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8993892d0 0x55b8993910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 --2- 172.21.15.86:0/2857903689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899390b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 >> 172.21.15.86:0/2857903689 conn(0x55b899384ec0 msgr2=0x55b89938abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.699+0000 7f434b400640 0 client.15219 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.700+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 shutdown_connections 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.700+0000 7f4373bd2740 1 -- 172.21.15.86:0/2857903689 wait complete. 2024-04-25T19:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.700+0000 7f4373bd2740 20 client.15219 trim_cache size 0 max 16384 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 Processor -- start 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 -- start start 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b8993847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899391420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899392ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1afd10 con 0x55b8993892d0 2024-04-25T19:03:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1b1850 con 0x55b8993889b0 2024-04-25T19:03:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.716+0000 7f4373bd2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1b03e0 con 0x55b899393b90 2024-04-25T19:03:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b8993847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899391420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b8993847a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43126/0 (socket says 172.21.15.86:43126) 2024-04-25T19:03:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 -- 172.21.15.86:0/3092337336 learned_addr learned my addr 172.21.15.86:0/3092337336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f436a800640 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899392ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 -- 172.21.15.86:0/3092337336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 msgr2=0x55b899392ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899392ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 -- 172.21.15.86:0/3092337336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 msgr2=0x55b899391420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899391420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369e00640 1 -- 172.21.15.86:0/3092337336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8993961b0 con 0x55b8993889b0 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.717+0000 7f4369400640 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899391420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.718+0000 7f4369e00640 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b8993847a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f435c0995a0 tx=0x7f435c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8fef9707368ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.718+0000 7f4356600640 1 -- 172.21.15.86:0/3092337336 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0a5840 con 0x55b8993889b0 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.718+0000 7f4356600640 1 -- 172.21.15.86:0/3092337336 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b8993028b0 con 0x55b8993889b0 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.718+0000 7f4356600640 1 -- 172.21.15.86:0/3092337336 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0bd250 con 0x55b8993889b0 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 -- 172.21.15.86:0/3092337336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 msgr2=0x55b8993847a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b8993847a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f435c0995a0 tx=0x7f435c01f000 comp rx=0 tx=0).stop 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 -- 172.21.15.86:0/3092337336 shutdown_connections 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899392ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899391420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3092337336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b8993847a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 -- 172.21.15.86:0/3092337336 >> 172.21.15.86:0/3092337336 conn(0x55b899384ec0 msgr2=0x55b8993923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 -- 172.21.15.86:0/3092337336 shutdown_connections 2024-04-25T19:03:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 -- 172.21.15.86:0/3092337336 wait complete. 2024-04-25T19:03:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 -- start start 2024-04-25T19:03:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.719+0000 7f4373bd2740 1 Processor -- start 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899399340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899399880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899399dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1b1180 con 0x55b8993892d0 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1b0ab0 con 0x55b8993889b0 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1adcf0 con 0x55b899393b90 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4373bd2740 10 client.0 _ll_register_callbacks cb 0x7f4358009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4369e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899399340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4369400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899399880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4369e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899399340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43138/0 (socket says 172.21.15.86:43138) 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f4369e00640 1 -- 172.21.15.86:0/3897196359 learned_addr learned my addr 172.21.15.86:0/3897196359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.720+0000 7f436a800640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899399dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369400640 1 -- 172.21.15.86:0/3897196359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 msgr2=0x55b899399dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369400640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899399dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369400640 1 -- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 msgr2=0x55b899399340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369400640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899399340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369400640 1 -- 172.21.15.86:0/3897196359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b899409360 con 0x55b8993892d0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369e00640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899399340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4369400640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899399880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4358001eb0 tx=0x7f435801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e7f7a53ec79d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f434b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f434b400640 1 -- 172.21.15.86:0/3897196359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358030020 con 0x55b8993892d0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f434b400640 1 -- 172.21.15.86:0/3897196359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f43580043d0 con 0x55b8993892d0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.721+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b8993028b0 con 0x55b8993892d0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f434b400640 1 -- 172.21.15.86:0/3897196359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4358011bb0 con 0x55b8993892d0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 10 client.15225 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b89940a5b0 con 0x55b8993892d0 2024-04-25T19:03:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f434b400640 1 -- 172.21.15.86:0/3897196359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b8993028b0 con 0x55b8993892d0 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f434b400640 1 -- 172.21.15.86:0/3897196359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f43580046f0 con 0x55b8993892d0 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 10 client.15225 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 10 client.15225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 20 client.15225 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b89940ad50 0x55b89940d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 4 client.15225 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.722+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8992fc320 con 0x55b89940ad50 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.723+0000 7f4369e00640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b89940ad50 0x55b89940d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.723+0000 7f4369e00640 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b89940ad50 0x55b89940d170 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f8efeaa8bdc334 server_cookie=7f520bdfc9ccd998 in_seq=0 out_seq=0 2024-04-25T19:03:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.723+0000 7f434b400640 10 client.15225 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.732+0000 7f434b400640 1 -- 172.21.15.86:0/3897196359 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8992fc320 con 0x55b89940ad50 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.732+0000 7f434b400640 10 client.15225 handle_command_reply: tid=0 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.732+0000 7f4373bd2740 1 client.15225 shutdown 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 msgr2=0x55b899399880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899399880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4358001eb0 tx=0x7f435801c6a0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 shutdown_connections 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b89940ad50 0x55b89940d170 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b899393b90 0x55b899399dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8993892d0 0x55b899399880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 --2- 172.21.15.86:0/3897196359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8993889b0 0x55b899399340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f434b400640 0 client.15225 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 >> 172.21.15.86:0/3897196359 conn(0x55b899384ec0 msgr2=0x55b899392c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 shutdown_connections 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 1 -- 172.21.15.86:0/3897196359 wait complete. 2024-04-25T19:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.733+0000 7f4373bd2740 20 client.15225 trim_cache size 0 max 16384 2024-04-25T19:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43500a3dd0 msgr2=0x7f43500a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43500a3dd0 0x7f43500a6290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4364009a60 tx=0x7f436402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 msgr2=0x7f436c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 0x7f436c19a720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f435c00e9f0 tx=0x7f435c00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 shutdown_connections 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43500a3dd0 0x7f43500a6290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f436c10e6a0 0x7f436c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f436c1014a0 0x7f436c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 --2- 172.21.15.86:0/2814179543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c100af0 0x7f436c19a720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 >> 172.21.15.86:0/2814179543 conn(0x7f436c0fc770 msgr2=0x7f436c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 shutdown_connections 2024-04-25T19:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:39.734+0000 7f436b200640 1 -- 172.21.15.86:0/2814179543 wait complete. 2024-04-25T19:03:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:40 smithi086 ceph-mon[27783]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-25T19:03:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:40 smithi183 ceph-mon[32344]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-25T19:03:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:40 smithi194 ceph-mon[32740]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 -- 172.21.15.86:0/73666428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4102960 msgr2=0x7fa5f4102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/73666428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4102960 0x7fa5f4102de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009990 tx=0x7fa5dc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 -- 172.21.15.86:0/73666428 shutdown_connections 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/73666428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4103320 0x7fa5f410f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/73666428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4108960 0x7fa5f4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/73666428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4102960 0x7fa5f4102de0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 -- 172.21.15.86:0/73666428 >> 172.21.15.86:0/73666428 conn(0x7fa5f40fe490 msgr2=0x7fa5f41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 -- 172.21.15.86:0/73666428 shutdown_connections 2024-04-25T19:03:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.414+0000 7fa5f9e00700 1 -- 172.21.15.86:0/73666428 wait complete. 2024-04-25T19:03:41.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.415+0000 7fa5f9e00700 1 Processor -- start 2024-04-25T19:03:41.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.415+0000 7fa5f9e00700 1 -- start start 2024-04-25T19:03:41.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f9e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 0x7fa5f419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4103320 0x7fa5f419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4108960 0x7fa5f419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f9e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f4112bf0 con 0x7fa5f4103320 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f9e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f4112a70 con 0x7fa5f4108960 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f9e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f4112d70 con 0x7fa5f4102960 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 0x7fa5f419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4103320 0x7fa5f419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 0x7fa5f419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51882/0 (socket says 172.21.15.86:51882) 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.416+0000 7fa5f8a00700 1 -- 172.21.15.86:0/662051813 learned_addr learned my addr 172.21.15.86:0/662051813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f9400700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4108960 0x7fa5f419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f8a00700 1 -- 172.21.15.86:0/662051813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4108960 msgr2=0x7fa5f419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4108960 0x7fa5f419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f8a00700 1 -- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4103320 msgr2=0x7fa5f419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4103320 0x7fa5f419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f8a00700 1 -- 172.21.15.86:0/662051813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f41a0270 con 0x7fa5f4102960 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f9400700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4108960 0x7fa5f419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:41.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4103320 0x7fa5f419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.417+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 0x7fa5f419b260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e9a0 tx=0x7fa5e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.418+0000 7fa5f2000700 1 -- 172.21.15.86:0/662051813 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cbd0 con 0x7fa5f4102960 2024-04-25T19:03:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.418+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f41a0560 con 0x7fa5f4102960 2024-04-25T19:03:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.418+0000 7fa5f2000700 1 -- 172.21.15.86:0/662051813 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e400cd70 con 0x7fa5f4102960 2024-04-25T19:03:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.418+0000 7fa5f2000700 1 -- 172.21.15.86:0/662051813 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4018680 con 0x7fa5f4102960 2024-04-25T19:03:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.418+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f41a7de0 con 0x7fa5f4102960 2024-04-25T19:03:41.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.422+0000 7fa5f2000700 1 -- 172.21.15.86:0/662051813 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa5e4010430 con 0x7fa5f4102960 2024-04-25T19:03:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.423+0000 7fa605e10b80 1 Processor -- start 2024-04-25T19:03:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.423+0000 7fa605e10b80 1 -- start start 2024-04-25T19:03:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.423+0000 7fa5f2000700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5d40a3f60 0x7fa5d40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.423+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975b8b0 0x55eaf975bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.423+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 0x55eaf9766b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf9767080 0x55eaf9769470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa605e10b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eaf96aee40 con 0x55eaf9767080 2024-04-25T19:03:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa605e10b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eaf96556e0 con 0x55eaf975b8b0 2024-04-25T19:03:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa605e10b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf96aae00 con 0x55eaf975c1d0 2024-04-25T19:03:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa5f2000700 1 -- 172.21.15.86:0/662051813 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fa5e40d0d60 con 0x7fa5f4102960 2024-04-25T19:03:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa5f9400700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5d40a3f60 0x7fa5d40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa5ebe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 0x55eaf9766b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa5ebe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 0x55eaf9766b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51896/0 (socket says 172.21.15.86:51896) 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa5ebe00700 1 -- 172.21.15.86:0/2641505834 learned_addr learned my addr 172.21.15.86:0/2641505834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.424+0000 7fa5f9400700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf9767080 0x55eaf9769470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975b8b0 0x55eaf975bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5ebe00700 1 -- 172.21.15.86:0/2641505834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975b8b0 msgr2=0x55eaf975bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975b8b0 0x55eaf975bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5ebe00700 1 -- 172.21.15.86:0/2641505834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf9767080 msgr2=0x55eaf9769470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf9767080 0x55eaf9769470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5ebe00700 1 -- 172.21.15.86:0/2641505834 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf964b630 con 0x55eaf975c1d0 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5f9400700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf9767080 0x55eaf9769470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975b8b0 0x55eaf975bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.425+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 0x55eaf9766b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc015000 tx=0x7fa5dc005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa57337efcab0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.426+0000 7fa5f9400700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5d40a3f60 0x7fa5d40a6420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0078b0 tx=0x7fa5ec009140 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.426+0000 7fa5ea000700 1 -- 172.21.15.86:0/2641505834 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc027070 con 0x55eaf975c1d0 2024-04-25T19:03:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.426+0000 7fa5ea000700 1 -- 172.21.15.86:0/2641505834 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5dc003680 con 0x55eaf975c1d0 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.426+0000 7fa5ea000700 1 -- 172.21.15.86:0/2641505834 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00f680 con 0x55eaf975c1d0 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.426+0000 7fa605e10b80 1 -- 172.21.15.86:0/2641505834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 msgr2=0x55eaf9766b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.426+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 0x55eaf9766b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc015000 tx=0x7fa5dc005f00 comp rx=0 tx=0).stop 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 -- 172.21.15.86:0/2641505834 shutdown_connections 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf9767080 0x55eaf9769470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975b8b0 0x55eaf975bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2641505834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975c1d0 0x55eaf9766b40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 -- 172.21.15.86:0/2641505834 >> 172.21.15.86:0/2641505834 conn(0x55eaf968af00 msgr2=0x55eaf9757d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 -- 172.21.15.86:0/2641505834 shutdown_connections 2024-04-25T19:03:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.427+0000 7fa605e10b80 1 -- 172.21.15.86:0/2641505834 wait complete. 2024-04-25T19:03:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 -- start start 2024-04-25T19:03:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 Processor -- start 2024-04-25T19:03:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf97667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf9763ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf9764000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9769ba0 con 0x55eaf975b8b0 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9668810 con 0x55eaf975c1d0 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.428+0000 7fa605e10b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9769d20 con 0x55eaf9767080 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa605e10b80 10 client.0 _ll_register_callbacks cb 0x55eaf96476d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf97667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf97667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36634/0 (socket says 172.21.15.86:36634) 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf9763ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 -- 172.21.15.86:0/2675760389 learned_addr learned my addr 172.21.15.86:0/2675760389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f9400700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf9764000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 -- 172.21.15.86:0/2675760389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 msgr2=0x55eaf9764000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf9764000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 -- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 msgr2=0x55eaf9763ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf9763ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.429+0000 7fa5f8a00700 1 -- 172.21.15.86:0/2675760389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf95fc880 con 0x55eaf975b8b0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf97667c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e970 tx=0x7fa5e409cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fabc4980d5646f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa5e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa5e2a00700 1 -- 172.21.15.86:0/2675760389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4010920 con 0x55eaf975b8b0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa5e2a00700 1 -- 172.21.15.86:0/2675760389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fa5e409bec0 con 0x55eaf975b8b0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5dc003680 con 0x55eaf975b8b0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa5e2a00700 1 -- 172.21.15.86:0/2675760389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5e40bd0b0 con 0x55eaf975b8b0 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa605e10b80 10 client.15237 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.430+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eaf9645aa0 con 0x55eaf975b8b0 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa5e2a00700 1 -- 172.21.15.86:0/2675760389 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5dc003680 con 0x55eaf975b8b0 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa5e2a00700 1 -- 172.21.15.86:0/2675760389 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa5e40987c0 con 0x55eaf975b8b0 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa605e10b80 10 client.15237 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa605e10b80 10 client.15237 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa605e10b80 20 client.15237 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97766e0 0x55eaf9777b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa605e10b80 4 client.15237 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:41.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.431+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 --> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaf96bc630 con 0x55eaf97766e0 2024-04-25T19:03:41.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.432+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97766e0 0x55eaf9777b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.432+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97766e0 0x55eaf9777b50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f33027dff0583aa server_cookie=99c04fd15a71b050 in_seq=0 out_seq=0 2024-04-25T19:03:41.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.432+0000 7fa5e2a00700 10 client.15237 ms_handle_connect on v2:172.21.15.183:6836/3273542947 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.433+0000 7fa5e2a00700 1 -- 172.21.15.86:0/2675760389 <== mds.2 v2:172.21.15.183:6836/3273542947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eaf96bc630 con 0x55eaf97766e0 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.433+0000 7fa5e2a00700 10 client.15237 handle_command_reply: tid=0 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.433+0000 7fa605e10b80 1 client.15237 shutdown 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.433+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 msgr2=0x55eaf97667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.433+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf97667c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e970 tx=0x7fa5e409cfb0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 shutdown_connections 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97766e0 0x55eaf9777b50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf9764000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa5e2a00700 0 client.15237 ms_handle_reset on v2:172.21.15.183:6836/3273542947 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf97667c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 --2- 172.21.15.86:0/2675760389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf9763ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 >> 172.21.15.86:0/2675760389 conn(0x55eaf968af00 msgr2=0x55eaf9757920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 shutdown_connections 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 1 -- 172.21.15.86:0/2675760389 wait complete. 2024-04-25T19:03:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.434+0000 7fa605e10b80 20 client.15237 trim_cache size 0 max 16384 2024-04-25T19:03:41.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.456+0000 7fa605e10b80 1 Processor -- start 2024-04-25T19:03:41.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 -- start start 2024-04-25T19:03:41.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 0x55eaf9776010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975c1d0 0x55eaf9760da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf9767080 0x55eaf97612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9775e90 con 0x55eaf975c1d0 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9776560 con 0x55eaf9767080 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa605e10b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9776c30 con 0x55eaf975b8b0 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa5ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975c1d0 0x55eaf9760da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 0x55eaf9776010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 0x55eaf9776010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51912/0 (socket says 172.21.15.86:51912) 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa5f8a00700 1 -- 172.21.15.86:0/3822066875 learned_addr learned my addr 172.21.15.86:0/3822066875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:41.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.457+0000 7fa5f9400700 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf9767080 0x55eaf97612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f8a00700 1 -- 172.21.15.86:0/3822066875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf9767080 msgr2=0x55eaf97612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf9767080 0x55eaf97612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f8a00700 1 -- 172.21.15.86:0/3822066875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975c1d0 msgr2=0x55eaf9760da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975c1d0 0x55eaf9760da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f8a00700 1 -- 172.21.15.86:0/3822066875 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf9762070 con 0x55eaf975b8b0 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f9400700 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf9767080 0x55eaf97612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 0x55eaf9776010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4098ed0 tx=0x7fa5e409c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df3a30834dc9576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5e9600700 1 -- 172.21.15.86:0/3822066875 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40a56e0 con 0x55eaf975b8b0 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5e9600700 1 -- 172.21.15.86:0/3822066875 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e409ca70 con 0x55eaf975b8b0 2024-04-25T19:03:41.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.458+0000 7fa5e9600700 1 -- 172.21.15.86:0/3822066875 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40873e0 con 0x55eaf975b8b0 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.459+0000 7fa605e10b80 1 -- 172.21.15.86:0/3822066875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 msgr2=0x55eaf9776010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.459+0000 7fa605e10b80 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 0x55eaf9776010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4098ed0 tx=0x7fa5e409c500 comp rx=0 tx=0).stop 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 -- 172.21.15.86:0/3822066875 shutdown_connections 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf9767080 0x55eaf97612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf975b8b0 0x55eaf9776010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 --2- 172.21.15.86:0/3822066875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975c1d0 0x55eaf9760da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 -- 172.21.15.86:0/3822066875 >> 172.21.15.86:0/3822066875 conn(0x55eaf968af00 msgr2=0x55eaf97638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 -- 172.21.15.86:0/3822066875 shutdown_connections 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 -- 172.21.15.86:0/3822066875 wait complete. 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 -- start start 2024-04-25T19:03:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.460+0000 7fa605e10b80 1 Processor -- start 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf9781350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf977d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf977dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eaf97779d0 con 0x55eaf975b8b0 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9777300 con 0x55eaf975c1d0 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf97780a0 con 0x55eaf9767080 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa605e10b80 10 client.0 _ll_register_callbacks cb 0x55eaf96476d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.461+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf9781350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf9781350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36654/0 (socket says 172.21.15.86:36654) 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 -- 172.21.15.86:0/4202334048 learned_addr learned my addr 172.21.15.86:0/4202334048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f9400700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf977dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf977d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 -- 172.21.15.86:0/4202334048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 msgr2=0x55eaf977dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf977dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 -- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 msgr2=0x55eaf977d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf977d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 -- 172.21.15.86:0/4202334048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf977e430 con 0x55eaf975b8b0 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5f8a00700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf9781350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4099440 tx=0x7fa5e4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528911b07fadc9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.462+0000 7fa5e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:41.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.463+0000 7fa5e2a00700 1 -- 172.21.15.86:0/4202334048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40a56e0 con 0x55eaf975b8b0 2024-04-25T19:03:41.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.463+0000 7fa5e2a00700 1 -- 172.21.15.86:0/4202334048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fa5e409b930 con 0x55eaf975b8b0 2024-04-25T19:03:41.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.463+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e409ca70 con 0x55eaf975b8b0 2024-04-25T19:03:41.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.463+0000 7fa5e2a00700 1 -- 172.21.15.86:0/4202334048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5e409bb70 con 0x55eaf975b8b0 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.463+0000 7fa605e10b80 10 client.15243 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.463+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eaf97de860 con 0x55eaf975b8b0 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa5e2a00700 1 -- 172.21.15.86:0/4202334048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e409ca70 con 0x55eaf975b8b0 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa5e2a00700 1 -- 172.21.15.86:0/4202334048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa5e40672c0 con 0x55eaf975b8b0 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa605e10b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa605e10b80 10 client.15243 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa605e10b80 20 client.15243 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa605e10b80 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97e2f00 0x55eaf97e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa605e10b80 4 client.15243 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.464+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 --> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] -- command(tid 0: {"prefix": "status"}) -- 0x55eaf97644c0 con 0x55eaf97e2f00 2024-04-25T19:03:41.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.465+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97e2f00 0x55eaf97e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:41.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.465+0000 7fa5ebe00700 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97e2f00 0x55eaf97e1c10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf3586b6296b2594 server_cookie=ef16a3f7b276d36c in_seq=0 out_seq=0 2024-04-25T19:03:41.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.465+0000 7fa5e2a00700 10 client.15243 ms_handle_connect on v2:172.21.15.183:6836/3273542947 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa5e2a00700 1 -- 172.21.15.86:0/4202334048 <== mds.2 v2:172.21.15.183:6836/3273542947 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eaf97644c0 con 0x55eaf97e2f00 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa5e2a00700 10 client.15243 handle_command_reply: tid=0 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 client.15243 shutdown 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 msgr2=0x55eaf9781350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf9781350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4099440 tx=0x7fa5e4009620 comp rx=0 tx=0).stop 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 shutdown_connections 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:6836/3273542947,v1:172.21.15.183:6837/3273542947] conn(0x55eaf97e2f00 0x55eaf97e1c10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf9767080 0x55eaf977dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eaf975b8b0 0x55eaf9781350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa5e2a00700 0 client.15243 ms_handle_reset on v2:172.21.15.183:6836/3273542947 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 --2- 172.21.15.86:0/4202334048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eaf975c1d0 0x55eaf977d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 >> 172.21.15.86:0/4202334048 conn(0x55eaf968af00 msgr2=0x55eaf97652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 shutdown_connections 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 1 -- 172.21.15.86:0/4202334048 wait complete. 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.466+0000 7fa605e10b80 20 client.15243 trim_cache size 0 max 16384 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24494, 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:03:41.471 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 273.01945799399999, 2024-04-25T19:03:41.472 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-25T19:03:41.472 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-25T19:03:41.472 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T19:03:41.472 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 326.61975801800003 2024-04-25T19:03:41.472 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:03:41.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5d40a3f60 msgr2=0x7fa5d40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5d40a3f60 0x7fa5d40a6420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0078b0 tx=0x7fa5ec009140 comp rx=0 tx=0).stop 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 msgr2=0x7fa5f419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 0x7fa5f419b260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e9a0 tx=0x7fa5e400ecb0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 shutdown_connections 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f4108960 0x7fa5f419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5d40a3f60 0x7fa5d40a6420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4102960 0x7fa5f419b260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 --2- 172.21.15.86:0/662051813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f4103320 0x7fa5f419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 >> 172.21.15.86:0/662051813 conn(0x7fa5f40fe490 msgr2=0x7fa5f40750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 shutdown_connections 2024-04-25T19:03:41.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:41.467+0000 7fa5f9e00700 1 -- 172.21.15.86:0/662051813 wait complete. 2024-04-25T19:03:42.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.g status 2024-04-25T19:03:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:42 smithi086 ceph-mon[27783]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T19:03:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:42 smithi183 ceph-mon[32344]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T19:03:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:42 smithi194 ceph-mon[32740]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T19:03:42.814 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:44.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:44 smithi183 ceph-mon[32344]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-25T19:03:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:44 smithi194 ceph-mon[32740]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-25T19:03:44.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:44 smithi086 ceph-mon[27783]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-04-25T19:03:46.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:46 smithi183 ceph-mon[32344]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T19:03:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:46 smithi194 ceph-mon[32740]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T19:03:46.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:46 smithi086 ceph-mon[27783]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 -- 172.21.15.86:0/118319315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 msgr2=0x7fc204108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 --2- 172.21.15.86:0/118319315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc204108d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009990 tx=0x7fc1f4009ca0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 -- 172.21.15.86:0/118319315 shutdown_connections 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 --2- 172.21.15.86:0/118319315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204103330 0x7fc20410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 --2- 172.21.15.86:0/118319315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc204108d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 --2- 172.21.15.86:0/118319315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc204102970 0x7fc204102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 -- 172.21.15.86:0/118319315 >> 172.21.15.86:0/118319315 conn(0x7fc2040fe4c0 msgr2=0x7fc2041008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 -- 172.21.15.86:0/118319315 shutdown_connections 2024-04-25T19:03:48.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.410+0000 7fc208c00700 1 -- 172.21.15.86:0/118319315 wait complete. 2024-04-25T19:03:48.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.411+0000 7fc208c00700 1 Processor -- start 2024-04-25T19:03:48.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.411+0000 7fc208c00700 1 -- start start 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc208c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204102970 0x7fc2041990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc208c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc204103330 0x7fc2041995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc208c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc20419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc208c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc204112bd0 con 0x7fc204108970 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc208c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc204112a50 con 0x7fc204102970 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc208c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc204112d50 con 0x7fc204103330 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc203e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc20419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc203e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc20419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36674/0 (socket says 172.21.15.86:36674) 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc203e00700 1 -- 172.21.15.86:0/1036522060 learned_addr learned my addr 172.21.15.86:0/1036522060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc203400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204102970 0x7fc2041990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.412+0000 7fc202a00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc204103330 0x7fc2041995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203400700 1 -- 172.21.15.86:0/1036522060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc204103330 msgr2=0x7fc2041995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203400700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc204103330 0x7fc2041995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203400700 1 -- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 msgr2=0x7fc20419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203400700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc20419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203400700 1 -- 172.21.15.86:0/1036522060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20419e0b0 con 0x7fc204102970 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203e00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc20419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc203400700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204102970 0x7fc2041990a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40098e0 tx=0x7fc1f40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc1fbe00700 1 -- 172.21.15.86:0/1036522060 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4027070 con 0x7fc204102970 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20419e340 con 0x7fc204102970 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc1fbe00700 1 -- 172.21.15.86:0/1036522060 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4022470 con 0x7fc204102970 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.413+0000 7fc1fbe00700 1 -- 172.21.15.86:0/1036522060 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4017650 con 0x7fc204102970 2024-04-25T19:03:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.414+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2041a5c20 con 0x7fc204102970 2024-04-25T19:03:48.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.416+0000 7fc1fbe00700 1 -- 172.21.15.86:0/1036522060 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fc1f402a070 con 0x7fc204102970 2024-04-25T19:03:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.419+0000 7fc1fbe00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e80a3fe0 0x7fc1e80a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.419+0000 7fc214b9eb80 1 Processor -- start 2024-04-25T19:03:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.419+0000 7fc214b9eb80 1 -- start start 2024-04-25T19:03:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.419+0000 7fc1fbe00700 1 -- 172.21.15.86:0/1036522060 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fc1f40d1770 con 0x7fc204102970 2024-04-25T19:03:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.419+0000 7fc202a00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e80a3fe0 0x7fc1e80a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bfe8b0 0x55ad47bfec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c10880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad47b51e40 con 0x55ad47c0e490 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad47af86e0 con 0x55ad47bfe8b0 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ad47b4de00 con 0x55ad47bff1d0 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc202a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc203400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bfe8b0 0x55ad47bfec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc202a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51940/0 (socket says 172.21.15.86:51940) 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc202a00700 1 -- 172.21.15.86:0/3390683123 learned_addr learned my addr 172.21.15.86:0/3390683123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc202a00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e80a3fe0 0x7fc1e80a64a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc204100b20 tx=0x7fc1f0009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.420+0000 7fc203e00700 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c10880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc202a00700 1 -- 172.21.15.86:0/3390683123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bfe8b0 msgr2=0x55ad47bfec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc202a00700 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bfe8b0 0x55ad47bfec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc202a00700 1 -- 172.21.15.86:0/3390683123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 msgr2=0x55ad47c10880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc202a00700 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c10880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc202a00700 1 -- 172.21.15.86:0/3390683123 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad47aee630 con 0x55ad47bff1d0 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc203e00700 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc202a00700 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0df50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0012d50 tx=0x7fc1f0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56fd469c4059de91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc1f8c00700 1 -- 172.21.15.86:0/3390683123 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001cbb0 con 0x55ad47bff1d0 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc1f8c00700 1 -- 172.21.15.86:0/3390683123 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f00044e0 con 0x55ad47bff1d0 2024-04-25T19:03:48.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.421+0000 7fc1f8c00700 1 -- 172.21.15.86:0/3390683123 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001b460 con 0x55ad47bff1d0 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 -- 172.21.15.86:0/3390683123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 msgr2=0x55ad47c0df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0df50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0012d50 tx=0x7fc1f0019000 comp rx=0 tx=0).stop 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 -- 172.21.15.86:0/3390683123 shutdown_connections 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bfe8b0 0x55ad47bfec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/3390683123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0df50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.422+0000 7fc214b9eb80 1 -- 172.21.15.86:0/3390683123 >> 172.21.15.86:0/3390683123 conn(0x55ad47b2df00 msgr2=0x55ad47bfad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.423+0000 7fc214b9eb80 1 -- 172.21.15.86:0/3390683123 shutdown_connections 2024-04-25T19:03:48.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.423+0000 7fc214b9eb80 1 -- 172.21.15.86:0/3390683123 wait complete. 2024-04-25T19:03:48.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.423+0000 7fc214b9eb80 1 -- start start 2024-04-25T19:03:48.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.423+0000 7fc214b9eb80 1 Processor -- start 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 0x55ad47c0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47c0e490 0x55ad47c0b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c11010 con 0x55ad47bfe8b0 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad47b0b810 con 0x55ad47c0e490 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c11190 con 0x55ad47bff1d0 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc214b9eb80 10 client.0 _ll_register_callbacks cb 0x55ad47aea6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc203400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 0x55ad47c0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc203400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 0x55ad47c0dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36682/0 (socket says 172.21.15.86:36682) 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc203400700 1 -- 172.21.15.86:0/4175143207 learned_addr learned my addr 172.21.15.86:0/4175143207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.424+0000 7fc202a00700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203e00700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47c0e490 0x55ad47c0b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203400700 1 -- 172.21.15.86:0/4175143207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 msgr2=0x55ad47c0aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203400700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203400700 1 -- 172.21.15.86:0/4175143207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47c0e490 msgr2=0x55ad47c0b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203400700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47c0e490 0x55ad47c0b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203400700 1 -- 172.21.15.86:0/4175143207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad47a9f880 con 0x55ad47bfe8b0 2024-04-25T19:03:48.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203e00700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47c0e490 0x55ad47c0b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc203400700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 0x55ad47c0dbe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000c00 tx=0x7fc1f4022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4561c37917a663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.425+0000 7fc1ed600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc1ed600700 1 -- 172.21.15.86:0/4175143207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40052b0 con 0x55ad47bfe8b0 2024-04-25T19:03:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc1ed600700 1 -- 172.21.15.86:0/4175143207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fc1f409d890 con 0x55ad47bfe8b0 2024-04-25T19:03:48.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f00044e0 con 0x55ad47bfe8b0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc1ed600700 1 -- 172.21.15.86:0/4175143207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1f409d630 con 0x55ad47bfe8b0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc214b9eb80 10 client.15255 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ad47ae8aa0 con 0x55ad47bfe8b0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc1ed600700 1 -- 172.21.15.86:0/4175143207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f00044e0 con 0x55ad47bfe8b0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.426+0000 7fc1ed600700 1 -- 172.21.15.86:0/4175143207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc1f40bc350 con 0x55ad47bfe8b0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc214b9eb80 10 client.15255 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc214b9eb80 10 client.15255 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc214b9eb80 20 client.15255 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c1dca0 0x55ad47c1f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc214b9eb80 4 client.15255 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 --> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad47b5f630 con 0x55ad47c1dca0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.427+0000 7fc202a00700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c1dca0 0x55ad47c1f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.428+0000 7fc202a00700 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c1dca0 0x55ad47c1f110 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34c534d42b0a7ab5 server_cookie=18dbccaea94712bd in_seq=0 out_seq=0 2024-04-25T19:03:48.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.428+0000 7fc1ed600700 10 client.15255 ms_handle_connect on v2:172.21.15.86:6838/3676043902 2024-04-25T19:03:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.428+0000 7fc1ed600700 1 -- 172.21.15.86:0/4175143207 <== mds.1 v2:172.21.15.86:6838/3676043902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad47b5f630 con 0x55ad47c1dca0 2024-04-25T19:03:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.428+0000 7fc1ed600700 10 client.15255 handle_command_reply: tid=0 2024-04-25T19:03:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 client.15255 shutdown 2024-04-25T19:03:48.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 msgr2=0x55ad47c0dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 0x55ad47c0dbe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000c00 tx=0x7fc1f4022f90 comp rx=0 tx=0).stop 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 shutdown_connections 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c1dca0 0x55ad47c1f110 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47c0e490 0x55ad47c0b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47bfe8b0 0x55ad47c0dbe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4175143207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bff1d0 0x55ad47c0aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 >> 172.21.15.86:0/4175143207 conn(0x55ad47b2df00 msgr2=0x55ad47bfa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc1ed600700 0 client.15255 ms_handle_reset on v2:172.21.15.86:6838/3676043902 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 shutdown_connections 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4175143207 wait complete. 2024-04-25T19:03:48.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.429+0000 7fc214b9eb80 20 client.15255 trim_cache size 0 max 16384 2024-04-25T19:03:48.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.452+0000 7fc214b9eb80 1 Processor -- start 2024-04-25T19:03:48.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.452+0000 7fc214b9eb80 1 -- start start 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c1ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c08160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c1d450 con 0x55ad47c0e490 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c1db20 con 0x55ad47bff1d0 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c1e1f0 con 0x55ad47bfe8b0 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc203e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc203e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36684/0 (socket says 172.21.15.86:36684) 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc203e00700 1 -- 172.21.15.86:0/1817634576 learned_addr learned my addr 172.21.15.86:0/1817634576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:48.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc203400700 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c1ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.453+0000 7fc202a00700 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c08160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc203e00700 1 -- 172.21.15.86:0/1817634576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 msgr2=0x55ad47c1ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc203e00700 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c1ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc203e00700 1 -- 172.21.15.86:0/1817634576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 msgr2=0x55ad47c08160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc203e00700 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c08160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc203e00700 1 -- 172.21.15.86:0/1817634576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad47c1d280 con 0x55ad47c0e490 2024-04-25T19:03:48.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc203e00700 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c086a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0079c0 tx=0x7fc1fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522c7e28056c4a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc1efe00700 1 -- 172.21.15.86:0/1817634576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc003c00 con 0x55ad47c0e490 2024-04-25T19:03:48.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc1efe00700 1 -- 172.21.15.86:0/1817634576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc003da0 con 0x55ad47c0e490 2024-04-25T19:03:48.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.454+0000 7fc1efe00700 1 -- 172.21.15.86:0/1817634576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004770 con 0x55ad47c0e490 2024-04-25T19:03:48.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 -- 172.21.15.86:0/1817634576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 msgr2=0x55ad47c086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c086a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0079c0 tx=0x7fc1fc00b040 comp rx=0 tx=0).stop 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 -- 172.21.15.86:0/1817634576 shutdown_connections 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c086a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/1817634576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c08160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 -- 172.21.15.86:0/1817634576 >> 172.21.15.86:0/1817634576 conn(0x55ad47b2df00 msgr2=0x55ad47c097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 -- 172.21.15.86:0/1817634576 shutdown_connections 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.455+0000 7fc214b9eb80 1 -- 172.21.15.86:0/1817634576 wait complete. 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.456+0000 7fc214b9eb80 1 -- start start 2024-04-25T19:03:48.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.456+0000 7fc214b9eb80 1 Processor -- start 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.456+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c03b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.456+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.456+0000 7fc214b9eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c28ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c1ef90 con 0x55ad47c0e490 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c1e8c0 con 0x55ad47bff1d0 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc214b9eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ad47c1f660 con 0x55ad47bfe8b0 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc214b9eb80 10 client.0 _ll_register_callbacks cb 0x55ad47aea6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc203e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c28ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc203400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c03b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:44010/0 (socket says 172.21.15.86:44010) 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 -- 172.21.15.86:0/4027088876 learned_addr learned my addr 172.21.15.86:0/4027088876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc203400700 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c03b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51968/0 (socket says 172.21.15.86:51968) 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 -- 172.21.15.86:0/4027088876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 msgr2=0x55ad47c03b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c03b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 -- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 msgr2=0x55ad47c28ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c28ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.457+0000 7fc202a00700 1 -- 172.21.15.86:0/4027088876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad47c04090 con 0x55ad47bff1d0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc202a00700 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c28580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0005950 tx=0x7fc1f0012d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1000a57ef4e025c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc1ed600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc1ed600700 1 -- 172.21.15.86:0/4027088876 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f002a040 con 0x55ad47bff1d0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1fc003da0 con 0x55ad47bff1d0 2024-04-25T19:03:48.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc1ed600700 1 -- 172.21.15.86:0/4027088876 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fc1f001cab0 con 0x55ad47bff1d0 2024-04-25T19:03:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc1ed600700 1 -- 172.21.15.86:0/4027088876 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1f001ccb0 con 0x55ad47bff1d0 2024-04-25T19:03:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc214b9eb80 10 client.25379 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.458+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ad47c89d60 con 0x55ad47bff1d0 2024-04-25T19:03:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc1ed600700 1 -- 172.21.15.86:0/4027088876 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc003da0 con 0x55ad47bff1d0 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc1ed600700 1 -- 172.21.15.86:0/4027088876 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc1f0023070 con 0x55ad47bff1d0 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc214b9eb80 10 client.25379 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc214b9eb80 10 client.25379 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc214b9eb80 20 client.25379 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c8e520 0x55ad47c8d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc214b9eb80 4 client.25379 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 --> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] -- command(tid 0: {"prefix": "status"}) -- 0x55ad47b596d0 con 0x55ad47c8e520 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.459+0000 7fc203400700 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c8e520 0x55ad47c8d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc203400700 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c8e520 0x55ad47c8d230 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74ee70895084a46f server_cookie=34137015fffa6113 in_seq=0 out_seq=0 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc1ed600700 10 client.25379 ms_handle_connect on v2:172.21.15.86:6838/3676043902 2024-04-25T19:03:48.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc1ed600700 1 -- 172.21.15.86:0/4027088876 <== mds.1 v2:172.21.15.86:6838/3676043902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ad47b596d0 con 0x55ad47c8e520 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc1ed600700 10 client.25379 handle_command_reply: tid=0 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc214b9eb80 1 client.25379 shutdown 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 msgr2=0x55ad47c28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.460+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c28580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0005950 tx=0x7fc1f0012d80 comp rx=0 tx=0).stop 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 shutdown_connections 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:6838/3676043902,v1:172.21.15.86:6839/3676043902] conn(0x55ad47c8e520 0x55ad47c8d230 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad47c0e490 0x55ad47c28ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad47bfe8b0 0x55ad47c03b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 --2- 172.21.15.86:0/4027088876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ad47bff1d0 0x55ad47c28580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc1ed600700 0 client.25379 ms_handle_reset on v2:172.21.15.86:6838/3676043902 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 >> 172.21.15.86:0/4027088876 conn(0x55ad47b2df00 msgr2=0x55ad47c09c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 shutdown_connections 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 1 -- 172.21.15.86:0/4027088876 wait complete. 2024-04-25T19:03:48.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.461+0000 7fc214b9eb80 20 client.25379 trim_cache size 0 max 16384 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24509, 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 281.02150642399999, 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T19:03:48.466 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 325.27249520999999 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e80a3fe0 msgr2=0x7fc1e80a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e80a3fe0 0x7fc1e80a64a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc204100b20 tx=0x7fc1f0009260 comp rx=0 tx=0).stop 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204102970 msgr2=0x7fc2041990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204102970 0x7fc2041990a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40098e0 tx=0x7fc1f40057c0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 shutdown_connections 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204108970 0x7fc20419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc204102970 0x7fc2041990a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e80a3fe0 0x7fc1e80a64a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 --2- 172.21.15.86:0/1036522060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc204103330 0x7fc2041995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 >> 172.21.15.86:0/1036522060 conn(0x7fc2040fe4c0 msgr2=0x7fc2040fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 shutdown_connections 2024-04-25T19:03:48.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:48.462+0000 7fc208c00700 1 -- 172.21.15.86:0/1036522060 wait complete. 2024-04-25T19:03:48.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:48 smithi086 ceph-mon[27783]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T19:03:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:48 smithi183 ceph-mon[32344]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T19:03:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:48 smithi194 ceph-mon[32740]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T19:03:49.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T19:03:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:03:50.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:50 smithi183 ceph-mon[32344]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-25T19:03:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:50 smithi194 ceph-mon[32740]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-25T19:03:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:50 smithi086 ceph-mon[27783]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 -- 172.21.15.86:0/501764108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c104eb0 msgr2=0x7f337c105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 --2- 172.21.15.86:0/501764108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c104eb0 0x7f337c105290 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f336c0099c0 tx=0x7f336c009cd0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 -- 172.21.15.86:0/501764108 shutdown_connections 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 --2- 172.21.15.86:0/501764108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c109f90 0x7f337c111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 --2- 172.21.15.86:0/501764108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c104eb0 0x7f337c105290 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 --2- 172.21.15.86:0/501764108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c105860 0x7f337c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 -- 172.21.15.86:0/501764108 >> 172.21.15.86:0/501764108 conn(0x7f337c100780 msgr2=0x7f337c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 -- 172.21.15.86:0/501764108 shutdown_connections 2024-04-25T19:03:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.189+0000 7f3382800700 1 -- 172.21.15.86:0/501764108 wait complete. 2024-04-25T19:03:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.190+0000 7f3382800700 1 Processor -- start 2024-04-25T19:03:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 -- start start 2024-04-25T19:03:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 0x7f337c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c105860 0x7f337c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c109f90 0x7f337c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f337c114e50 con 0x7f337c109f90 2024-04-25T19:03:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f337c114cd0 con 0x7f337c105860 2024-04-25T19:03:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.191+0000 7f3382800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f337c114fd0 con 0x7f337c104eb0 2024-04-25T19:03:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c109f90 0x7f337c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 0x7f337c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 0x7f337c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59590/0 (socket says 172.21.15.86:59590) 2024-04-25T19:03:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 -- 172.21.15.86:0/3582806450 learned_addr learned my addr 172.21.15.86:0/3582806450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f3373e00700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c105860 0x7f337c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 -- 172.21.15.86:0/3582806450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c105860 msgr2=0x7f337c19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c105860 0x7f337c19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 -- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c109f90 msgr2=0x7f337c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c109f90 0x7f337c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 -- 172.21.15.86:0/3582806450 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c1a0350 con 0x7f337c104eb0 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.192+0000 7f337b400700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 0x7f337c19b340 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f336c006010 tx=0x7f336c005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.193+0000 7f3378c00700 1 -- 172.21.15.86:0/3582806450 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c026070 con 0x7f337c104eb0 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.193+0000 7f3378c00700 1 -- 172.21.15.86:0/3582806450 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c005680 con 0x7f337c104eb0 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.193+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1a05e0 con 0x7f337c104eb0 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.193+0000 7f3378c00700 1 -- 172.21.15.86:0/3582806450 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c00fb20 con 0x7f337c104eb0 2024-04-25T19:03:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.193+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1a7ec0 con 0x7f337c104eb0 2024-04-25T19:03:52.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f3378c00700 1 -- 172.21.15.86:0/3582806450 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f336c029070 con 0x7f337c104eb0 2024-04-25T19:03:52.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 Processor -- start 2024-04-25T19:03:52.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 -- start start 2024-04-25T19:03:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b62a713e40 con 0x55b62a7c02f0 2024-04-25T19:03:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b62a6ba6e0 con 0x55b62a7c0c10 2024-04-25T19:03:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.198+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b62a70fe00 con 0x55b62a7cbab0 2024-04-25T19:03:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f337b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f337be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f337be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50162/0 (socket says 172.21.15.86:50162) 2024-04-25T19:03:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f337be00700 1 -- 172.21.15.86:0/50390681 learned_addr learned my addr 172.21.15.86:0/50390681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3373e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3373e00700 1 -- 172.21.15.86:0/50390681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 msgr2=0x55b62a7cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3373e00700 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3378c00700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f33600a3fe0 0x7f33600a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3373e00700 1 -- 172.21.15.86:0/50390681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 msgr2=0x55b62a7c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3373e00700 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.199+0000 7f3373e00700 1 -- 172.21.15.86:0/50390681 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62a6b0630 con 0x55b62a7cbab0 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.200+0000 7f337b400700 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.200+0000 7f3378c00700 1 -- 172.21.15.86:0/3582806450 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f336c0d1130 con 0x7f337c104eb0 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.200+0000 7f337be00700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f33600a3fe0 0x7f33600a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.200+0000 7f3373e00700 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7cdea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f337c06b530 tx=0x7f3364009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584edd4262065de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.200+0000 7f3370c00700 1 -- 172.21.15.86:0/50390681 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364011d10 con 0x55b62a7cbab0 2024-04-25T19:03:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.201+0000 7f3370c00700 1 -- 172.21.15.86:0/50390681 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364004500 con 0x55b62a7cbab0 2024-04-25T19:03:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.201+0000 7f3370c00700 1 -- 172.21.15.86:0/50390681 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364005280 con 0x55b62a7cbab0 2024-04-25T19:03:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 -- 172.21.15.86:0/50390681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 msgr2=0x55b62a7cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f337be00700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f33600a3fe0 0x7f33600a64a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f33740075c0 tx=0x7f3374009040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7cdea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f337c06b530 tx=0x7f3364009f40 comp rx=0 tx=0).stop 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 -- 172.21.15.86:0/50390681 shutdown_connections 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7cdea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/50390681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.202+0000 7f338d1cdb80 1 -- 172.21.15.86:0/50390681 >> 172.21.15.86:0/50390681 conn(0x55b62a6ef550 msgr2=0x55b62a7bc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.203+0000 7f338d1cdb80 1 -- 172.21.15.86:0/50390681 shutdown_connections 2024-04-25T19:03:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.203+0000 7f338d1cdb80 1 -- 172.21.15.86:0/50390681 wait complete. 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.203+0000 7f338d1cdb80 1 -- start start 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.203+0000 7f338d1cdb80 1 Processor -- start 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7ce5a0 con 0x55b62a7c02f0 2024-04-25T19:03:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b62a6cd810 con 0x55b62a7cbab0 2024-04-25T19:03:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7ce720 con 0x55b62a7c0c10 2024-04-25T19:03:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f338d1cdb80 10 client.0 _ll_register_callbacks cb 0x55b62a6ac6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f337b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f337b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50170/0 (socket says 172.21.15.86:50170) 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f337b400700 1 -- 172.21.15.86:0/995132941 learned_addr learned my addr 172.21.15.86:0/995132941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f337be00700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.204+0000 7f3373e00700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f337b400700 1 -- 172.21.15.86:0/995132941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 msgr2=0x55b62a7c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f337b400700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f337b400700 1 -- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 msgr2=0x55b62a7c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f337b400700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f337b400700 1 -- 172.21.15.86:0/995132941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62a661880 con 0x55b62a7c02f0 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f337b400700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7cb210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f336c0059d0 tx=0x7f336c09d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4596a1be327c647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f3369600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f3369600700 1 -- 172.21.15.86:0/995132941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c099210 con 0x55b62a7c02f0 2024-04-25T19:03:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f3369600700 1 -- 172.21.15.86:0/995132941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f336c09c3c0 con 0x55b62a7c02f0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.205+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3364004500 con 0x55b62a7c02f0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.206+0000 7f3369600700 1 -- 172.21.15.86:0/995132941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f336c09c8d0 con 0x55b62a7c02f0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.206+0000 7f338d1cdb80 10 client.15273 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.206+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b62a6aaaa0 con 0x55b62a7c02f0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.206+0000 7f3369600700 1 -- 172.21.15.86:0/995132941 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364004500 con 0x55b62a7c02f0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.206+0000 7f3369600700 1 -- 172.21.15.86:0/995132941 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f336c0b6230 con 0x55b62a7c02f0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.206+0000 7f338d1cdb80 10 client.15273 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.207+0000 7f338d1cdb80 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.207+0000 7f338d1cdb80 20 client.15273 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.207+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a7db2e0 0x55b62a7dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.207+0000 7f338d1cdb80 4 client.15273 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.207+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b62a721630 con 0x55b62a7db2e0 2024-04-25T19:03:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.207+0000 7f3373e00700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a7db2e0 0x55b62a7dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.208+0000 7f3373e00700 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a7db2e0 0x55b62a7dc750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14073177789b26c server_cookie=9ba5fd1f6e84b4ce in_seq=0 out_seq=0 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.208+0000 7f3369600700 10 client.15273 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.208+0000 7f3369600700 1 -- 172.21.15.86:0/995132941 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b62a721630 con 0x55b62a7db2e0 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.208+0000 7f3369600700 10 client.15273 handle_command_reply: tid=0 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.208+0000 7f338d1cdb80 1 client.15273 shutdown 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 msgr2=0x55b62a7cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7cb210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f336c0059d0 tx=0x7f336c09d4b0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 shutdown_connections 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a7db2e0 0x55b62a7dc750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7cb210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/995132941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 >> 172.21.15.86:0/995132941 conn(0x55b62a6ef550 msgr2=0x55b62a7bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f3369600700 0 client.15273 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 shutdown_connections 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 1 -- 172.21.15.86:0/995132941 wait complete. 2024-04-25T19:03:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.209+0000 7f338d1cdb80 20 client.15273 trim_cache size 0 max 16384 2024-04-25T19:03:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.231+0000 7f338d1cdb80 1 Processor -- start 2024-04-25T19:03:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 -- start start 2024-04-25T19:03:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7daa90 con 0x55b62a7c02f0 2024-04-25T19:03:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7db160 con 0x55b62a7c0c10 2024-04-25T19:03:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7db830 con 0x55b62a7cbab0 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f337b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f337b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50180/0 (socket says 172.21.15.86:50180) 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f337b400700 1 -- 172.21.15.86:0/4246786486 learned_addr learned my addr 172.21.15.86:0/4246786486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f3373e00700 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.232+0000 7f337be00700 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337b400700 1 -- 172.21.15.86:0/4246786486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 msgr2=0x55b62a7e60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337b400700 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7e60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337b400700 1 -- 172.21.15.86:0/4246786486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 msgr2=0x55b62a7e5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337b400700 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7e5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337b400700 1 -- 172.21.15.86:0/4246786486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62a7b4020 con 0x55b62a7c02f0 2024-04-25T19:03:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337be00700 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f337b400700 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c3cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f336c09b220 tx=0x7f336c0d4880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7760a304f91edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f336be00700 1 -- 172.21.15.86:0/4246786486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c09cab0 con 0x55b62a7c02f0 2024-04-25T19:03:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f336be00700 1 -- 172.21.15.86:0/4246786486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c003e20 con 0x55b62a7c02f0 2024-04-25T19:03:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.233+0000 7f336be00700 1 -- 172.21.15.86:0/4246786486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c09cd40 con 0x55b62a7c02f0 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 -- 172.21.15.86:0/4246786486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 msgr2=0x55b62a7c3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c3cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f336c09b220 tx=0x7f336c0d4880 comp rx=0 tx=0).stop 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 -- 172.21.15.86:0/4246786486 shutdown_connections 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7cbab0 0x55b62a7e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7c3cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/4246786486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7c0c10 0x55b62a7e5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.234+0000 7f338d1cdb80 1 -- 172.21.15.86:0/4246786486 >> 172.21.15.86:0/4246786486 conn(0x55b62a6ef550 msgr2=0x55b62a7c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.235+0000 7f338d1cdb80 1 -- 172.21.15.86:0/4246786486 shutdown_connections 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.235+0000 7f338d1cdb80 1 -- 172.21.15.86:0/4246786486 wait complete. 2024-04-25T19:03:52.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.235+0000 7f338d1cdb80 1 -- start start 2024-04-25T19:03:52.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.235+0000 7f338d1cdb80 1 Processor -- start 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7dc5d0 con 0x55b62a7c02f0 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7dbf00 con 0x55b62a7cbab0 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b62a7dcca0 con 0x55b62a7c0c10 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f338d1cdb80 10 client.0 _ll_register_callbacks cb 0x55b62a6ac6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f337b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f337b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7e5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50192/0 (socket says 172.21.15.86:50192) 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f337b400700 1 -- 172.21.15.86:0/3727783415 learned_addr learned my addr 172.21.15.86:0/3727783415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f3373e00700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.236+0000 7f337be00700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f337b400700 1 -- 172.21.15.86:0/3727783415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 msgr2=0x55b62a7e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f337b400700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f337b400700 1 -- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 msgr2=0x55b62a7e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f337b400700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f337b400700 1 -- 172.21.15.86:0/3727783415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62a7e62f0 con 0x55b62a7c02f0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f3373e00700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f337b400700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7e5bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f336c0059d0 tx=0x7f336c0d4df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40753ccfa0030b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f3369600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f3369600700 1 -- 172.21.15.86:0/3727783415 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c0ba020 con 0x55b62a7c02f0 2024-04-25T19:03:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f3369600700 1 -- 172.21.15.86:0/3727783415 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f336c0ce9a0 con 0x55b62a7c02f0 2024-04-25T19:03:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.237+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b62a71b6d0 con 0x55b62a7c02f0 2024-04-25T19:03:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f3369600700 1 -- 172.21.15.86:0/3727783415 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f336c09cab0 con 0x55b62a7c02f0 2024-04-25T19:03:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 10 client.15279 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b62a7e62f0 con 0x55b62a7c02f0 2024-04-25T19:03:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f3369600700 1 -- 172.21.15.86:0/3727783415 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b62a71b6d0 con 0x55b62a7c02f0 2024-04-25T19:03:52.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f3369600700 1 -- 172.21.15.86:0/3727783415 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f336c097ae0 con 0x55b62a7c02f0 2024-04-25T19:03:52.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 10 client.15279 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 20 client.15279 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a8472b0 0x55b62a845fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 4 client.15279 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.238+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "status"}) -- 0x55b62a594200 con 0x55b62a8472b0 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.239+0000 7f3373e00700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a8472b0 0x55b62a845fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.239+0000 7f3373e00700 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a8472b0 0x55b62a845fc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8392fb0c3ef607f9 server_cookie=3f67ee0302d75bda in_seq=0 out_seq=0 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.239+0000 7f3369600700 10 client.15279 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f3369600700 1 -- 172.21.15.86:0/3727783415 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b62a594200 con 0x55b62a8472b0 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f3369600700 10 client.15279 handle_command_reply: tid=0 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 client.15279 shutdown 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 msgr2=0x55b62a7e5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7e5bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f336c0059d0 tx=0x7f336c0d4df0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 shutdown_connections 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b62a8472b0 0x55b62a845fc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62a7cbab0 0x55b62a7e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62a7c02f0 0x55b62a7e5bd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 --2- 172.21.15.86:0/3727783415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62a7c0c10 0x55b62a7e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 >> 172.21.15.86:0/3727783415 conn(0x55b62a6ef550 msgr2=0x55b62a7c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f3369600700 0 client.15279 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 shutdown_connections 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.240+0000 7f338d1cdb80 1 -- 172.21.15.86:0/3727783415 wait complete. 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.241+0000 7f338d1cdb80 20 client.15279 trim_cache size 0 max 16384 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:03:52.244 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1ec2eb56-0334-11ef-bc93-c7b262605968", 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24527, 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 300.663446593, 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 321.17156270599997 2024-04-25T19:03:52.245 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.241+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f33600a3fe0 msgr2=0x7f33600a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.241+0000 7f3382800700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f33600a3fe0 0x7f33600a64a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f33740075c0 tx=0x7f3374009040 comp rx=0 tx=0).stop 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.241+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 msgr2=0x7f337c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.241+0000 7f3382800700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 0x7f337c19b340 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f336c006010 tx=0x7f336c005280 comp rx=0 tx=0).stop 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 shutdown_connections 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337c109f90 0x7f337c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f33600a3fe0 0x7f33600a64a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f337c104eb0 0x7f337c19b340 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 --2- 172.21.15.86:0/3582806450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c105860 0x7f337c19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 >> 172.21.15.86:0/3582806450 conn(0x7f337c100780 msgr2=0x7f337c109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 shutdown_connections 2024-04-25T19:03:52.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:03:52.242+0000 7f3382800700 1 -- 172.21.15.86:0/3582806450 wait complete. 2024-04-25T19:03:52.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:52 smithi086 ceph-mon[27783]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T19:03:52.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:52 smithi183 ceph-mon[32344]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T19:03:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:52 smithi194 ceph-mon[32740]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T19:03:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 -- 172.21.15.86:0/412987570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10104d60 msgr2=0x7f8a10108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 --2- 172.21.15.86:0/412987570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10104d60 0x7f8a10108da0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009900 tx=0x7f89fc02a410 comp rx=0 tx=0).stop 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 -- 172.21.15.86:0/412987570 shutdown_connections 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 --2- 172.21.15.86:0/412987570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a10109480 0x7f8a10111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 --2- 172.21.15.86:0/412987570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10104d60 0x7f8a10108da0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 --2- 172.21.15.86:0/412987570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a101043b0 0x7f8a10104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 -- 172.21.15.86:0/412987570 >> 172.21.15.86:0/412987570 conn(0x7f8a100ffc50 msgr2=0x7f8a10102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 -- 172.21.15.86:0/412987570 shutdown_connections 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.876+0000 7f8a18600640 1 -- 172.21.15.86:0/412987570 wait complete. 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.877+0000 7f8a18600640 1 Processor -- start 2024-04-25T19:03:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.877+0000 7f8a18600640 1 -- start start 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a18600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 0x7f8a101a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a18600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10109480 0x7f8a101a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a18600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1019a6f0 0x7f8a1019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a18600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10114410 con 0x7f8a1019a6f0 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a18600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10114290 con 0x7f8a10109480 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a18600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10114590 con 0x7f8a101043b0 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 0x7f8a101a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10109480 0x7f8a101a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 0x7f8a101a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59634/0 (socket says 172.21.15.86:59634) 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 -- 172.21.15.86:0/2821620273 learned_addr learned my addr 172.21.15.86:0/2821620273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a16600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1019a6f0 0x7f8a1019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 -- 172.21.15.86:0/2821620273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10109480 msgr2=0x7f8a101a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10109480 0x7f8a101a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 -- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1019a6f0 msgr2=0x7f8a1019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1019a6f0 0x7f8a1019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.878+0000 7f8a15c00640 1 -- 172.21.15.86:0/2821620273 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1019b2f0 con 0x7f8a101043b0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.879+0000 7f8a15c00640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 0x7f8a101a0630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400e9f0 tx=0x7f8a0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.879+0000 7f8a0e000640 1 -- 172.21.15.86:0/2821620273 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400cd80 con 0x7f8a101043b0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.879+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a101a73d0 con 0x7f8a101043b0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.879+0000 7f8a0e000640 1 -- 172.21.15.86:0/2821620273 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a040108e0 con 0x7f8a101043b0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.879+0000 7f8a0e000640 1 -- 172.21.15.86:0/2821620273 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04018980 con 0x7f8a101043b0 2024-04-25T19:03:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.879+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101a77d0 con 0x7f8a101043b0 2024-04-25T19:03:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.882+0000 7f8a1b478740 1 Processor -- start 2024-04-25T19:03:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 -- start start 2024-04-25T19:03:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a0e000640 1 -- 172.21.15.86:0/2821620273 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f8a04010430 con 0x7f8a101043b0 2024-04-25T19:03:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d27430ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d2743194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d27431bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d274274900 con 0x55d274319a00 2024-04-25T19:03:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d27421aef0 con 0x55d27430f140 2024-04-25T19:03:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a1b478740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d2742708c0 con 0x55d27430e820 2024-04-25T19:03:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d27431bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a16600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d27430ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d27431bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50208/0 (socket says 172.21.15.86:50208) 2024-04-25T19:03:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a15200640 1 -- 172.21.15.86:0/3609293669 learned_addr learned my addr 172.21.15.86:0/3609293669 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.883+0000 7f8a15c00640 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d2743194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a15200640 1 -- 172.21.15.86:0/3609293669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 msgr2=0x55d27430ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a15200640 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d27430ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a15200640 1 -- 172.21.15.86:0/3609293669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 msgr2=0x55d2743194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a15200640 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d2743194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a15200640 1 -- 172.21.15.86:0/3609293669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d273f87210 con 0x55d274319a00 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a15200640 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d27431bdf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f89fc006010 tx=0x7f89fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e48f3173590820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.884+0000 7f8a02a00640 1 -- 172.21.15.86:0/3609293669 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc035040 con 0x55d274319a00 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a02a00640 1 -- 172.21.15.86:0/3609293669 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc0054f0 con 0x55d274319a00 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a02a00640 1 -- 172.21.15.86:0/3609293669 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0035f0 con 0x55d274319a00 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a0e000640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89f80a3f70 0x7f89f80a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a0e000640 1 -- 172.21.15.86:0/2821620273 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f8a040d0df0 con 0x7f8a101043b0 2024-04-25T19:03:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a16600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89f80a3f70 0x7f89f80a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a1b478740 1 -- 172.21.15.86:0/3609293669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 msgr2=0x55d27431bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.885+0000 7f8a1b478740 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d27431bdf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f89fc006010 tx=0x7f89fc004040 comp rx=0 tx=0).stop 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 -- 172.21.15.86:0/3609293669 shutdown_connections 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d27431bdf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d2743194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 --2- 172.21.15.86:0/3609293669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d27430ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 -- 172.21.15.86:0/3609293669 >> 172.21.15.86:0/3609293669 conn(0x55d27430ad30 msgr2=0x55d27430b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 -- 172.21.15.86:0/3609293669 shutdown_connections 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a16600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89f80a3f70 0x7f89f80a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005fd0 tx=0x7f8a08009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.886+0000 7f8a1b478740 1 -- 172.21.15.86:0/3609293669 wait complete. 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 -- start start 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 Processor -- start 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d2743169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d274316ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d274317420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d27431c5d0 con 0x55d27430f140 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d27422e310 con 0x55d27430e820 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d27431c750 con 0x55d274319a00 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.887+0000 7f8a1b478740 10 client.0 _ll_register_callbacks cb 0x55d2741fccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d274316ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d274316ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50210/0 (socket says 172.21.15.86:50210) 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 -- 172.21.15.86:0/187094353 learned_addr learned my addr 172.21.15.86:0/187094353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15c00640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d2743169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a16600640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d274317420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 -- 172.21.15.86:0/187094353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 msgr2=0x55d274317420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d274317420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 -- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 msgr2=0x55d2743169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d2743169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 -- 172.21.15.86:0/187094353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d274053620 con 0x55d27430f140 2024-04-25T19:03:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a16600640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d274317420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15c00640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d2743169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f8a15200640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d274316ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89fc02a930 tx=0x7f89fc0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b600cf3b864a2194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.888+0000 7f89f7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.889+0000 7f89f7400640 1 -- 172.21.15.86:0/187094353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc052060 con 0x55d27430f140 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.889+0000 7f89f7400640 1 -- 172.21.15.86:0/187094353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f89fc004ec0 con 0x55d27430f140 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.889+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc0054f0 con 0x55d27430f140 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.889+0000 7f89f7400640 1 -- 172.21.15.86:0/187094353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89fc035050 con 0x55d27430f140 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.889+0000 7f8a1b478740 10 client.15291 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.889+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d274328850 con 0x55d27430f140 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f89f7400640 1 -- 172.21.15.86:0/187094353 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc0054f0 con 0x55d27430f140 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f89f7400640 1 -- 172.21.15.86:0/187094353 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f89fc003ba0 con 0x55d27430f140 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a1b478740 10 client.15291 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a1b478740 10 client.15291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a1b478740 20 client.15291 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a1b478740 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d274329ba0 0x55d27432bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a1b478740 4 client.15291 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2742888b0 con 0x55d274329ba0 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.890+0000 7f8a15c00640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d274329ba0 0x55d27432bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.891+0000 7f8a15c00640 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d274329ba0 0x55d27432bf90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5a0f04b3b949b9 server_cookie=1824368902603904 in_seq=0 out_seq=0 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.891+0000 7f89f7400640 10 client.15291 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f89f7400640 1 -- 172.21.15.86:0/187094353 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2742888b0 con 0x55d274329ba0 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f89f7400640 10 client.15291 handle_command_reply: tid=0 2024-04-25T19:03:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 client.15291 shutdown 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 msgr2=0x55d274316ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d274316ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89fc02a930 tx=0x7f89fc0045a0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 shutdown_connections 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d274329ba0 0x55d27432bf90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d274317420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d274316ee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f89f7400640 0 client.15291 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 --2- 172.21.15.86:0/187094353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d2743169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 >> 172.21.15.86:0/187094353 conn(0x55d27430ad30 msgr2=0x55d274310a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 shutdown_connections 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 1 -- 172.21.15.86:0/187094353 wait complete. 2024-04-25T19:03:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.892+0000 7f8a1b478740 20 client.15291 trim_cache size 0 max 16384 2024-04-25T19:03:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.908+0000 7f8a1b478740 1 Processor -- start 2024-04-25T19:03:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 -- start start 2024-04-25T19:03:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d274325490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d27430a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d2743108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101afea0 con 0x55d274319a00 2024-04-25T19:03:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b19e0 con 0x55d27430f140 2024-04-25T19:03:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a1b478740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b0570 con 0x55d27430e820 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a15c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d274325490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a16600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d2743108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a15c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d274325490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59660/0 (socket says 172.21.15.86:59660) 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a15c00640 1 -- 172.21.15.86:0/646310496 learned_addr learned my addr 172.21.15.86:0/646310496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.909+0000 7f8a15200640 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d27430a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a15c00640 1 -- 172.21.15.86:0/646310496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 msgr2=0x55d27430a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a15c00640 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d27430a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a15c00640 1 -- 172.21.15.86:0/646310496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 msgr2=0x55d2743108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a15c00640 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d2743108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a15c00640 1 -- 172.21.15.86:0/646310496 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27430ab50 con 0x55d27430e820 2024-04-25T19:03:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a15c00640 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d274325490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a040d4640 tx=0x7f8a040d32b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f91a3d616f027456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a02000640 1 -- 172.21.15.86:0/646310496 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04010720 con 0x55d27430e820 2024-04-25T19:03:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a02000640 1 -- 172.21.15.86:0/646310496 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a0409cbb0 con 0x55d27430e820 2024-04-25T19:03:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.910+0000 7f8a02000640 1 -- 172.21.15.86:0/646310496 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04018b20 con 0x55d27430e820 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 -- 172.21.15.86:0/646310496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 msgr2=0x55d274325490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d274325490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a040d4640 tx=0x7f8a040d32b0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 -- 172.21.15.86:0/646310496 shutdown_connections 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d274319a00 0x55d2743108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430f140 0x55d27430a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 --2- 172.21.15.86:0/646310496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d27430e820 0x55d274325490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 -- 172.21.15.86:0/646310496 >> 172.21.15.86:0/646310496 conn(0x55d27430ad30 msgr2=0x55d2743101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 -- 172.21.15.86:0/646310496 shutdown_connections 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.911+0000 7f8a1b478740 1 -- 172.21.15.86:0/646310496 wait complete. 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 -- start start 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 Processor -- start 2024-04-25T19:03:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d274321ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d27431ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d27431f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b1310 con 0x55d27430f140 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101b0c40 con 0x55d27430e820 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101ade80 con 0x55d274319a00 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.912+0000 7f8a1b478740 10 client.0 _ll_register_callbacks cb 0x7f89fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d27431ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a16600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d27431f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d274321ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a16600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d27431f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59674/0 (socket says 172.21.15.86:59674) 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a16600640 1 -- 172.21.15.86:0/502579689 learned_addr learned my addr 172.21.15.86:0/502579689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 -- 172.21.15.86:0/502579689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 msgr2=0x55d27431f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d27431f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 -- 172.21.15.86:0/502579689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 msgr2=0x55d27431ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d27431ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 -- 172.21.15.86:0/502579689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27431f9c0 con 0x55d27430e820 2024-04-25T19:03:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15200640 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d27431ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f8a15c00640 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d274321ba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a040a1070 tx=0x7f8a0400c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbec2e2cae0ee05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.913+0000 7f89f7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:03:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f89f7400640 1 -- 172.21.15.86:0/502579689 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04010720 con 0x55d27430e820 2024-04-25T19:03:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f89f7400640 1 -- 172.21.15.86:0/502579689 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f8a04010a90 con 0x55d27430e820 2024-04-25T19:03:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0409cbb0 con 0x55d27430e820 2024-04-25T19:03:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f89f7400640 1 -- 172.21.15.86:0/502579689 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a040a0eb0 con 0x55d27430e820 2024-04-25T19:03:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f8a1b478740 10 client.25415 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d27438fb90 con 0x55d27430e820 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.914+0000 7f89f7400640 1 -- 172.21.15.86:0/502579689 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a0409cbb0 con 0x55d27430e820 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f89f7400640 1 -- 172.21.15.86:0/502579689 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8a04098900 con 0x55d27430e820 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a1b478740 10 client.25415 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a1b478740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a1b478740 20 client.25415 populate_metadata read hostname 'smithi086' 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a1b478740 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2743901f0 0x55d2743925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a1b478740 4 client.25415 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d274282320 con 0x55d2743901f0 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.915+0000 7f8a15200640 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2743901f0 0x55d2743925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.916+0000 7f89f7400640 10 client.25415 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.916+0000 7f8a15200640 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2743901f0 0x55d2743925e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522bba18a0dfaf77 server_cookie=a9dee9cf2c173e45 in_seq=0 out_seq=0 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f89f7400640 1 -- 172.21.15.86:0/502579689 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d274282320 con 0x55d2743901f0 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f89f7400640 10 client.25415 handle_command_reply: tid=0 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 client.25415 shutdown 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 msgr2=0x55d274321ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d274321ba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a040a1070 tx=0x7f8a0400c7d0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 shutdown_connections 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2743901f0 0x55d2743925e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d274319a00 0x55d27431f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d27430f140 0x55d27431ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 --2- 172.21.15.86:0/502579689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d27430e820 0x55d274321ba0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 >> 172.21.15.86:0/502579689 conn(0x55d27430ad30 msgr2=0x55d274310a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f89f7400640 0 client.25415 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 shutdown_connections 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.920+0000 7f8a1b478740 1 -- 172.21.15.86:0/502579689 wait complete. 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a1b478740 20 client.25415 trim_cache size 0 max 16384 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:03:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89f80a3f70 msgr2=0x7f89f80a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89f80a3f70 0x7f89f80a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005fd0 tx=0x7f8a08009500 comp rx=0 tx=0).stop 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 msgr2=0x7f8a101a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 0x7f8a101a0630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400e9f0 tx=0x7f8a0400eeb0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 shutdown_connections 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89f80a3f70 0x7f89f80a6430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1019a6f0 0x7f8a1019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a10109480 0x7f8a101a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.921+0000 7f8a18600640 1 --2- 172.21.15.86:0/2821620273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a101043b0 0x7f8a101a0630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.922+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 >> 172.21.15.86:0/2821620273 conn(0x7f8a100ffc50 msgr2=0x7f8a10101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.922+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 shutdown_connections 2024-04-25T19:03:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:03:52.922+0000 7f8a18600640 1 -- 172.21.15.86:0/2821620273 wait complete. 2024-04-25T19:03:53.076 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-25T19:03:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:54 smithi183 ceph-mon[32344]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-25T19:03:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:54 smithi194 ceph-mon[32740]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-25T19:03:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:54 smithi086 ceph-mon[27783]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-25T19:03:56.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:56 smithi183 ceph-mon[32344]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T19:03:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:56 smithi194 ceph-mon[32740]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T19:03:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:56 smithi086 ceph-mon[27783]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T19:03:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:03:58 smithi183 ceph-mon[32344]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:03:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:03:58 smithi194 ceph-mon[32740]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:03:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:03:58 smithi086 ceph-mon[27783]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:04:00.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:00 smithi183 ceph-mon[32344]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T19:04:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:00 smithi194 ceph-mon[32740]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T19:04:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:00 smithi086 ceph-mon[27783]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T19:04:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:02 smithi183 ceph-mon[32344]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T19:04:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:02 smithi194 ceph-mon[32740]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T19:04:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:02 smithi086 ceph-mon[27783]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T19:04:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:04 smithi183 ceph-mon[32344]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T19:04:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:04 smithi194 ceph-mon[32740]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T19:04:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:04 smithi086 ceph-mon[27783]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T19:04:05.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:05 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:04:05.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:05 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:04:05.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:05 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:04:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 -- 172.21.15.86:0/300297179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c106fb0 msgr2=0x7f1e2c107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 --2- 172.21.15.86:0/300297179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c107390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009900 tx=0x7f1e1c02a3e0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 -- 172.21.15.86:0/300297179 shutdown_connections 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 --2- 172.21.15.86:0/300297179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c101b60 0x7f1e2c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 --2- 172.21.15.86:0/300297179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c1011a0 0x7f1e2c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 --2- 172.21.15.86:0/300297179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c107390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.075+0000 7f1e30c00640 1 -- 172.21.15.86:0/300297179 >> 172.21.15.86:0/300297179 conn(0x7f1e2c076c50 msgr2=0x7f1e2c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.076+0000 7f1e30c00640 1 -- 172.21.15.86:0/300297179 shutdown_connections 2024-04-25T19:04:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.076+0000 7f1e30c00640 1 -- 172.21.15.86:0/300297179 wait complete. 2024-04-25T19:04:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.077+0000 7f1e30c00640 1 Processor -- start 2024-04-25T19:04:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 -- start start 2024-04-25T19:04:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c1011a0 0x7f1e2c073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 0x7f1e2c06e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1110e0 con 0x7f1e2c101b60 2024-04-25T19:04:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c110f60 con 0x7f1e2c106fb0 2024-04-25T19:04:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e30c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c111260 con 0x7f1e2c1011a0 2024-04-25T19:04:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 0x7f1e2c06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 0x7f1e2c06e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40198/0 (socket says 172.21.15.86:40198) 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e29400640 1 -- 172.21.15.86:0/2917302281 learned_addr learned my addr 172.21.15.86:0/2917302281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e29e00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c1011a0 0x7f1e2c073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.078+0000 7f1e2a800640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e2a800640 1 -- 172.21.15.86:0/2917302281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c1011a0 msgr2=0x7f1e2c073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e2a800640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c1011a0 0x7f1e2c073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e2a800640 1 -- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 msgr2=0x7f1e2c06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e2a800640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 0x7f1e2c06e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e2a800640 1 -- 172.21.15.86:0/2917302281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c06f320 con 0x7f1e2c106fb0 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e29e00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c1011a0 0x7f1e2c073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e29400640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 0x7f1e2c06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.079+0000 7f1e2a800640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c06ed60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009fd0 tx=0x7f1e2400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.080+0000 7f1e22a00640 1 -- 172.21.15.86:0/2917302281 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400ce20 con 0x7f1e2c106fb0 2024-04-25T19:04:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.080+0000 7f1e22a00640 1 -- 172.21.15.86:0/2917302281 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e24005020 con 0x7f1e2c106fb0 2024-04-25T19:04:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.080+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c06f610 con 0x7f1e2c106fb0 2024-04-25T19:04:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.080+0000 7f1e22a00640 1 -- 172.21.15.86:0/2917302281 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24010690 con 0x7f1e2c106fb0 2024-04-25T19:04:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.080+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c1ad970 con 0x7f1e2c106fb0 2024-04-25T19:04:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.084+0000 7f1e22a00640 1 -- 172.21.15.86:0/2917302281 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1e240054d0 con 0x7f1e2c106fb0 2024-04-25T19:04:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.085+0000 7f1e33bc5740 1 Processor -- start 2024-04-25T19:04:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.085+0000 7f1e33bc5740 1 -- start start 2024-04-25T19:04:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe11c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe12140 0x555a6fe1c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe1edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a6fd77900 con 0x555a6fe11820 2024-04-25T19:04:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a6fd1def0 con 0x555a6fe12140 2024-04-25T19:04:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555a6fd738c0 con 0x555a6fe1ca00 2024-04-25T19:04:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.086+0000 7f1e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe11c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe11c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40210/0 (socket says 172.21.15.86:40210) 2024-04-25T19:04:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 -- 172.21.15.86:0/1709867735 learned_addr learned my addr 172.21.15.86:0/1709867735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e22a00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e100a4070 0x7f1e100a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e22a00640 1 -- 172.21.15.86:0/2917302281 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1e240d10a0 con 0x7f1e2c106fb0 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 -- 172.21.15.86:0/1709867735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 msgr2=0x555a6fe1edf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe1edf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 -- 172.21.15.86:0/1709867735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe12140 msgr2=0x555a6fe1c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe12140 0x555a6fe1c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.087+0000 7f1e29400640 1 -- 172.21.15.86:0/1709867735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a6fa8a210 con 0x555a6fe11820 2024-04-25T19:04:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.088+0000 7f1e29400640 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe11c00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e180115d0 tx=0x7f1e18009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59568b261c3a4ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.088+0000 7f1e2a800640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e100a4070 0x7f1e100a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.088+0000 7f1e17000640 1 -- 172.21.15.86:0/1709867735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18005020 con 0x555a6fe11820 2024-04-25T19:04:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.088+0000 7f1e17000640 1 -- 172.21.15.86:0/1709867735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e180047a0 con 0x555a6fe11820 2024-04-25T19:04:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.088+0000 7f1e17000640 1 -- 172.21.15.86:0/1709867735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800f930 con 0x555a6fe11820 2024-04-25T19:04:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.089+0000 7f1e33bc5740 1 -- 172.21.15.86:0/1709867735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 msgr2=0x555a6fe11c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.089+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe11c00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e180115d0 tx=0x7f1e18009f90 comp rx=0 tx=0).stop 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.089+0000 7f1e2a800640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e100a4070 0x7f1e100a6530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ce5c0 tx=0x7f1e24005f70 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 -- 172.21.15.86:0/1709867735 shutdown_connections 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe1edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe12140 0x555a6fe1c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/1709867735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe11c00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 -- 172.21.15.86:0/1709867735 >> 172.21.15.86:0/1709867735 conn(0x555a6fe0dd30 msgr2=0x555a6fe0e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 -- 172.21.15.86:0/1709867735 shutdown_connections 2024-04-25T19:04:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 -- 172.21.15.86:0/1709867735 wait complete. 2024-04-25T19:04:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.090+0000 7f1e33bc5740 1 -- start start 2024-04-25T19:04:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 Processor -- start 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe19990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 0x555a6fe19ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1ca00 0x555a6fe1a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a6fe27db0 con 0x555a6fe1ca00 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a6fd31310 con 0x555a6fe11820 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555a6fe27f30 con 0x555a6fe12140 2024-04-25T19:04:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.091+0000 7f1e33bc5740 10 client.0 _ll_register_callbacks cb 0x555a6fcffcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 0x555a6fe19ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 0x555a6fe19ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44156/0 (socket says 172.21.15.86:44156) 2024-04-25T19:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 -- 172.21.15.86:0/3229618163 learned_addr learned my addr 172.21.15.86:0/3229618163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29400640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1ca00 0x555a6fe1a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 -- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 msgr2=0x555a6fe19990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe19990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 -- 172.21.15.86:0/3229618163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1ca00 msgr2=0x555a6fe1a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1ca00 0x555a6fe1a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29e00640 1 -- 172.21.15.86:0/3229618163 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6fb56620 con 0x555a6fe12140 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.092+0000 7f1e29400640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1ca00 0x555a6fe1a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e29e00640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 0x555a6fe19ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009900 tx=0x7f1e1c009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f9903a24f2ac1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e0b400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:04:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e0b400640 1 -- 172.21.15.86:0/3229618163 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c035040 con 0x555a6fe12140 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e0b400640 1 -- 172.21.15.86:0/3229618163 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1e1c004770 con 0x555a6fe12140 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e180047a0 con 0x555a6fe12140 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e0b400640 1 -- 172.21.15.86:0/3229618163 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e1c005610 con 0x555a6fe12140 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e33bc5740 10 client.25414 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.093+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a6fe34040 con 0x555a6fe12140 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e0b400640 1 -- 172.21.15.86:0/3229618163 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e180047a0 con 0x555a6fe12140 2024-04-25T19:04:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e0b400640 1 -- 172.21.15.86:0/3229618163 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1e1c003c00 con 0x555a6fe12140 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e33bc5740 10 client.25414 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e33bc5740 10 client.25414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e33bc5740 20 client.25414 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe35570 0x555a6fe379c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e33bc5740 4 client.25414 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.094+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a6fd8b8b0 con 0x555a6fe35570 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.095+0000 7f1e29400640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe35570 0x555a6fe379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.096+0000 7f1e29400640 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe35570 0x555a6fe379c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4fc60dab6a84e5 server_cookie=c93de949da496aa8 in_seq=0 out_seq=0 2024-04-25T19:04:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.096+0000 7f1e0b400640 10 client.25414 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e0b400640 1 -- 172.21.15.86:0/3229618163 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a6fd8b8b0 con 0x555a6fe35570 2024-04-25T19:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e0b400640 10 client.25414 handle_command_reply: tid=0 2024-04-25T19:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 client.25414 shutdown 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 msgr2=0x555a6fe19ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 0x555a6fe19ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009900 tx=0x7f1e1c009630 comp rx=0 tx=0).stop 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 shutdown_connections 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe35570 0x555a6fe379c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1ca00 0x555a6fe1a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe12140 0x555a6fe19ed0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e0b400640 0 client.25414 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3229618163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe19990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.097+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 >> 172.21.15.86:0/3229618163 conn(0x555a6fe0dd30 msgr2=0x555a6fe13ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.098+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 shutdown_connections 2024-04-25T19:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.098+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3229618163 wait complete. 2024-04-25T19:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.098+0000 7f1e33bc5740 20 client.25414 trim_cache size 0 max 16384 2024-04-25T19:04:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 Processor -- start 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 -- start start 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe30d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1a1e0 0x555a6fe3d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1b42d0 con 0x555a6fe1a1e0 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1b5e10 con 0x555a6fe11820 2024-04-25T19:04:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.126+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1b49a0 con 0x555a6fe1ca00 2024-04-25T19:04:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44160/0 (socket says 172.21.15.86:44160) 2024-04-25T19:04:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 -- 172.21.15.86:0/3383798076 learned_addr learned my addr 172.21.15.86:0/3383798076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29400640 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe30d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29400640 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1a1e0 0x555a6fe3d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 -- 172.21.15.86:0/3383798076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 msgr2=0x555a6fe30d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe30d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 -- 172.21.15.86:0/3383798076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1a1e0 msgr2=0x555a6fe3d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1a1e0 0x555a6fe3d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.127+0000 7f1e29e00640 1 -- 172.21.15.86:0/3383798076 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a6fe18940 con 0x555a6fe1ca00 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.128+0000 7f1e29e00640 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe141e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c005ab0 tx=0x7f1e1c009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84e0fff98c3182f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.128+0000 7f1e16600640 1 -- 172.21.15.86:0/3383798076 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c035070 con 0x555a6fe1ca00 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.128+0000 7f1e16600640 1 -- 172.21.15.86:0/3383798076 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e1c0035f0 con 0x555a6fe1ca00 2024-04-25T19:04:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.128+0000 7f1e16600640 1 -- 172.21.15.86:0/3383798076 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0049d0 con 0x555a6fe1ca00 2024-04-25T19:04:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3383798076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 msgr2=0x555a6fe141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe141e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c005ab0 tx=0x7f1e1c009850 comp rx=0 tx=0).stop 2024-04-25T19:04:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3383798076 shutdown_connections 2024-04-25T19:04:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe1a1e0 0x555a6fe3d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe141e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/3383798076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe11820 0x555a6fe30d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3383798076 >> 172.21.15.86:0/3383798076 conn(0x555a6fe0dd30 msgr2=0x555a6fe0e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3383798076 shutdown_connections 2024-04-25T19:04:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.129+0000 7f1e33bc5740 1 -- 172.21.15.86:0/3383798076 wait complete. 2024-04-25T19:04:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.130+0000 7f1e33bc5740 1 -- start start 2024-04-25T19:04:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.130+0000 7f1e33bc5740 1 Processor -- start 2024-04-25T19:04:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe2b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe1a1e0 0x555a6fe2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1b5740 con 0x555a6fe11820 2024-04-25T19:04:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1b5070 con 0x555a6fe1a1e0 2024-04-25T19:04:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c1b22b0 con 0x555a6fe1ca00 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e33bc5740 10 client.0 _ll_register_callbacks cb 0x7f1e1c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe2b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe2b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40244/0 (socket says 172.21.15.86:40244) 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e29400640 1 -- 172.21.15.86:0/2940395561 learned_addr learned my addr 172.21.15.86:0/2940395561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.131+0000 7f1e29e00640 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe1a1e0 0x555a6fe2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e29400640 1 -- 172.21.15.86:0/2940395561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 msgr2=0x555a6fe9aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e29400640 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e29400640 1 -- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe1a1e0 msgr2=0x555a6fe2b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e29400640 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe1a1e0 0x555a6fe2b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e29400640 1 -- 172.21.15.86:0/2940395561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6fe11f70 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e29400640 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe2b130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1e18011f20 tx=0x7f1e18009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398b09a7c82280e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.132+0000 7f1e0b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.133+0000 7f1e0b400640 1 -- 172.21.15.86:0/2940395561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800f850 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.133+0000 7f1e0b400640 1 -- 172.21.15.86:0/2940395561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1e1800f9f0 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.133+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1c0035f0 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.133+0000 7f1e0b400640 1 -- 172.21.15.86:0/2940395561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e1803d050 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.133+0000 7f1e33bc5740 10 client.15312 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.133+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a6fe9c0e0 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e0b400640 1 -- 172.21.15.86:0/2940395561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e1c0035f0 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e0b400640 1 -- 172.21.15.86:0/2940395561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1e18036460 con 0x555a6fe11820 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e33bc5740 10 client.15312 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e33bc5740 10 client.15312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e33bc5740 20 client.15312 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe9c610 0x555a6fe9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e33bc5740 4 client.15312 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.134+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a6fd85320 con 0x555a6fe9c610 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.135+0000 7f1e29e00640 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe9c610 0x555a6fe9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.135+0000 7f1e0b400640 10 client.15312 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.135+0000 7f1e29e00640 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe9c610 0x555a6fe9ea00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8e948d013d84ea server_cookie=3d6836c894297364 in_seq=0 out_seq=0 2024-04-25T19:04:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.142+0000 7f1e0b400640 1 -- 172.21.15.86:0/2940395561 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a6fd85320 con 0x555a6fe9c610 2024-04-25T19:04:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.142+0000 7f1e0b400640 10 client.15312 handle_command_reply: tid=0 2024-04-25T19:04:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.142+0000 7f1e33bc5740 1 client.15312 shutdown 2024-04-25T19:04:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.142+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 msgr2=0x555a6fe2b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.142+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe2b130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1e18011f20 tx=0x7f1e18009310 comp rx=0 tx=0).stop 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 shutdown_connections 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x555a6fe9c610 0x555a6fe9ea00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a6fe1ca00 0x555a6fe9aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e0b400640 0 client.15312 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a6fe1a1e0 0x555a6fe2b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 --2- 172.21.15.86:0/2940395561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a6fe11820 0x555a6fe2b130 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 >> 172.21.15.86:0/2940395561 conn(0x555a6fe0dd30 msgr2=0x555a6fe0c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 shutdown_connections 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 1 -- 172.21.15.86:0/2940395561 wait complete. 2024-04-25T19:04:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.143+0000 7f1e33bc5740 20 client.15312 trim_cache size 0 max 16384 2024-04-25T19:04:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:04:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:04:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:04:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e100a4070 msgr2=0x7f1e100a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e100a4070 0x7f1e100a6530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ce5c0 tx=0x7f1e24005f70 comp rx=0 tx=0).stop 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c106fb0 msgr2=0x7f1e2c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c06ed60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009fd0 tx=0x7f1e2400ef10 comp rx=0 tx=0).stop 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 shutdown_connections 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e100a4070 0x7f1e100a6530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e2c106fb0 0x7f1e2c06ed60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e2c101b60 0x7f1e2c06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 --2- 172.21.15.86:0/2917302281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2c1011a0 0x7f1e2c073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.144+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 >> 172.21.15.86:0/2917302281 conn(0x7f1e2c076c50 msgr2=0x7f1e2c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.145+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 shutdown_connections 2024-04-25T19:04:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:06.145+0000 7f1e30c00640 1 -- 172.21.15.86:0/2917302281 wait complete. 2024-04-25T19:04:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:06 smithi183 ceph-mon[32344]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T19:04:06.734 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:06 smithi194 ceph-mon[32740]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T19:04:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:06 smithi086 ceph-mon[27783]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T19:04:07.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:07 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:04:07.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:07 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:04:07.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:07 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:04:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:07 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:04:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:07 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:04:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:07 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:04:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:07 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:04:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:07 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:04:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:07 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:04:08.076 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-04-25T19:04:08.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:04:08.592 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:04:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:08 smithi183 ceph-mon[32344]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T19:04:08.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:08 smithi086 ceph-mon[27783]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T19:04:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:08 smithi194 ceph-mon[32740]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T19:04:10.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:10 smithi086 ceph-mon[27783]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-04-25T19:04:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:10 smithi183 ceph-mon[32344]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-04-25T19:04:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:10 smithi194 ceph-mon[32740]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-04-25T19:04:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 -- 172.21.15.86:0/561912146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 msgr2=0x7fe62c10f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 --2- 172.21.15.86:0/561912146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c10f840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b0a0 tx=0x7fe62400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 -- 172.21.15.86:0/561912146 shutdown_connections 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 --2- 172.21.15.86:0/561912146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c10f840 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 --2- 172.21.15.86:0/561912146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c108970 0x7fe62c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 --2- 172.21.15.86:0/561912146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 0x7fe62c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.008+0000 7fe631000700 1 -- 172.21.15.86:0/561912146 >> 172.21.15.86:0/561912146 conn(0x7fe62c0fe4c0 msgr2=0x7fe62c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.009+0000 7fe631000700 1 -- 172.21.15.86:0/561912146 shutdown_connections 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.009+0000 7fe631000700 1 -- 172.21.15.86:0/561912146 wait complete. 2024-04-25T19:04:11.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 Processor -- start 2024-04-25T19:04:11.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 -- start start 2024-04-25T19:04:11.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 0x7fe62c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c108970 0x7fe62c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c110990 con 0x7fe62c103330 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c110810 con 0x7fe62c102970 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.010+0000 7fe631000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c110b10 con 0x7fe62c108970 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 0x7fe62c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 0x7fe62c196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60084/0 (socket says 172.21.15.86:60084) 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62b400700 1 -- 172.21.15.86:0/2793368065 learned_addr learned my addr 172.21.15.86:0/2793368065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62aa00700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62be00700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c108970 0x7fe62c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62aa00700 1 -- 172.21.15.86:0/2793368065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c108970 msgr2=0x7fe62c19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62aa00700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c108970 0x7fe62c19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62aa00700 1 -- 172.21.15.86:0/2793368065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 msgr2=0x7fe62c196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62aa00700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 0x7fe62c196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.011+0000 7fe62aa00700 1 -- 172.21.15.86:0/2793368065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c19bd70 con 0x7fe62c103330 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.012+0000 7fe62aa00700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c1972a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe61800e9f0 tx=0x7fe61800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.012+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61800cbd0 con 0x7fe62c103330 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.012+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c19c060 con 0x7fe62c103330 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.012+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1a38e0 con 0x7fe62c103330 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.012+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe61800cd70 con 0x7fe62c103330 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.012+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618010e60 con 0x7fe62c103330 2024-04-25T19:04:11.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.014+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c005330 con 0x7fe62c103330 2024-04-25T19:04:11.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.016+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe618010430 con 0x7fe62c103330 2024-04-25T19:04:11.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.018+0000 7fe623e00700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe6100a4060 0x7fe6100a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:11.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.018+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fe6180d0c30 con 0x7fe62c103330 2024-04-25T19:04:11.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.018+0000 7fe62b400700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe6100a4060 0x7fe6100a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:11.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.019+0000 7fe62b400700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe6100a4060 0x7fe6100a6520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009c90 tx=0x7fe61c009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:11.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.023+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe61809d040 con 0x7fe62c103330 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.347+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe60c0058f0 con 0x7fe62c103330 2024-04-25T19:04:11.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.348+0000 7fe623e00700 1 -- 172.21.15.86:0/2793368065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fe61809c940 con 0x7fe62c103330 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:04:11.350 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe6100a4060 msgr2=0x7fe6100a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe6100a4060 0x7fe6100a6520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009c90 tx=0x7fe61c009260 comp rx=0 tx=0).stop 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 msgr2=0x7fe62c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:11.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c1972a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe61800e9f0 tx=0x7fe61800ed00 comp rx=0 tx=0).stop 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 shutdown_connections 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe6100a4060 0x7fe6100a6520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe62c108970 0x7fe62c19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe62c102970 0x7fe62c196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 --2- 172.21.15.86:0/2793368065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62c103330 0x7fe62c1972a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 >> 172.21.15.86:0/2793368065 conn(0x7fe62c0fe4c0 msgr2=0x7fe62c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 shutdown_connections 2024-04-25T19:04:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:11.351+0000 7fe631000700 1 -- 172.21.15.86:0/2793368065 wait complete. 2024-04-25T19:04:11.354 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:04:11.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:11 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2793368065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:04:11.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:11 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2793368065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:04:12.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:11 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2793368065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:04:12.267 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 73.8 --await-for 42.0' 2024-04-25T19:04:12.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 73.8 --await-for 42.0 2024-04-25T19:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:04:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:12 smithi183 ceph-mon[32344]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T19:04:12.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:12 smithi086 ceph-mon[27783]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T19:04:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:12 smithi194 ceph-mon[32740]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T19:04:14.754 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:14 smithi086 ceph-mon[27783]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-25T19:04:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:14 smithi183 ceph-mon[32344]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 -- 172.21.15.86:0/152033023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 msgr2=0x7fd044104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 --2- 172.21.15.86:0/152033023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044104830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd030009990 tx=0x7fd030009ca0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 -- 172.21.15.86:0/152033023 shutdown_connections 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 --2- 172.21.15.86:0/152033023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd044104d70 0x7fd04410f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 --2- 172.21.15.86:0/152033023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 0x7fd044103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 --2- 172.21.15.86:0/152033023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044104830 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.918+0000 7fd04aa00700 1 -- 172.21.15.86:0/152033023 >> 172.21.15.86:0/152033023 conn(0x7fd0440fe3a0 msgr2=0x7fd0441007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.919+0000 7fd04aa00700 1 -- 172.21.15.86:0/152033023 shutdown_connections 2024-04-25T19:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.919+0000 7fd04aa00700 1 -- 172.21.15.86:0/152033023 wait complete. 2024-04-25T19:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.920+0000 7fd04aa00700 1 Processor -- start 2024-04-25T19:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 -- start start 2024-04-25T19:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 0x7fd044074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd044104d70 0x7fd0440757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd044112bc0 con 0x7fd044102c60 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd044112a40 con 0x7fd0441043b0 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd04aa00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd044112d40 con 0x7fd044104d70 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd048c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd048c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60096/0 (socket says 172.21.15.86:60096) 2024-04-25T19:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.921+0000 7fd048c00700 1 -- 172.21.15.86:0/3805656021 learned_addr learned my addr 172.21.15.86:0/3805656021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd048c00700 1 -- 172.21.15.86:0/3805656021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd044104d70 msgr2=0x7fd0440757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd048c00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd044104d70 0x7fd0440757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd049600700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 0x7fd044074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd048c00700 1 -- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 msgr2=0x7fd044074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd048c00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 0x7fd044074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd048c00700 1 -- 172.21.15.86:0/3805656021 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0441a74d0 con 0x7fd0441043b0 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.922+0000 7fd049600700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 0x7fd044074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.923+0000 7fd048c00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044075280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd030005910 tx=0x7fd03000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.923+0000 7fd042000700 1 -- 172.21.15.86:0/3805656021 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030027070 con 0x7fd0441043b0 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.923+0000 7fd042000700 1 -- 172.21.15.86:0/3805656021 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd030022470 con 0x7fd0441043b0 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.923+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0441a7760 con 0x7fd0441043b0 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.923+0000 7fd042000700 1 -- 172.21.15.86:0/3805656021 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030017650 con 0x7fd0441043b0 2024-04-25T19:04:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.923+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441a7c90 con 0x7fd0441043b0 2024-04-25T19:04:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.928+0000 7fd042000700 1 -- 172.21.15.86:0/3805656021 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fd03002a070 con 0x7fd0441043b0 2024-04-25T19:04:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.929+0000 7fd056966b80 1 Processor -- start 2024-04-25T19:04:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 -- start start 2024-04-25T19:04:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 0x560d1ed06310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed1c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d1ec59020 con 0x560d1ed06850 2024-04-25T19:04:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d1ebff8c0 con 0x560d1ed19f10 2024-04-25T19:04:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.930+0000 7fd056966b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560d1ec54fe0 con 0x560d1ed05f30 2024-04-25T19:04:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd04a000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 0x560d1ed06310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd048c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed1c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd04a000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 0x560d1ed06310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39076/0 (socket says 172.21.15.86:39076) 2024-04-25T19:04:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd048c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed1c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60104/0 (socket says 172.21.15.86:60104) 2024-04-25T19:04:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd04a000700 1 -- 172.21.15.86:0/3667902574 learned_addr learned my addr 172.21.15.86:0/3667902574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd049600700 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd04a000700 1 -- 172.21.15.86:0/3667902574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 msgr2=0x560d1ed1c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd04a000700 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed1c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.931+0000 7fd04a000700 1 -- 172.21.15.86:0/3667902574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 msgr2=0x560d1ed199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.932+0000 7fd042000700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd02c0a4080 0x7fd02c0a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.932+0000 7fd04a000700 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.932+0000 7fd04a000700 1 -- 172.21.15.86:0/3667902574 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1ebf5810 con 0x560d1ed05f30 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.932+0000 7fd042000700 1 -- 172.21.15.86:0/3805656021 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd0300d17e0 con 0x7fd0441043b0 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.932+0000 7fd049600700 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.932+0000 7fd04a000700 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 0x560d1ed06310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e380 tx=0x7fd03c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b57f1b9c8202698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.933+0000 7fd036a00700 1 -- 172.21.15.86:0/3667902574 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00ee30 con 0x560d1ed05f30 2024-04-25T19:04:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.933+0000 7fd048c00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd02c0a4080 0x7fd02c0a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.933+0000 7fd036a00700 1 -- 172.21.15.86:0/3667902574 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c004bb0 con 0x560d1ed05f30 2024-04-25T19:04:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.933+0000 7fd036a00700 1 -- 172.21.15.86:0/3667902574 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c004300 con 0x560d1ed05f30 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 -- 172.21.15.86:0/3667902574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 msgr2=0x560d1ed06310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 0x560d1ed06310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e380 tx=0x7fd03c00b040 comp rx=0 tx=0).stop 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd048c00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd02c0a4080 0x7fd02c0a6540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ced00 tx=0x7fd0300d7530 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 -- 172.21.15.86:0/3667902574 shutdown_connections 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed1c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed05f30 0x560d1ed06310 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 --2- 172.21.15.86:0/3667902574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.934+0000 7fd056966b80 1 -- 172.21.15.86:0/3667902574 >> 172.21.15.86:0/3667902574 conn(0x560d1ec350e0 msgr2=0x560d1ed02480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.935+0000 7fd056966b80 1 -- 172.21.15.86:0/3667902574 shutdown_connections 2024-04-25T19:04:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.935+0000 7fd056966b80 1 -- 172.21.15.86:0/3667902574 wait complete. 2024-04-25T19:04:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.935+0000 7fd056966b80 1 -- start start 2024-04-25T19:04:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.935+0000 7fd056966b80 1 Processor -- start 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed16f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d1ed1c940 con 0x560d1ed06850 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d1ec129f0 con 0x560d1ed05f30 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560d1ed1cac0 con 0x560d1ed19f10 2024-04-25T19:04:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd056966b80 10 client.0 _ll_register_callbacks cb 0x560d1ebf18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd04a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd04a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed196c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60108/0 (socket says 172.21.15.86:60108) 2024-04-25T19:04:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd04a000700 1 -- 172.21.15.86:0/819114925 learned_addr learned my addr 172.21.15.86:0/819114925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.936+0000 7fd04a000700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed16f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd049600700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd04a000700 1 -- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 msgr2=0x560d1ed196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd04a000700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd04a000700 1 -- 172.21.15.86:0/819114925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 msgr2=0x560d1ed169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd04a000700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd04a000700 1 -- 172.21.15.86:0/819114925 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1eba6a60 con 0x560d1ed19f10 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.937+0000 7fd049600700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.938+0000 7fd04a000700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed16f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e350 tx=0x7fd03c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c6e03fd83d83e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.938+0000 7fd02b400700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:04:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.938+0000 7fd02b400700 1 -- 172.21.15.86:0/819114925 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0040a0 con 0x560d1ed19f10 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.938+0000 7fd02b400700 1 -- 172.21.15.86:0/819114925 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd03c004240 con 0x560d1ed19f10 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.938+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd03c004bb0 con 0x560d1ed19f10 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.938+0000 7fd02b400700 1 -- 172.21.15.86:0/819114925 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd03c004440 con 0x560d1ed19f10 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.939+0000 7fd056966b80 10 client.25432 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.939+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d1ebefc80 con 0x560d1ed19f10 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.939+0000 7fd02b400700 1 -- 172.21.15.86:0/819114925 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c004bb0 con 0x560d1ed19f10 2024-04-25T19:04:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd02b400700 1 -- 172.21.15.86:0/819114925 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd03c01c020 con 0x560d1ed19f10 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd056966b80 10 client.25432 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd056966b80 10 client.25432 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd056966b80 20 client.25432 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd056966b80 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed295f0 0x560d1ed2b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd056966b80 4 client.25432 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d1ec66810 con 0x560d1ed295f0 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.940+0000 7fd049600700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed295f0 0x560d1ed2b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.941+0000 7fd049600700 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed295f0 0x560d1ed2b9e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dce9022917855d1 server_cookie=aaf27036f9835170 in_seq=0 out_seq=0 2024-04-25T19:04:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.941+0000 7fd02b400700 10 client.25432 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd02b400700 1 -- 172.21.15.86:0/819114925 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d1ec66810 con 0x560d1ed295f0 2024-04-25T19:04:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd02b400700 10 client.25432 handle_command_reply: tid=0 2024-04-25T19:04:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 client.25432 shutdown 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 msgr2=0x560d1ed16f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed16f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e350 tx=0x7fd03c009b60 comp rx=0 tx=0).stop 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 shutdown_connections 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed295f0 0x560d1ed2b9e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed16f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 --2- 172.21.15.86:0/819114925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd02b400700 0 client.25432 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.942+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 >> 172.21.15.86:0/819114925 conn(0x560d1ec350e0 msgr2=0x560d1ed02070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.943+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 shutdown_connections 2024-04-25T19:04:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.943+0000 7fd056966b80 1 -- 172.21.15.86:0/819114925 wait complete. 2024-04-25T19:04:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.943+0000 7fd056966b80 20 client.25432 trim_cache size 0 max 16384 2024-04-25T19:04:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.965+0000 7fd056966b80 1 Processor -- start 2024-04-25T19:04:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.965+0000 7fd056966b80 1 -- start start 2024-04-25T19:04:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd056966b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed12200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd056966b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed13ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd056966b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd056966b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441aafc0 con 0x560d1ed06850 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd056966b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441acb00 con 0x560d1ed05f30 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd056966b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441ab690 con 0x560d1ed19f10 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd04a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed12200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd04a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed12200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60120/0 (socket says 172.21.15.86:60120) 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd04a000700 1 -- 172.21.15.86:0/2236301147 learned_addr learned my addr 172.21.15.86:0/2236301147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.966+0000 7fd04a000700 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd049600700 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed13ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd04a000700 1 -- 172.21.15.86:0/2236301147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 msgr2=0x560d1ed141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd04a000700 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd04a000700 1 -- 172.21.15.86:0/2236301147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 msgr2=0x560d1ed13ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd04a000700 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed13ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd04a000700 1 -- 172.21.15.86:0/2236301147 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1ed06ea0 con 0x560d1ed05f30 2024-04-25T19:04:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.967+0000 7fd04a000700 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed12200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007da0 tx=0x7fd03c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f09b80ad215404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.968+0000 7fd036000700 1 -- 172.21.15.86:0/2236301147 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c009320 con 0x560d1ed05f30 2024-04-25T19:04:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.968+0000 7fd036000700 1 -- 172.21.15.86:0/2236301147 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c004bb0 con 0x560d1ed05f30 2024-04-25T19:04:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.968+0000 7fd036000700 1 -- 172.21.15.86:0/2236301147 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c007760 con 0x560d1ed05f30 2024-04-25T19:04:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 -- 172.21.15.86:0/2236301147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 msgr2=0x560d1ed12200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed12200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007da0 tx=0x7fd03c000f40 comp rx=0 tx=0).stop 2024-04-25T19:04:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 -- 172.21.15.86:0/2236301147 shutdown_connections 2024-04-25T19:04:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed19f10 0x560d1ed141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed05f30 0x560d1ed12200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 --2- 172.21.15.86:0/2236301147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed06850 0x560d1ed13ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 -- 172.21.15.86:0/2236301147 >> 172.21.15.86:0/2236301147 conn(0x560d1ec350e0 msgr2=0x560d1ed16e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 -- 172.21.15.86:0/2236301147 shutdown_connections 2024-04-25T19:04:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.969+0000 7fd056966b80 1 -- 172.21.15.86:0/2236301147 wait complete. 2024-04-25T19:04:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.970+0000 7fd056966b80 1 -- start start 2024-04-25T19:04:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.970+0000 7fd056966b80 1 Processor -- start 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 0x560d1ed2eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed06850 0x560d1ed2ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed2d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441ac430 con 0x560d1ed05f30 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd04a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 0x560d1ed2eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd04a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 0x560d1ed2eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57860/0 (socket says 172.21.15.86:57860) 2024-04-25T19:04:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd04a000700 1 -- 172.21.15.86:0/2990153702 learned_addr learned my addr 172.21.15.86:0/2990153702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd049600700 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed06850 0x560d1ed2ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441abd60 con 0x560d1ed19f10 2024-04-25T19:04:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd04419cfa0 con 0x560d1ed06850 2024-04-25T19:04:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.971+0000 7fd056966b80 10 client.0 _ll_register_callbacks cb 0x560d1ebf18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.972+0000 7fd04a000700 1 -- 172.21.15.86:0/2990153702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed06850 msgr2=0x560d1ed2ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.972+0000 7fd04a000700 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed06850 0x560d1ed2ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.972+0000 7fd04a000700 1 -- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 msgr2=0x560d1ed2d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.972+0000 7fd04a000700 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed2d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.972+0000 7fd04a000700 1 -- 172.21.15.86:0/2990153702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1ed2d910 con 0x560d1ed05f30 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.973+0000 7fd04a000700 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 0x560d1ed2eab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007e20 tx=0x7fd03c007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231d5a1c2b990769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.973+0000 7fd02b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.973+0000 7fd02b400700 1 -- 172.21.15.86:0/2990153702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c038030 con 0x560d1ed05f30 2024-04-25T19:04:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.974+0000 7fd02b400700 1 -- 172.21.15.86:0/2990153702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd03c019040 con 0x560d1ed05f30 2024-04-25T19:04:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.974+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd03c004bb0 con 0x560d1ed05f30 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.974+0000 7fd02b400700 1 -- 172.21.15.86:0/2990153702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd03c0040a0 con 0x560d1ed05f30 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.974+0000 7fd056966b80 10 client.15330 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.974+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d1ed90540 con 0x560d1ed05f30 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd02b400700 1 -- 172.21.15.86:0/2990153702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c004bb0 con 0x560d1ed05f30 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd02b400700 1 -- 172.21.15.86:0/2990153702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd03c033040 con 0x560d1ed05f30 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd056966b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd056966b80 10 client.15330 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd056966b80 20 client.15330 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd056966b80 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed90b50 0x560d1ed92f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd056966b80 4 client.15330 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 73.8, "await_for": 42.0}] 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 73.8, "await_for": 42.0}) -- 0x560d1ec608b0 con 0x560d1ed90b50 2024-04-25T19:04:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.975+0000 7fd049600700 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed90b50 0x560d1ed92f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.976+0000 7fd02b400700 10 client.15330 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.976+0000 7fd049600700 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed90b50 0x560d1ed92f70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e7fc2f891b4dc73 server_cookie=dfb55c7278bec68e in_seq=0 out_seq=0 2024-04-25T19:04:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:14 smithi194 ceph-mon[32740]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-25T19:04:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.990+0000 7fd02b400700 1 -- 172.21.15.86:0/2990153702 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x560d1ec608b0 con 0x560d1ed90b50 2024-04-25T19:04:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.990+0000 7fd02b400700 10 client.15330 handle_command_reply: tid=0 2024-04-25T19:04:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.990+0000 7fd056966b80 1 client.15330 shutdown 2024-04-25T19:04:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.990+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 msgr2=0x560d1ed2eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.990+0000 7fd056966b80 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 0x560d1ed2eab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007e20 tx=0x7fd03c007d20 comp rx=0 tx=0).stop 2024-04-25T19:04:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 shutdown_connections 2024-04-25T19:04:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560d1ed90b50 0x560d1ed92f70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d1ed19f10 0x560d1ed2d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d1ed05f30 0x560d1ed2eab0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 --2- 172.21.15.86:0/2990153702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1ed06850 0x560d1ed2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 >> 172.21.15.86:0/2990153702 conn(0x560d1ec350e0 msgr2=0x560d1ed181f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd02b400700 0 client.15330 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 shutdown_connections 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 1 -- 172.21.15.86:0/2990153702 wait complete. 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.991+0000 7fd056966b80 20 client.15330 trim_cache size 0 max 16384 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 2, 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "526b8fed": { 2024-04-25T19:04:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2, 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 73.8, 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:04:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:04:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.992+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd02c0a4080 msgr2=0x7fd02c0a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.992+0000 7fd04aa00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd02c0a4080 0x7fd02c0a6540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ced00 tx=0x7fd0300d7530 comp rx=0 tx=0).stop 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.992+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 msgr2=0x7fd044075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.992+0000 7fd04aa00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044075280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd030005910 tx=0x7fd03000b670 comp rx=0 tx=0).stop 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 shutdown_connections 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd02c0a4080 0x7fd02c0a6540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd044104d70 0x7fd0440757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd044102c60 0x7fd044074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 --2- 172.21.15.86:0/3805656021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0441043b0 0x7fd044075280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 >> 172.21.15.86:0/3805656021 conn(0x7fd0440fe3a0 msgr2=0x7fd044108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 shutdown_connections 2024-04-25T19:04:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:14.993+0000 7fd04aa00700 1 -- 172.21.15.86:0/3805656021 wait complete. 2024-04-25T19:04:16.047 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '526b8fed', quiesce took 8.0 seconds. Will release after: 13.8 2024-04-25T19:04:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:16 smithi183 ceph-mon[32344]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-04-25T19:04:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:16 smithi194 ceph-mon[32740]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-04-25T19:04:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:16 smithi086 ceph-mon[27783]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-04-25T19:04:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:18 smithi183 ceph-mon[32344]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T19:04:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:18 smithi194 ceph-mon[32740]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T19:04:19.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:18 smithi086 ceph-mon[27783]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 -- 172.21.15.86:0/1043893125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 msgr2=0x7f2284110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 --2- 172.21.15.86:0/1043893125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f2284110e90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c02a8c0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 -- 172.21.15.86:0/1043893125 shutdown_connections 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 --2- 172.21.15.86:0/1043893125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f2284110e90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 --2- 172.21.15.86:0/1043893125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2284104ba0 0x7f2284108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 --2- 172.21.15.86:0/1043893125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.274+0000 7f2289000640 1 -- 172.21.15.86:0/1043893125 >> 172.21.15.86:0/1043893125 conn(0x7f22840ffa30 msgr2=0x7f2284101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.275+0000 7f2289000640 1 -- 172.21.15.86:0/1043893125 shutdown_connections 2024-04-25T19:04:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.275+0000 7f2289000640 1 -- 172.21.15.86:0/1043893125 wait complete. 2024-04-25T19:04:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.276+0000 7f2289000640 1 Processor -- start 2024-04-25T19:04:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.276+0000 7f2289000640 1 -- start start 2024-04-25T19:04:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2289000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2289000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f22841a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2289000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f228419a620 0x7f228419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2289000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2284114260 con 0x7f22841092c0 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2289000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22841140e0 con 0x7f228419a620 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2289000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22841143e0 con 0x7f22841041f0 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39108/0 (socket says 172.21.15.86:39108) 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 -- 172.21.15.86:0/2785715310 learned_addr learned my addr 172.21.15.86:0/2785715310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2282a00640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f22841a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283e00640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f228419a620 0x7f228419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 -- 172.21.15.86:0/2785715310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f228419a620 msgr2=0x7f228419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f228419a620 0x7f228419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 -- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 msgr2=0x7f22841a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f22841a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2283400640 1 -- 172.21.15.86:0/2785715310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228419b140 con 0x7f22841041f0 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.277+0000 7f2282a00640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f22841a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.278+0000 7f2283400640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841a0470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f227400d950 tx=0x7f227400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.278+0000 7f227be00640 1 -- 172.21.15.86:0/2785715310 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274017030 con 0x7f22841041f0 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.278+0000 7f227be00640 1 -- 172.21.15.86:0/2785715310 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2274004020 con 0x7f22841041f0 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.278+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228419b430 con 0x7f22841041f0 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.278+0000 7f227be00640 1 -- 172.21.15.86:0/2785715310 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740047a0 con 0x7f22841041f0 2024-04-25T19:04:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.278+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841a76d0 con 0x7f22841041f0 2024-04-25T19:04:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.283+0000 7f228d451740 1 Processor -- start 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.283+0000 7f228d451740 1 -- start start 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.283+0000 7f227be00640 1 -- 172.21.15.86:0/2785715310 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2274009a30 con 0x7f22841041f0 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.283+0000 7f228d451740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e93fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.283+0000 7f228d451740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e952e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f228d451740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e955790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f228d451740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558f7e8a5900 con 0x558f7e93f8f0 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f228d451740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f7e84bef0 con 0x558f7e940210 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f228d451740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f7e8a18c0 con 0x558f7e9533a0 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e955790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e955790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39120/0 (socket says 172.21.15.86:39120) 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 -- 172.21.15.86:0/1973887590 learned_addr learned my addr 172.21.15.86:0/1973887590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2283400640 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e952e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2283e00640 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e93fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 -- 172.21.15.86:0/1973887590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 msgr2=0x558f7e952e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e952e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 -- 172.21.15.86:0/1973887590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 msgr2=0x558f7e93fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e93fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.284+0000 7f2282a00640 1 -- 172.21.15.86:0/1973887590 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f7e5b8210 con 0x558f7e9533a0 2024-04-25T19:04:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.285+0000 7f2282a00640 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e955790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f226c009bf0 tx=0x7f226c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e46953d7ecc350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.285+0000 7f2278c00640 1 -- 172.21.15.86:0/1973887590 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c012450 con 0x558f7e9533a0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.285+0000 7f227be00640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22680a3f40 0x7f22680a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.285+0000 7f2278c00640 1 -- 172.21.15.86:0/1973887590 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c012aa0 con 0x558f7e9533a0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.285+0000 7f2278c00640 1 -- 172.21.15.86:0/1973887590 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c00fad0 con 0x558f7e9533a0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.285+0000 7f227be00640 1 -- 172.21.15.86:0/2785715310 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f22740d05d0 con 0x7f22841041f0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.286+0000 7f2283e00640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22680a3f40 0x7f22680a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.286+0000 7f228d451740 1 -- 172.21.15.86:0/1973887590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 msgr2=0x558f7e955790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.286+0000 7f228d451740 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e955790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f226c009bf0 tx=0x7f226c0117b0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f2283e00640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22680a3f40 0x7f22680a6400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f227c005fe0 tx=0x7f227c004a40 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 -- 172.21.15.86:0/1973887590 shutdown_connections 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e955790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e952e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 --2- 172.21.15.86:0/1973887590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e93fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 -- 172.21.15.86:0/1973887590 >> 172.21.15.86:0/1973887590 conn(0x558f7e93bec0 msgr2=0x558f7e93c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 -- 172.21.15.86:0/1973887590 shutdown_connections 2024-04-25T19:04:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 -- 172.21.15.86:0/1973887590 wait complete. 2024-04-25T19:04:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.287+0000 7f228d451740 1 -- start start 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 Processor -- start 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e94ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e94d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e952b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558f7e942890 con 0x558f7e93f8f0 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f7e85f310 con 0x558f7e940210 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f7e942a10 con 0x558f7e9533a0 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.288+0000 7f228d451740 10 client.0 _ll_register_callbacks cb 0x558f7e82dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2283400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e94ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e94d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e94d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60162/0 (socket says 172.21.15.86:60162) 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 -- 172.21.15.86:0/2425562151 learned_addr learned my addr 172.21.15.86:0/2425562151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2283e00640 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e952b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 -- 172.21.15.86:0/2425562151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 msgr2=0x558f7e952b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e952b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 -- 172.21.15.86:0/2425562151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 msgr2=0x558f7e94ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e94ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 -- 172.21.15.86:0/2425562151 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f7e684620 con 0x558f7e940210 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2282a00640 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e94d370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f226c00f670 tx=0x7f226c012f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e42390c6256a646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.289+0000 7f2271600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.290+0000 7f2271600640 1 -- 172.21.15.86:0/2425562151 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c029020 con 0x558f7e940210 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.290+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226c012aa0 con 0x558f7e940210 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.290+0000 7f2271600640 1 -- 172.21.15.86:0/2425562151 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f226c00f870 con 0x558f7e940210 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.290+0000 7f2271600640 1 -- 172.21.15.86:0/2425562151 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f226c034050 con 0x558f7e940210 2024-04-25T19:04:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.290+0000 7f228d451740 10 client.25460 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.290+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f7e96aa10 con 0x558f7e940210 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f2271600640 1 -- 172.21.15.86:0/2425562151 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c012aa0 con 0x558f7e940210 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f2271600640 1 -- 172.21.15.86:0/2425562151 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f226c0035f0 con 0x558f7e940210 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f228d451740 10 client.25460 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f228d451740 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f228d451740 20 client.25460 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f228d451740 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e96be20 0x558f7e96e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f228d451740 4 client.25460 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f7e8b98b0 con 0x558f7e96be20 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.291+0000 7f2283400640 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e96be20 0x558f7e96e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.292+0000 7f2283400640 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e96be20 0x558f7e96e210 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720e189d2ead319e server_cookie=4d513404e718ee5a in_seq=0 out_seq=0 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.292+0000 7f2271600640 10 client.25460 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f2271600640 1 -- 172.21.15.86:0/2425562151 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f7e8b98b0 con 0x558f7e96be20 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f2271600640 10 client.25460 handle_command_reply: tid=0 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 client.25460 shutdown 2024-04-25T19:04:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 msgr2=0x558f7e94d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e94d370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f226c00f670 tx=0x7f226c012f00 comp rx=0 tx=0).stop 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 shutdown_connections 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e96be20 0x558f7e96e210 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e9533a0 0x558f7e952b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e94d370 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 --2- 172.21.15.86:0/2425562151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e94ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 >> 172.21.15.86:0/2425562151 conn(0x558f7e93bec0 msgr2=0x558f7e941c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f2271600640 0 client.25460 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.293+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 shutdown_connections 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.294+0000 7f228d451740 1 -- 172.21.15.86:0/2425562151 wait complete. 2024-04-25T19:04:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.294+0000 7f228d451740 20 client.25460 trim_cache size 0 max 16384 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 Processor -- start 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 -- start start 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e952ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e940210 0x558f7e973840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e9533a0 0x558f7e973d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22841ab9f0 con 0x558f7e93f8f0 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22841ad530 con 0x558f7e9533a0 2024-04-25T19:04:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.309+0000 7f228d451740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22841ac0c0 con 0x558f7e940210 2024-04-25T19:04:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e952ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e952ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57910/0 (socket says 172.21.15.86:57910) 2024-04-25T19:04:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 -- 172.21.15.86:0/2312412341 learned_addr learned my addr 172.21.15.86:0/2312412341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2282a00640 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e940210 0x558f7e973840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283e00640 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e9533a0 0x558f7e973d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 -- 172.21.15.86:0/2312412341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e940210 msgr2=0x558f7e973840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e940210 0x558f7e973840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 -- 172.21.15.86:0/2312412341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e9533a0 msgr2=0x558f7e973d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e9533a0 0x558f7e973d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283400640 1 -- 172.21.15.86:0/2312412341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f7e940040 con 0x558f7e93f8f0 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.310+0000 7f2283e00640 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e9533a0 0x558f7e973d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.311+0000 7f2283400640 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e952ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22740cdb80 tx=0x7f2274004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab14096ab4bb6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.311+0000 7f2273e00640 1 -- 172.21.15.86:0/2312412341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227409be20 con 0x558f7e93f8f0 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.311+0000 7f2273e00640 1 -- 172.21.15.86:0/2312412341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2274096e50 con 0x558f7e93f8f0 2024-04-25T19:04:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.311+0000 7f2273e00640 1 -- 172.21.15.86:0/2312412341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227409bc40 con 0x558f7e93f8f0 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.311+0000 7f228d451740 1 -- 172.21.15.86:0/2312412341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 msgr2=0x558f7e952ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.311+0000 7f228d451740 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e952ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22740cdb80 tx=0x7f2274004710 comp rx=0 tx=0).stop 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 -- 172.21.15.86:0/2312412341 shutdown_connections 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e9533a0 0x558f7e973d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e940210 0x558f7e973840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 --2- 172.21.15.86:0/2312412341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e93f8f0 0x558f7e952ac0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 -- 172.21.15.86:0/2312412341 >> 172.21.15.86:0/2312412341 conn(0x558f7e93bec0 msgr2=0x558f7e9433f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 -- 172.21.15.86:0/2312412341 shutdown_connections 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 -- 172.21.15.86:0/2312412341 wait complete. 2024-04-25T19:04:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 -- start start 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.312+0000 7f228d451740 1 Processor -- start 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e93f8f0 0x558f7e97a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e97adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e9533a0 0x558f7e97b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22841ace60 con 0x558f7e9533a0 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22841ac790 con 0x558f7e940210 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22841a99d0 con 0x558f7e93f8f0 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f228d451740 10 client.0 _ll_register_callbacks cb 0x7f226c005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f2283e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e9533a0 0x558f7e97b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f2282a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e97adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f2282a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e97adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60186/0 (socket says 172.21.15.86:60186) 2024-04-25T19:04:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f2282a00640 1 -- 172.21.15.86:0/3559276979 learned_addr learned my addr 172.21.15.86:0/3559276979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f2283e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e9533a0 0x558f7e97b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57920/0 (socket says 172.21.15.86:57920) 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.313+0000 7f2283400640 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e93f8f0 0x558f7e97a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2282a00640 1 -- 172.21.15.86:0/3559276979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e93f8f0 msgr2=0x558f7e97a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2282a00640 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e93f8f0 0x558f7e97a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2282a00640 1 -- 172.21.15.86:0/3559276979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e9533a0 msgr2=0x558f7e97b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2282a00640 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e9533a0 0x558f7e97b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2282a00640 1 -- 172.21.15.86:0/3559276979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f7e9d0bc0 con 0x558f7e940210 2024-04-25T19:04:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2282a00640 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e97adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f226c011cb0 tx=0x7f226c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee9b01064a6add0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2271600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:04:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2271600640 1 -- 172.21.15.86:0/3559276979 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c029070 con 0x558f7e940210 2024-04-25T19:04:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2274096e50 con 0x558f7e940210 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.314+0000 7f2271600640 1 -- 172.21.15.86:0/3559276979 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f226c00f6a0 con 0x558f7e940210 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f2271600640 1 -- 172.21.15.86:0/3559276979 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f226c00fad0 con 0x558f7e940210 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 10 client.25466 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f7e9d1e90 con 0x558f7e940210 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f2271600640 1 -- 172.21.15.86:0/3559276979 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2274096e50 con 0x558f7e940210 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f2271600640 1 -- 172.21.15.86:0/3559276979 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f226c012450 con 0x558f7e940210 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 10 client.25466 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 10 client.25466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 20 client.25466 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e9d2540 0x558f7e9d49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 4 client.25466 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.315+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f7e8b3320 con 0x558f7e9d2540 2024-04-25T19:04:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.316+0000 7f2283400640 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e9d2540 0x558f7e9d49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.316+0000 7f2283400640 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e9d2540 0x558f7e9d49e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f3870b2283768c server_cookie=e436a76def7d6c06 in_seq=0 out_seq=0 2024-04-25T19:04:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.316+0000 7f2271600640 10 client.25466 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.321+0000 7f2271600640 1 -- 172.21.15.86:0/3559276979 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f7e8b3320 con 0x558f7e9d2540 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.321+0000 7f2271600640 10 client.25466 handle_command_reply: tid=0 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 client.25466 shutdown 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 msgr2=0x558f7e97adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e97adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f226c011cb0 tx=0x7f226c005f80 comp rx=0 tx=0).stop 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 shutdown_connections 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558f7e9d2540 0x558f7e9d49e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558f7e9533a0 0x558f7e97b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f7e940210 0x558f7e97adf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 --2- 172.21.15.86:0/3559276979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f7e93f8f0 0x558f7e97a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f2271600640 0 client.25466 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 >> 172.21.15.86:0/3559276979 conn(0x558f7e93bec0 msgr2=0x558f7e93a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 shutdown_connections 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 1 -- 172.21.15.86:0/3559276979 wait complete. 2024-04-25T19:04:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.322+0000 7f228d451740 20 client.25466 trim_cache size 0 max 16384 2024-04-25T19:04:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:04:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:04:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:04:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22680a3f40 msgr2=0x7f22680a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22680a3f40 0x7f22680a6400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f227c005fe0 tx=0x7f227c004a40 comp rx=0 tx=0).stop 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 msgr2=0x7f22841a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841a0470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f227400d950 tx=0x7f227400de10 comp rx=0 tx=0).stop 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 shutdown_connections 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f22680a3f40 0x7f22680a6400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f228419a620 0x7f228419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22841092c0 0x7f22841a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 --2- 172.21.15.86:0/2785715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22841041f0 0x7f22841a0470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 >> 172.21.15.86:0/2785715310 conn(0x7f22840ffa30 msgr2=0x7f2284101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 shutdown_connections 2024-04-25T19:04:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:19.323+0000 7f2289000640 1 -- 172.21.15.86:0/2785715310 wait complete. 2024-04-25T19:04:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:20 smithi183 ceph-mon[32344]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-25T19:04:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:20 smithi194 ceph-mon[32740]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-25T19:04:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:20 smithi086 ceph-mon[27783]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-04-25T19:04:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:22 smithi183 ceph-mon[32344]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:04:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:22 smithi194 ceph-mon[32740]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:04:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:22 smithi086 ceph-mon[27783]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:04:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:24 smithi183 ceph-mon[32344]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T19:04:24.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:24 smithi194 ceph-mon[32740]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T19:04:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:24 smithi086 ceph-mon[27783]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T19:04:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:26 smithi183 ceph-mon[32344]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T19:04:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:26 smithi194 ceph-mon[32740]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T19:04:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:26 smithi086 ceph-mon[27783]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-25T19:04:27.572 INFO:tasks.workunit.client.0.smithi086.stderr:Updating files: 67% (8391/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr:Note: switching to '7797745e6d8b3ac036504dc61491d368615024da'. 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T19:04:27.980 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr: git switch -c 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr:Or undo this operation with: 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr: git switch - 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-25T19:04:27.981 INFO:tasks.workunit.client.0.smithi086.stderr:HEAD is now at 7797745e6d8 qa/suites/fs: add quiescer to the fs suite 2024-04-25T19:04:27.984 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T19:04:28.044 INFO:tasks.workunit.client.0.smithi086.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T19:04:28.050 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T19:04:28.051 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T19:04:28.939 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T19:04:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:28 smithi194 ceph-mon[32740]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T19:04:29.039 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T19:04:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:28 smithi086 ceph-mon[27783]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T19:04:29.118 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T19:04:29.124 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T19:04:29.125 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T19:04:29.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:28 smithi183 ceph-mon[32344]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T19:04:29.208 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T19:04:29.218 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T19:04:29.218 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T19:04:29.264 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-25T19:04:29.265 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-25T19:04:29.265 DEBUG:teuthology.orchestra.run.smithi086:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7797745e6d8b3ac036504dc61491d368615024da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T19:04:29.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:04:30.284 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:04:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:30 smithi194 ceph-mon[32740]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-25T19:04:30.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:30 smithi194 ceph-mon[32740]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T19:04:31.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:30 smithi086 ceph-mon[27783]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-25T19:04:31.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:30 smithi086 ceph-mon[27783]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T19:04:31.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:30 smithi183 ceph-mon[32344]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-25T19:04:31.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:30 smithi183 ceph-mon[32344]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T19:04:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.465+0000 7fcde2c00640 1 -- 172.21.15.86:0/1776547838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 msgr2=0x7fcddc10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.465+0000 7fcde2c00640 1 --2- 172.21.15.86:0/1776547838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc10e9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009be0 tx=0x7fcdd402a7e0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 -- 172.21.15.86:0/1776547838 shutdown_connections 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 --2- 172.21.15.86:0/1776547838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc10e9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 --2- 172.21.15.86:0/1776547838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc103b00 0x7fcddc10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 --2- 172.21.15.86:0/1776547838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc1031e0 0x7fcddc1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 -- 172.21.15.86:0/1776547838 >> 172.21.15.86:0/1776547838 conn(0x7fcddc077fd0 msgr2=0x7fcddc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 -- 172.21.15.86:0/1776547838 shutdown_connections 2024-04-25T19:04:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.466+0000 7fcde2c00640 1 -- 172.21.15.86:0/1776547838 wait complete. 2024-04-25T19:04:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.467+0000 7fcde2c00640 1 Processor -- start 2024-04-25T19:04:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.467+0000 7fcde2c00640 1 -- start start 2024-04-25T19:04:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc1031e0 0x7fcddc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde2c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc103b00 0x7fcddc1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde2c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1121b0 con 0x7fcddc1031e0 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde2c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc112030 con 0x7fcddc103b00 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde2c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc112330 con 0x7fcddc10c520 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc1031e0 0x7fcddc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc1031e0 0x7fcddc1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52134/0 (socket says 172.21.15.86:52134) 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcddbe00640 1 -- 172.21.15.86:0/33649033 learned_addr learned my addr 172.21.15.86:0/33649033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.468+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc103b00 0x7fcddc1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde0c00640 1 -- 172.21.15.86:0/33649033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc103b00 msgr2=0x7fcddc1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde0c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc103b00 0x7fcddc1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde0c00640 1 -- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc1031e0 msgr2=0x7fcddc1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde0c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc1031e0 0x7fcddc1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde0c00640 1 -- 172.21.15.86:0/33649033 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc19b020 con 0x7fcddc10c520 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde0c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc19a850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009b30 tx=0x7fcdd4003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcdd9600640 1 -- 172.21.15.86:0/33649033 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4041020 con 0x7fcddc10c520 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcdd9600640 1 -- 172.21.15.86:0/33649033 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd40318f0 con 0x7fcddc10c520 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcdd9600640 1 -- 172.21.15.86:0/33649033 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4036040 con 0x7fcddc10c520 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.469+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc19b2b0 con 0x7fcddc10c520 2024-04-25T19:04:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.470+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1a74e0 con 0x7fcddc10c520 2024-04-25T19:04:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.474+0000 7fcde5a1c740 1 Processor -- start 2024-04-25T19:04:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.474+0000 7fcdd9600640 1 -- 172.21.15.86:0/33649033 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcdd4044020 con 0x7fcddc10c520 2024-04-25T19:04:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.474+0000 7fcde5a1c740 1 -- start start 2024-04-25T19:04:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab7820 0x560499ab7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ac24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ac2a00 0x560499ac4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560499a1d900 con 0x560499ab7820 2024-04-25T19:04:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604999c3ef0 con 0x560499ab8140 2024-04-25T19:04:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560499a198c0 con 0x560499ac2a00 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab7820 0x560499ab7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ac24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab7820 0x560499ab7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52150/0 (socket says 172.21.15.86:52150) 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcdd3e00640 1 -- 172.21.15.86:0/1906802373 learned_addr learned my addr 172.21.15.86:0/1906802373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ac24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:54114/0 (socket says 172.21.15.86:54114) 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ac2a00 0x560499ac4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcddbe00640 1 -- 172.21.15.86:0/1906802373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ac2a00 msgr2=0x560499ac4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.475+0000 7fcddbe00640 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ac2a00 0x560499ac4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcddbe00640 1 -- 172.21.15.86:0/1906802373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab7820 msgr2=0x560499ab7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcddbe00640 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab7820 0x560499ab7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcddbe00640 1 -- 172.21.15.86:0/1906802373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560499730210 con 0x560499ab8140 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcddbe00640 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ac24c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009cf0 tx=0x7fcdcc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30bc0e49d1acb4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcdd9600640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdc00a3e40 0x7fcdc00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcdd1200640 1 -- 172.21.15.86:0/1906802373 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc005220 con 0x560499ab8140 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcdd9600640 1 -- 172.21.15.86:0/33649033 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fcdd40f1240 con 0x7fcddc10c520 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdc00a3e40 0x7fcdc00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcdd1200640 1 -- 172.21.15.86:0/1906802373 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc004c50 con 0x560499ab8140 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.476+0000 7fcdd1200640 1 -- 172.21.15.86:0/1906802373 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0197f0 con 0x560499ab8140 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1906802373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 msgr2=0x560499ac24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ac24c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009cf0 tx=0x7fcdcc011810 comp rx=0 tx=0).stop 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdc00a3e40 0x7fcdc00a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x560499ab5150 tx=0x7fcdc4015040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1906802373 shutdown_connections 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ac2a00 0x560499ac4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ac24c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1906802373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab7820 0x560499ab7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1906802373 >> 172.21.15.86:0/1906802373 conn(0x560499ab3d30 msgr2=0x560499ab4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1906802373 shutdown_connections 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.477+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1906802373 wait complete. 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.478+0000 7fcde5a1c740 1 -- start start 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.478+0000 7fcde5a1c740 1 Processor -- start 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 0x560499abf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab8140 0x560499abfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ac0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560499ac5630 con 0x560499ac2a00 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604999d7310 con 0x560499ab7820 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560499ac57b0 con 0x560499ab8140 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde5a1c740 10 client.0 _ll_register_callbacks cb 0x5604999a5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 0x560499abf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ac0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab8140 0x560499abfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 0x560499abf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:54120/0 (socket says 172.21.15.86:54120) 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 -- 172.21.15.86:0/1489793252 learned_addr learned my addr 172.21.15.86:0/1489793252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 -- 172.21.15.86:0/1489793252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab8140 msgr2=0x560499abfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab8140 0x560499abfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 -- 172.21.15.86:0/1489793252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 msgr2=0x560499ac0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.479+0000 7fcddbe00640 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ac0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.480+0000 7fcddbe00640 1 -- 172.21.15.86:0/1489793252 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604997fc620 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.480+0000 7fcddbe00640 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 0x560499abf8c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc004ba0 tx=0x7fcdcc004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb84c7cf46a41015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.480+0000 7fcdc9600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.480+0000 7fcdc9600640 1 -- 172.21.15.86:0/1489793252 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc029070 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.480+0000 7fcdc9600640 1 -- 172.21.15.86:0/1489793252 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fcdcc0052d0 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.480+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc004c50 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcdc9600640 1 -- 172.21.15.86:0/1489793252 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc035050 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 10 client.25478 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560499ad17c0 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcdc9600640 1 -- 172.21.15.86:0/1489793252 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc004c50 con 0x560499ab7820 2024-04-25T19:04:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcdc9600640 1 -- 172.21.15.86:0/1489793252 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdcc0234e0 con 0x560499ab7820 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 10 client.25478 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 20 client.25478 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499ad2b70 0x560499ad4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 4 client.25478 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.481+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560499a318b0 con 0x560499ad2b70 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.482+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499ad2b70 0x560499ad4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.483+0000 7fcdc9600640 10 client.25478 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.483+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499ad2b70 0x560499ad4f60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13458496a4f1ac6c server_cookie=a5b9364fae5718a9 in_seq=0 out_seq=0 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcdc9600640 1 -- 172.21.15.86:0/1489793252 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560499a318b0 con 0x560499ad2b70 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcdc9600640 10 client.25478 handle_command_reply: tid=0 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 client.25478 shutdown 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 msgr2=0x560499abf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 0x560499abf8c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc004ba0 tx=0x7fcdcc004bd0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 shutdown_connections 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499ad2b70 0x560499ad4f60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ac0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcdc9600640 0 client.25478 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab8140 0x560499abfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/1489793252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab7820 0x560499abf8c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 >> 172.21.15.86:0/1489793252 conn(0x560499ab3d30 msgr2=0x560499ab9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 shutdown_connections 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 1 -- 172.21.15.86:0/1489793252 wait complete. 2024-04-25T19:04:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.484+0000 7fcde5a1c740 20 client.25478 trim_cache size 0 max 16384 2024-04-25T19:04:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.500+0000 7fcde5a1c740 1 Processor -- start 2024-04-25T19:04:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 -- start start 2024-04-25T19:04:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499ace4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ab2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ada740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1afdd0 con 0x560499ac2a00 2024-04-25T19:04:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b1910 con 0x560499ab8140 2024-04-25T19:04:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b04a0 con 0x560499ab7820 2024-04-25T19:04:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ab2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcde0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ada740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ab2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:54134/0 (socket says 172.21.15.86:54134) 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3703529528 learned_addr learned my addr 172.21.15.86:0/3703529528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.501+0000 7fcddbe00640 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499ace4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3703529528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 msgr2=0x560499ace4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499ace4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3703529528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 msgr2=0x560499ada740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ada740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3703529528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560499aaf460 con 0x560499ab8140 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ab2d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005fd0 tx=0x7fcdc4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24fa692ad433cf03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdcbe00640 1 -- 172.21.15.86:0/3703529528 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc402bd50 con 0x560499ab8140 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdcbe00640 1 -- 172.21.15.86:0/3703529528 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc40054d0 con 0x560499ab8140 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.502+0000 7fcdcbe00640 1 -- 172.21.15.86:0/3703529528 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40047a0 con 0x560499ab8140 2024-04-25T19:04:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3703529528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 msgr2=0x560499ab2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ab2d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005fd0 tx=0x7fcdc4004730 comp rx=0 tx=0).stop 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3703529528 shutdown_connections 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ac2a00 0x560499ada740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ab8140 0x560499ab2d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3703529528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499ace4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3703529528 >> 172.21.15.86:0/3703529528 conn(0x560499ab3d30 msgr2=0x560499ab2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3703529528 shutdown_connections 2024-04-25T19:04:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.503+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3703529528 wait complete. 2024-04-25T19:04:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 -- start start 2024-04-25T19:04:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 Processor -- start 2024-04-25T19:04:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499acab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 0x560499ac7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ac2a00 0x560499ac8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b1240 con 0x560499ab8140 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b0b70 con 0x560499ac2a00 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1addb0 con 0x560499ab7820 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.504+0000 7fcde5a1c740 10 client.0 _ll_register_callbacks cb 0x7fcdcc004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 0x560499ac7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 0x560499ac7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52180/0 (socket says 172.21.15.86:52180) 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3666197206 learned_addr learned my addr 172.21.15.86:0/3666197206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcde0c00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ac2a00 0x560499ac8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcddbe00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499acab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3666197206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 msgr2=0x560499acab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499acab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ac2a00 msgr2=0x560499ac8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ac2a00 0x560499ac8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcdd3e00640 1 -- 172.21.15.86:0/3666197206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560499ac8a50 con 0x560499ab8140 2024-04-25T19:04:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcddbe00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499acab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.505+0000 7fcde0c00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ac2a00 0x560499ac8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcdd3e00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 0x560499ac7fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400a010 tx=0x7fcdc4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224ef6edd32f8fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcdc9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcdc9600640 1 -- 172.21.15.86:0/3666197206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4037030 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcdc9600640 1 -- 172.21.15.86:0/3666197206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fcdc402b390 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc40054d0 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcdc9600640 1 -- 172.21.15.86:0/3666197206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdc402b6f0 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcde5a1c740 10 client.15366 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.506+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560499b38d00 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcdc9600640 1 -- 172.21.15.86:0/3666197206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc40054d0 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcdc9600640 1 -- 172.21.15.86:0/3666197206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdc4033910 con 0x560499ab8140 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcde5a1c740 10 client.15366 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcde5a1c740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcde5a1c740 20 client.15366 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499b393c0 0x560499b3b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcde5a1c740 4 client.15366 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560499a2b320 con 0x560499b393c0 2024-04-25T19:04:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.507+0000 7fcddbe00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499b393c0 0x560499b3b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.508+0000 7fcdc9600640 10 client.15366 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.508+0000 7fcddbe00640 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499b393c0 0x560499b3b7e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ec17f35c82eb30 server_cookie=cd685e164853d78b in_seq=0 out_seq=0 2024-04-25T19:04:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.513+0000 7fcdc9600640 1 -- 172.21.15.86:0/3666197206 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560499a2b320 con 0x560499b393c0 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.513+0000 7fcdc9600640 10 client.15366 handle_command_reply: tid=0 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.513+0000 7fcde5a1c740 1 client.15366 shutdown 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 msgr2=0x560499ac7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 0x560499ac7fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400a010 tx=0x7fcdc4009ac0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 shutdown_connections 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560499b393c0 0x560499b3b7e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560499ac2a00 0x560499ac8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560499ab8140 0x560499ac7fd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcdc9600640 0 client.15366 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 --2- 172.21.15.86:0/3666197206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560499ab7820 0x560499acab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 >> 172.21.15.86:0/3666197206 conn(0x560499ab3d30 msgr2=0x560499ac00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 shutdown_connections 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 1 -- 172.21.15.86:0/3666197206 wait complete. 2024-04-25T19:04:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.514+0000 7fcde5a1c740 20 client.15366 trim_cache size 0 max 16384 2024-04-25T19:04:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:04:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:04:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:04:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdc00a3e40 msgr2=0x7fcdc00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdc00a3e40 0x7fcdc00a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x560499ab5150 tx=0x7fcdc4015040 comp rx=0 tx=0).stop 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 msgr2=0x7fcddc19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc19a850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009b30 tx=0x7fcdd4003b80 comp rx=0 tx=0).stop 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 shutdown_connections 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdc00a3e40 0x7fcdc00a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcddc10c520 0x7fcddc19a850 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc103b00 0x7fcddc1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 --2- 172.21.15.86:0/33649033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc1031e0 0x7fcddc1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 >> 172.21.15.86:0/33649033 conn(0x7fcddc077fd0 msgr2=0x7fcddc108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 shutdown_connections 2024-04-25T19:04:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:32.515+0000 7fcde2c00640 1 -- 172.21.15.86:0/33649033 wait complete. 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 -- 172.21.15.86:0/2457362271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8075700 msgr2=0x7f15b8113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 --2- 172.21.15.86:0/2457362271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8075700 0x7f15b8113440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009990 tx=0x7f15a4009ca0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 -- 172.21.15.86:0/2457362271 shutdown_connections 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 --2- 172.21.15.86:0/2457362271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8113980 0x7f15b8115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 --2- 172.21.15.86:0/2457362271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8074de0 0x7f15b80751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 --2- 172.21.15.86:0/2457362271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8075700 0x7f15b8113440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 -- 172.21.15.86:0/2457362271 >> 172.21.15.86:0/2457362271 conn(0x7f15b80fe4c0 msgr2=0x7f15b81008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 -- 172.21.15.86:0/2457362271 shutdown_connections 2024-04-25T19:04:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.583+0000 7f15be000700 1 -- 172.21.15.86:0/2457362271 wait complete. 2024-04-25T19:04:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 Processor -- start 2024-04-25T19:04:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 -- start start 2024-04-25T19:04:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 0x7f15b819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8075700 0x7f15b819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8113980 0x7f15b81a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81191b0 con 0x7f15b8074de0 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8119030 con 0x7f15b8075700 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.585+0000 7f15be000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8119330 con 0x7f15b8113980 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 0x7f15b819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15afe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8075700 0x7f15b819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 0x7f15b819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52204/0 (socket says 172.21.15.86:52204) 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 -- 172.21.15.86:0/466465688 learned_addr learned my addr 172.21.15.86:0/466465688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bd600700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8113980 0x7f15b81a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 -- 172.21.15.86:0/466465688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8113980 msgr2=0x7f15b81a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8113980 0x7f15b81a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 -- 172.21.15.86:0/466465688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8075700 msgr2=0x7f15b819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8075700 0x7f15b819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 -- 172.21.15.86:0/466465688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b81a46b0 con 0x7f15b8074de0 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15afe00700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8075700 0x7f15b819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bd600700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8113980 0x7f15b81a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.586+0000 7f15bcc00700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 0x7f15b819f6a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9a0 tx=0x7f15a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.587+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a800c9c0 con 0x7f15b8074de0 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.587+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b81a49a0 con 0x7f15b8074de0 2024-04-25T19:04:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.587+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a800cb60 con 0x7f15b8074de0 2024-04-25T19:04:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.587+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8018680 con 0x7f15b8074de0 2024-04-25T19:04:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.587+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b81ac220 con 0x7f15b8074de0 2024-04-25T19:04:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.588+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b8069730 con 0x7f15b8074de0 2024-04-25T19:04:32.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.591+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15a8010430 con 0x7f15b8074de0 2024-04-25T19:04:32.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.592+0000 7f15b6000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f159c0a3db0 0x7f159c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.593+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f15a80d0b70 con 0x7f15b8074de0 2024-04-25T19:04:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.593+0000 7f15afe00700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f159c0a3db0 0x7f159c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.594+0000 7f15afe00700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f159c0a3db0 0x7f159c0a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15b8100bc0 tx=0x7f15a400b260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.598+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15a809d160 con 0x7f15b8074de0 2024-04-25T19:04:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.880+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15b802df80 con 0x7f15b8074de0 2024-04-25T19:04:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.882+0000 7f15b6000700 1 -- 172.21.15.86:0/466465688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f15a809ca60 con 0x7f15b8074de0 2024-04-25T19:04:32.884 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:04:32.884 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f159c0a3db0 msgr2=0x7f159c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f159c0a3db0 0x7f159c0a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15b8100bc0 tx=0x7f15a400b260 comp rx=0 tx=0).stop 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 msgr2=0x7f15b819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 0x7f15b819f6a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9a0 tx=0x7f15a800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 shutdown_connections 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f159c0a3db0 0x7f159c0a6270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15b8113980 0x7f15b81a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b8074de0 0x7f15b819f6a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 --2- 172.21.15.86:0/466465688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b8075700 0x7f15b819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.885+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 >> 172.21.15.86:0/466465688 conn(0x7f15b80fe4c0 msgr2=0x7f15b80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.886+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 shutdown_connections 2024-04-25T19:04:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:32.886+0000 7f15be000700 1 -- 172.21.15.86:0/466465688 wait complete. 2024-04-25T19:04:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:04:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:32 smithi194 ceph-mon[32740]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:04:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:32 smithi086 ceph-mon[27783]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:04:33.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:32 smithi183 ceph-mon[32344]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T19:04:33.793 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 526b8fed --release --await' 2024-04-25T19:04:33.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 526b8fed --release --await 2024-04-25T19:04:33.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:33 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/466465688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:04:34.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:33 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/466465688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:04:34.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:33 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/466465688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:04:34.236 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:04:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:34 smithi194 ceph-mon[32740]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-04-25T19:04:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:34 smithi086 ceph-mon[27783]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-04-25T19:04:35.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:34 smithi183 ceph-mon[32344]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-04-25T19:04:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:35 smithi194 ceph-mon[32740]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T19:04:35.994 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:35 smithi086 ceph-mon[27783]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T19:04:36.068 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:35 smithi183 ceph-mon[32344]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T19:04:36.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 -- 172.21.15.86:0/3546461786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 msgr2=0x7f433410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 --2- 172.21.15.86:0/3546461786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433410dcd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43200099c0 tx=0x7f4320009cd0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 -- 172.21.15.86:0/3546461786 shutdown_connections 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 --2- 172.21.15.86:0/3546461786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433410e3b0 0x7f4334115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 --2- 172.21.15.86:0/3546461786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f43341096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 --2- 172.21.15.86:0/3546461786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433410dcd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 -- 172.21.15.86:0/3546461786 >> 172.21.15.86:0/3546461786 conn(0x7f4334076210 msgr2=0x7f4334107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 -- 172.21.15.86:0/3546461786 shutdown_connections 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.469+0000 7f433ae00700 1 -- 172.21.15.86:0/3546461786 wait complete. 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 Processor -- start 2024-04-25T19:04:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 -- start start 2024-04-25T19:04:36.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f433419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433410e3b0 0x7f43341a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4334119220 con 0x7f4334109c80 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43341190a0 con 0x7f43341092d0 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.470+0000 7f433ae00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43341193a0 con 0x7f433410e3b0 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f433419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f433419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:54174/0 (socket says 172.21.15.86:54174) 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52226/0 (socket says 172.21.15.86:52226) 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 -- 172.21.15.86:0/3686683879 learned_addr learned my addr 172.21.15.86:0/3686683879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4338e00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433410e3b0 0x7f43341a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 -- 172.21.15.86:0/3686683879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433410e3b0 msgr2=0x7f43341a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433410e3b0 0x7f43341a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 -- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 msgr2=0x7f433419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333e00700 1 -- 172.21.15.86:0/3686683879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43341a4730 con 0x7f43341092d0 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.471+0000 7f4333400700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.472+0000 7f4333e00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f433419f770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f432400e9a0 tx=0x7f432400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.472+0000 7f4330c00700 1 -- 172.21.15.86:0/3686683879 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432400c9c0 con 0x7f43341092d0 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.472+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43341a4a20 con 0x7f43341092d0 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.472+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341ac2a0 con 0x7f43341092d0 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.472+0000 7f4330c00700 1 -- 172.21.15.86:0/3686683879 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432400cb60 con 0x7f43341092d0 2024-04-25T19:04:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.472+0000 7f4330c00700 1 -- 172.21.15.86:0/3686683879 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324026060 con 0x7f43341092d0 2024-04-25T19:04:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.476+0000 7f4330c00700 1 -- 172.21.15.86:0/3686683879 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4324010430 con 0x7f43341092d0 2024-04-25T19:04:36.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.477+0000 7f43457b8b80 1 Processor -- start 2024-04-25T19:04:36.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.477+0000 7f43457b8b80 1 -- start start 2024-04-25T19:04:36.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b14ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b1560f0 0x56347b1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f43457b8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56347b09df90 con 0x56347b1560f0 2024-04-25T19:04:36.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f43457b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56347b044830 con 0x56347b14a920 2024-04-25T19:04:36.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f43457b8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56347b099f50 con 0x56347b14b240 2024-04-25T19:04:36.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4330c00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43180a3db0 0x7f43180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4330c00700 1 -- 172.21.15.86:0/3686683879 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f43240d09d0 con 0x7f43341092d0 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4333e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4338e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b14ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4333e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41116/0 (socket says 172.21.15.86:41116) 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4333e00700 1 -- 172.21.15.86:0/3037476350 learned_addr learned my addr 172.21.15.86:0/3037476350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.478+0000 7f4333400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b1560f0 0x56347b1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4333e00700 1 -- 172.21.15.86:0/3037476350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 msgr2=0x56347b14ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4333e00700 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b14ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4333e00700 1 -- 172.21.15.86:0/3037476350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b1560f0 msgr2=0x56347b1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4333e00700 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b1560f0 0x56347b1584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4338e00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43180a3db0 0x7f43180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4333e00700 1 -- 172.21.15.86:0/3037476350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347b03a780 con 0x56347b14b240 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.479+0000 7f4333e00700 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b155bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43240ce0d0 tx=0x7f43240073b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c99b5a4cc7a5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.480+0000 7f4329600700 1 -- 172.21.15.86:0/3037476350 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432409c8c0 con 0x56347b14b240 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.480+0000 7f4329600700 1 -- 172.21.15.86:0/3037476350 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43240a0c70 con 0x56347b14b240 2024-04-25T19:04:36.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.480+0000 7f4329600700 1 -- 172.21.15.86:0/3037476350 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432409cb20 con 0x56347b14b240 2024-04-25T19:04:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.480+0000 7f43457b8b80 1 -- 172.21.15.86:0/3037476350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 msgr2=0x56347b155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.480+0000 7f4338e00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43180a3db0 0x7f43180a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f432c009fd0 tx=0x7f432c004b40 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.480+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b155bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43240ce0d0 tx=0x7f43240073b0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 -- 172.21.15.86:0/3037476350 shutdown_connections 2024-04-25T19:04:36.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b1560f0 0x56347b1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b14ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3037476350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b155bb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 -- 172.21.15.86:0/3037476350 >> 172.21.15.86:0/3037476350 conn(0x56347b0796a0 msgr2=0x56347b146dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 -- 172.21.15.86:0/3037476350 shutdown_connections 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 -- 172.21.15.86:0/3037476350 wait complete. 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.481+0000 7f43457b8b80 1 -- start start 2024-04-25T19:04:36.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 Processor -- start 2024-04-25T19:04:36.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b155800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 0x56347b152b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56347b158c10 con 0x56347b14b240 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56347b057960 con 0x56347b14a920 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56347b158d90 con 0x56347b1560f0 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.482+0000 7f43457b8b80 10 client.0 _ll_register_callbacks cb 0x56347b036820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 0x56347b152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 0x56347b152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52254/0 (socket says 172.21.15.86:52254) 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 -- 172.21.15.86:0/2221098646 learned_addr learned my addr 172.21.15.86:0/2221098646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:54190/0 (socket says 172.21.15.86:54190) 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4338e00700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 -- 172.21.15.86:0/2221098646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 msgr2=0x56347b153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 -- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 msgr2=0x56347b155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 -- 172.21.15.86:0/2221098646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347afeb9d0 con 0x56347b14b240 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333e00700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4338e00700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f4333400700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 0x56347b152b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f433406c260 tx=0x7f432000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e1cc395cc828f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.483+0000 7f431e000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f431e000700 1 -- 172.21.15.86:0/2221098646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320029060 con 0x56347b14b240 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f431e000700 1 -- 172.21.15.86:0/2221098646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f43200054d0 con 0x56347b14b240 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43240a0c70 con 0x56347b14b240 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f431e000700 1 -- 172.21.15.86:0/2221098646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43200056d0 con 0x56347b14b240 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f43457b8b80 10 client.15384 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:36.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56347b034bf0 con 0x56347b14b240 2024-04-25T19:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.484+0000 7f431e000700 1 -- 172.21.15.86:0/2221098646 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43240a0c70 con 0x56347b14b240 2024-04-25T19:04:36.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f431e000700 1 -- 172.21.15.86:0/2221098646 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f432000f950 con 0x56347b14b240 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f43457b8b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f43457b8b80 10 client.15384 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f43457b8b80 20 client.15384 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f43457b8b80 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1658a0 0x56347b167c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f43457b8b80 4 client.15384 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56347b0ab780 con 0x56347b1658a0 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.485+0000 7f4333e00700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1658a0 0x56347b167c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.486+0000 7f4333e00700 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1658a0 0x56347b167c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440aa4f61d7edd80 server_cookie=86df5120c3136ac8 in_seq=0 out_seq=0 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.486+0000 7f431e000700 10 client.15384 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f431e000700 1 -- 172.21.15.86:0/2221098646 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56347b0ab780 con 0x56347b1658a0 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f431e000700 10 client.15384 handle_command_reply: tid=0 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 client.15384 shutdown 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 msgr2=0x56347b152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 0x56347b152b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f433406c260 tx=0x7f432000b8d0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 shutdown_connections 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1658a0 0x56347b167c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14a920 0x56347b155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 --2- 172.21.15.86:0/2221098646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14b240 0x56347b152b00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 >> 172.21.15.86:0/2221098646 conn(0x56347b0796a0 msgr2=0x56347b1469c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f431e000700 0 client.15384 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 shutdown_connections 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.487+0000 7f43457b8b80 1 -- 172.21.15.86:0/2221098646 wait complete. 2024-04-25T19:04:36.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.488+0000 7f43457b8b80 20 client.15384 trim_cache size 0 max 16384 2024-04-25T19:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 Processor -- start 2024-04-25T19:04:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 -- start start 2024-04-25T19:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b146f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14b240 0x56347b161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b14e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431400d700 con 0x56347b14a920 2024-04-25T19:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f431400f240 con 0x56347b14b240 2024-04-25T19:04:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f43457b8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f431400ddd0 con 0x56347b1560f0 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f4333e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f4333e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b146f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52256/0 (socket says 172.21.15.86:52256) 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f4333e00700 1 -- 172.21.15.86:0/986355441 learned_addr learned my addr 172.21.15.86:0/986355441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.509+0000 7f4338e00700 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4333400700 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14b240 0x56347b161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4338e00700 1 -- 172.21.15.86:0/986355441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14b240 msgr2=0x56347b161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4338e00700 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14b240 0x56347b161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4338e00700 1 -- 172.21.15.86:0/986355441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 msgr2=0x56347b146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4338e00700 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4338e00700 1 -- 172.21.15.86:0/986355441 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347b1620b0 con 0x56347b1560f0 2024-04-25T19:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.510+0000 7f4338e00700 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b14e060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f432c009400 tx=0x7f432c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396ff95ce6fa70c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f4328c00700 1 -- 172.21.15.86:0/986355441 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c029070 con 0x56347b1560f0 2024-04-25T19:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f4328c00700 1 -- 172.21.15.86:0/986355441 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432c013e40 con 0x56347b1560f0 2024-04-25T19:04:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f4328c00700 1 -- 172.21.15.86:0/986355441 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c007d70 con 0x56347b1560f0 2024-04-25T19:04:36.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 -- 172.21.15.86:0/986355441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 msgr2=0x56347b14e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b14e060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f432c009400 tx=0x7f432c009920 comp rx=0 tx=0).stop 2024-04-25T19:04:36.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 -- 172.21.15.86:0/986355441 shutdown_connections 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b1560f0 0x56347b14e060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 --2- 172.21.15.86:0/986355441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b14b240 0x56347b161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 -- 172.21.15.86:0/986355441 >> 172.21.15.86:0/986355441 conn(0x56347b0796a0 msgr2=0x56347b14d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 -- 172.21.15.86:0/986355441 shutdown_connections 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.511+0000 7f43457b8b80 1 -- 172.21.15.86:0/986355441 wait complete. 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.512+0000 7f43457b8b80 1 -- start start 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.512+0000 7f43457b8b80 1 Processor -- start 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.512+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b15e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.512+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.512+0000 7f43457b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b1560f0 0x56347b15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f43457b8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431400eb70 con 0x56347b14a920 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f43457b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f431400e4a0 con 0x56347b1560f0 2024-04-25T19:04:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f43457b8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f431400b6a0 con 0x56347b14b240 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f43457b8b80 10 client.0 _ll_register_callbacks cb 0x56347b036820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b15e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52270/0 (socket says 172.21.15.86:52270) 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333e00700 1 -- 172.21.15.86:0/3251914286 learned_addr learned my addr 172.21.15.86:0/3251914286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4338e00700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b1560f0 0x56347b15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333400700 1 -- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b1560f0 msgr2=0x56347b15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333400700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b1560f0 0x56347b15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333400700 1 -- 172.21.15.86:0/3251914286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 msgr2=0x56347b15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333400700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b15e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333400700 1 -- 172.21.15.86:0/3251914286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347b15b2e0 con 0x56347b14b240 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.513+0000 7f4333e00700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f4333400700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b15a6a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f432000ff10 tx=0x7f4320005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ca704b0ffd9c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f431e000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:04:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f431e000700 1 -- 172.21.15.86:0/3251914286 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43200056e0 con 0x56347b14b240 2024-04-25T19:04:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432c013e40 con 0x56347b14b240 2024-04-25T19:04:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f431e000700 1 -- 172.21.15.86:0/3251914286 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f43200177c0 con 0x56347b14b240 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f431e000700 1 -- 172.21.15.86:0/3251914286 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4320044050 con 0x56347b14b240 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f43457b8b80 10 client.25498 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.514+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56347b1ccdc0 con 0x56347b14b240 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f431e000700 1 -- 172.21.15.86:0/3251914286 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432c013e40 con 0x56347b14b240 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f431e000700 1 -- 172.21.15.86:0/3251914286 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4320003c10 con 0x56347b14b240 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f43457b8b80 10 client.25498 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f43457b8b80 10 client.25498 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f43457b8b80 20 client.25498 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1cd520 0x56347b1cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f43457b8b80 4 client.25498 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "526b8fed", "release": true, "await": true}] 2024-04-25T19:04:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "526b8fed", "release": true, "await": true}) -- 0x56347b0a5820 con 0x56347b1cd520 2024-04-25T19:04:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.515+0000 7f4333e00700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1cd520 0x56347b1cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.516+0000 7f4333e00700 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1cd520 0x56347b1cf940 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b3210ccdb4a387 server_cookie=8026f638fb4464d3 in_seq=0 out_seq=0 2024-04-25T19:04:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.516+0000 7f431e000700 10 client.25498 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:36.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f431e000700 1 -- 172.21.15.86:0/3251914286 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x56347b0a5820 con 0x56347b1cd520 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f431e000700 10 client.25498 handle_command_reply: tid=0 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 client.25498 shutdown 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 msgr2=0x56347b15a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b15a6a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f432000ff10 tx=0x7f4320005e80 comp rx=0 tx=0).stop 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 shutdown_connections 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56347b1cd520 0x56347b1cf940 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56347b1560f0 0x56347b15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56347b14a920 0x56347b15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f431e000700 0 client.25498 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 --2- 172.21.15.86:0/3251914286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56347b14b240 0x56347b15a6a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 >> 172.21.15.86:0/3251914286 conn(0x56347b0796a0 msgr2=0x56347b154bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:36.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.520+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 shutdown_connections 2024-04-25T19:04:36.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.521+0000 7f43457b8b80 1 -- 172.21.15.86:0/3251914286 wait complete. 2024-04-25T19:04:36.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.521+0000 7f43457b8b80 20 client.25498 trim_cache size 0 max 16384 2024-04-25T19:04:36.523 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 4, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "526b8fed": { 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 73.8, 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:04:36.524 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43180a3db0 msgr2=0x7f43180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43180a3db0 0x7f43180a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f432c009fd0 tx=0x7f432c004b40 comp rx=0 tx=0).stop 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 msgr2=0x7f433419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f433419f770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f432400e9a0 tx=0x7f432400ecb0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 shutdown_connections 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43180a3db0 0x7f43180a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433410e3b0 0x7f43341a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43341092d0 0x7f433419f770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 --2- 172.21.15.86:0/3686683879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4334109c80 0x7f433419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 >> 172.21.15.86:0/3686683879 conn(0x7f4334076210 msgr2=0x7f4334077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 shutdown_connections 2024-04-25T19:04:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:04:36.522+0000 7f433ae00700 1 -- 172.21.15.86:0/3686683879 wait complete. 2024-04-25T19:04:36.534 INFO:tasks.workunit.client.0.smithi086.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T19:04:36.536 INFO:tasks.workunit.client.0.smithi086.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-25T19:04:36.536 INFO:tasks.workunit.client.0.smithi086.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-25T19:04:37.480 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '526b8fed', seconds elapsed: 29.4 2024-04-25T19:04:38.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:37 smithi194 ceph-mon[32740]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T19:04:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:37 smithi086 ceph-mon[27783]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T19:04:38.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:37 smithi183 ceph-mon[32344]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T19:04:40.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:39 smithi194 ceph-mon[32740]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-04-25T19:04:40.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:39 smithi086 ceph-mon[27783]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-04-25T19:04:40.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:39 smithi183 ceph-mon[32344]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-04-25T19:04:42.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:41 smithi194 ceph-mon[32740]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T19:04:42.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:41 smithi086 ceph-mon[27783]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T19:04:42.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:41 smithi183 ceph-mon[32344]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T19:04:44.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:43 smithi194 ceph-mon[32740]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T19:04:44.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:43 smithi086 ceph-mon[27783]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T19:04:44.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:43 smithi183 ceph-mon[32344]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-25T19:04:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 -- 172.21.15.86:0/2260512383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104104d70 msgr2=0x7fd104108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 --2- 172.21.15.86:0/2260512383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104104d70 0x7fd104108db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009860 tx=0x7fd0f002a340 comp rx=0 tx=0).stop 2024-04-25T19:04:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 -- 172.21.15.86:0/2260512383 shutdown_connections 2024-04-25T19:04:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 --2- 172.21.15.86:0/2260512383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104109490 0x7fd104111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 --2- 172.21.15.86:0/2260512383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104104d70 0x7fd104108db0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 --2- 172.21.15.86:0/2260512383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1041043c0 0x7fd1041047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 -- 172.21.15.86:0/2260512383 >> 172.21.15.86:0/2260512383 conn(0x7fd1040ffc20 msgr2=0x7fd104102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 -- 172.21.15.86:0/2260512383 shutdown_connections 2024-04-25T19:04:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.656+0000 7fd109c00640 1 -- 172.21.15.86:0/2260512383 wait complete. 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 Processor -- start 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 -- start start 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 0x7fd1041a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 0x7fd1041a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd10419a710 0x7fd10419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041143b0 con 0x7fd10419a710 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114230 con 0x7fd1041043c0 2024-04-25T19:04:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.658+0000 7fd109c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114530 con 0x7fd104109490 2024-04-25T19:04:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 0x7fd1041a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 0x7fd1041a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 0x7fd1041a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:35648/0 (socket says 172.21.15.86:35648) 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102a00640 1 -- 172.21.15.86:0/2123512782 learned_addr learned my addr 172.21.15.86:0/2123512782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 0x7fd1041a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53564/0 (socket says 172.21.15.86:53564) 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd103400640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd10419a710 0x7fd10419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 -- 172.21.15.86:0/2123512782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 msgr2=0x7fd1041a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 0x7fd1041a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 -- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd10419a710 msgr2=0x7fd10419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd10419a710 0x7fd10419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102000640 1 -- 172.21.15.86:0/2123512782 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10419b310 con 0x7fd104109490 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.659+0000 7fd102a00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 0x7fd1041a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.660+0000 7fd102000640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 0x7fd1041a0b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0035040 tx=0x7fd0f0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.660+0000 7fd0fb400640 1 -- 172.21.15.86:0/2123512782 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0040020 con 0x7fd104109490 2024-04-25T19:04:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.660+0000 7fd0fb400640 1 -- 172.21.15.86:0/2123512782 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0030d50 con 0x7fd104109490 2024-04-25T19:04:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.660+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10419b5a0 con 0x7fd104109490 2024-04-25T19:04:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.660+0000 7fd0fb400640 1 -- 172.21.15.86:0/2123512782 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037040 con 0x7fd104109490 2024-04-25T19:04:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.660+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a76e0 con 0x7fd104109490 2024-04-25T19:04:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.663+0000 7fd10cb66740 1 Processor -- start 2024-04-25T19:04:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 -- start start 2024-04-25T19:04:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794030 0x561230794410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794950 0x56123079ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612306f1900 con 0x561230794950 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561230697ef0 con 0x561230794030 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd10cb66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612306ed8c0 con 0x56123079f210 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd0fb400640 1 -- 172.21.15.86:0/2123512782 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0f0043020 con 0x7fd104109490 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd102a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794950 0x56123079ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd102a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794950 0x56123079ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57356/0 (socket says 172.21.15.86:57356) 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd103400640 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794030 0x561230794410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.664+0000 7fd102a00640 1 -- 172.21.15.86:0/2996017013 learned_addr learned my addr 172.21.15.86:0/2996017013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd102000640 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd103400640 1 -- 172.21.15.86:0/2996017013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 msgr2=0x5612307a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd103400640 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd103400640 1 -- 172.21.15.86:0/2996017013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794950 msgr2=0x56123079ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd103400640 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794950 0x56123079ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd103400640 1 -- 172.21.15.86:0/2996017013 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561230404210 con 0x561230794030 2024-04-25T19:04:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd102000640 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd103400640 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794030 0x561230794410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009f10 tx=0x7fd0fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81eb2414e19836d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd0efe00640 1 -- 172.21.15.86:0/2996017013 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc004d20 con 0x561230794030 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd0efe00640 1 -- 172.21.15.86:0/2996017013 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc004050 con 0x561230794030 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.665+0000 7fd0efe00640 1 -- 172.21.15.86:0/2996017013 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc005220 con 0x561230794030 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd0fb400640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0e80a3dc0 0x7fd0e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd0fb400640 1 -- 172.21.15.86:0/2123512782 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd0f00f02b0 con 0x7fd104109490 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 -- 172.21.15.86:0/2996017013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794030 msgr2=0x561230794410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794030 0x561230794410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009f10 tx=0x7fd0fc011810 comp rx=0 tx=0).stop 2024-04-25T19:04:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd102a00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0e80a3dc0 0x7fd0e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 -- 172.21.15.86:0/2996017013 shutdown_connections 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794950 0x56123079ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2996017013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794030 0x561230794410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 -- 172.21.15.86:0/2996017013 >> 172.21.15.86:0/2996017013 conn(0x561230790540 msgr2=0x561230790920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.666+0000 7fd10cb66740 1 -- 172.21.15.86:0/2996017013 shutdown_connections 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.667+0000 7fd10cb66740 1 -- 172.21.15.86:0/2996017013 wait complete. 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.667+0000 7fd102a00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0e80a3dc0 0x7fd0e80a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5612307955d0 tx=0x7fd0f4009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.667+0000 7fd10cb66740 1 -- start start 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.667+0000 7fd10cb66740 1 Processor -- start 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794030 0x56123079c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x56123079c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 0x56123079cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612307a1e40 con 0x56123079f210 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612306ab310 con 0x561230794950 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612307a1fc0 con 0x561230794030 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd10cb66740 10 client.0 _ll_register_callbacks cb 0x561230679cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 0x56123079cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 0x56123079cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57364/0 (socket says 172.21.15.86:57364) 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 -- 172.21.15.86:0/1784970259 learned_addr learned my addr 172.21.15.86:0/1784970259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd102000640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x56123079c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd102a00640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794030 0x56123079c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 -- 172.21.15.86:0/1784970259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794030 msgr2=0x56123079c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794030 0x56123079c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 -- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 msgr2=0x56123079c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.668+0000 7fd103400640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x56123079c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd103400640 1 -- 172.21.15.86:0/1784970259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612304d0620 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd102000640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x56123079c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd103400640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 0x56123079cc00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc005f40 tx=0x7fd0fc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a37bfad9b35dc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd0ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd0ecc00640 1 -- 172.21.15.86:0/1784970259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc027020 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd0ecc00640 1 -- 172.21.15.86:0/1784970259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd0fc004c40 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0fc004050 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.669+0000 7fd0ecc00640 1 -- 172.21.15.86:0/1784970259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc032050 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 10 client.15402 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612307adff0 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd0ecc00640 1 -- 172.21.15.86:0/1784970259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc004050 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd0ecc00640 1 -- 172.21.15.86:0/1784970259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0fc018070 con 0x56123079f210 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 10 client.15402 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 10 client.15402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 20 client.15402 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5612307af3f0 0x5612307b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 4 client.15402 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.670+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612307058b0 con 0x5612307af3f0 2024-04-25T19:04:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.671+0000 7fd102a00640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5612307af3f0 0x5612307b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.671+0000 7fd102a00640 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5612307af3f0 0x5612307b17e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3186c46e47ab87c server_cookie=b4302645b983a841 in_seq=0 out_seq=0 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.671+0000 7fd0ecc00640 10 client.15402 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd0ecc00640 1 -- 172.21.15.86:0/1784970259 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612307058b0 con 0x5612307af3f0 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd0ecc00640 10 client.15402 handle_command_reply: tid=0 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 client.15402 shutdown 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 msgr2=0x56123079cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 0x56123079cc00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc005f40 tx=0x7fd0fc004410 comp rx=0 tx=0).stop 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 shutdown_connections 2024-04-25T19:04:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5612307af3f0 0x5612307b17e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123079f210 0x56123079cc00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x56123079c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1784970259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794030 0x56123079c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd0ecc00640 0 client.15402 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.672+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 >> 172.21.15.86:0/1784970259 conn(0x561230790540 msgr2=0x5612307962c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.673+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 shutdown_connections 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.673+0000 7fd10cb66740 1 -- 172.21.15.86:0/1784970259 wait complete. 2024-04-25T19:04:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.673+0000 7fd10cb66740 20 client.15402 trim_cache size 0 max 16384 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 Processor -- start 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 -- start start 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x5612307969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041aba50 con 0x561230794030 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ad590 con 0x561230794950 2024-04-25T19:04:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.689+0000 7fd10cb66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ac120 con 0x56123079f210 2024-04-25T19:04:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307b6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53606/0 (socket says 172.21.15.86:53606) 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 -- 172.21.15.86:0/1629522671 learned_addr learned my addr 172.21.15.86:0/1629522671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd102000640 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x5612307969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd102a00640 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 -- 172.21.15.86:0/1629522671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 msgr2=0x5612307969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x5612307969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 -- 172.21.15.86:0/1629522671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 msgr2=0x5612307aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.690+0000 7fd103400640 1 -- 172.21.15.86:0/1629522671 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561230797d80 con 0x56123079f210 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.691+0000 7fd103400640 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307b6f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc005a80 tx=0x7fd0fc005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621edf34a9260f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.691+0000 7fd0ef400640 1 -- 172.21.15.86:0/1629522671 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc027040 con 0x56123079f210 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.691+0000 7fd0ef400640 1 -- 172.21.15.86:0/1629522671 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc009820 con 0x56123079f210 2024-04-25T19:04:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.691+0000 7fd0ef400640 1 -- 172.21.15.86:0/1629522671 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc01e040 con 0x56123079f210 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 -- 172.21.15.86:0/1629522671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 msgr2=0x5612307b6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307b6f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc005a80 tx=0x7fd0fc005d30 comp rx=0 tx=0).stop 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 -- 172.21.15.86:0/1629522671 shutdown_connections 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56123079f210 0x5612307b6f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561230794950 0x5612307969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 --2- 172.21.15.86:0/1629522671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 -- 172.21.15.86:0/1629522671 >> 172.21.15.86:0/1629522671 conn(0x561230790540 msgr2=0x561230786e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 -- 172.21.15.86:0/1629522671 shutdown_connections 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 -- 172.21.15.86:0/1629522671 wait complete. 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 -- start start 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.692+0000 7fd10cb66740 1 Processor -- start 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794950 0x5612307a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123079f210 0x5612307a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041acec0 con 0x561230794030 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ac7f0 con 0x56123079f210 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041a9a30 con 0x561230794950 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd10cb66740 10 client.0 _ll_register_callbacks cb 0x7fd0fc0047f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd102a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd102a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57386/0 (socket says 172.21.15.86:57386) 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd102a00640 1 -- 172.21.15.86:0/2816283488 learned_addr learned my addr 172.21.15.86:0/2816283488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd103400640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123079f210 0x5612307a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd102000640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794950 0x5612307a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd102a00640 1 -- 172.21.15.86:0/2816283488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794950 msgr2=0x5612307a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.693+0000 7fd102a00640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794950 0x5612307a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd102a00640 1 -- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123079f210 msgr2=0x5612307a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd102a00640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123079f210 0x5612307a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd102a00640 1 -- 172.21.15.86:0/2816283488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612307a5270 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd102a00640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307a7430 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5612307b1ef0 tx=0x7fd0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65841df9f38948a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd0ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd0ecc00640 1 -- 172.21.15.86:0/2816283488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f402f030 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd0ecc00640 1 -- 172.21.15.86:0/2816283488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd0f4018030 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0fc009820 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd0ecc00640 1 -- 172.21.15.86:0/2816283488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f403a050 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd10cb66740 10 client.15408 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.694+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561230815600 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd0ecc00640 1 -- 172.21.15.86:0/2816283488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc009820 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd0ecc00640 1 -- 172.21.15.86:0/2816283488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0f4003730 con 0x561230794030 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd10cb66740 10 client.15408 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd10cb66740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd10cb66740 20 client.15408 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561230815d80 0x5612308181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd10cb66740 4 client.15408 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612306ff320 con 0x561230815d80 2024-04-25T19:04:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.695+0000 7fd102000640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561230815d80 0x5612308181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.696+0000 7fd102000640 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561230815d80 0x5612308181a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a703b50aacb96d0a server_cookie=3d96d999fef576fd in_seq=0 out_seq=0 2024-04-25T19:04:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.696+0000 7fd0ecc00640 10 client.15408 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd0ecc00640 1 -- 172.21.15.86:0/2816283488 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612306ff320 con 0x561230815d80 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd0ecc00640 10 client.15408 handle_command_reply: tid=0 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 client.15408 shutdown 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 msgr2=0x5612307a7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307a7430 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5612307b1ef0 tx=0x7fd0f40046c0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 shutdown_connections 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561230815d80 0x5612308181a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123079f210 0x5612307a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561230794950 0x5612307a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 --2- 172.21.15.86:0/2816283488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561230794030 0x5612307a7430 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd0ecc00640 0 client.15408 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 >> 172.21.15.86:0/2816283488 conn(0x561230790540 msgr2=0x56123079be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 shutdown_connections 2024-04-25T19:04:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 1 -- 172.21.15.86:0/2816283488 wait complete. 2024-04-25T19:04:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.705+0000 7fd10cb66740 20 client.15408 trim_cache size 0 max 16384 2024-04-25T19:04:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:04:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:04:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:04:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0e80a3dc0 msgr2=0x7fd0e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0e80a3dc0 0x7fd0e80a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5612307955d0 tx=0x7fd0f4009260 comp rx=0 tx=0).stop 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 msgr2=0x7fd1041a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 0x7fd1041a0b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0035040 tx=0x7fd0f0032d50 comp rx=0 tx=0).stop 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 shutdown_connections 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0e80a3dc0 0x7fd0e80a6280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd10419a710 0x7fd10419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104109490 0x7fd1041a0b90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 --2- 172.21.15.86:0/2123512782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041043c0 0x7fd1041a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 >> 172.21.15.86:0/2123512782 conn(0x7fd1040ffc20 msgr2=0x7fd104100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 shutdown_connections 2024-04-25T19:04:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:45.706+0000 7fd109c00640 1 -- 172.21.15.86:0/2123512782 wait complete. 2024-04-25T19:04:46.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:45 smithi194 ceph-mon[32740]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:04:46.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:45 smithi086 ceph-mon[27783]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:04:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:45 smithi183 ceph-mon[32344]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:04:48.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:47 smithi194 ceph-mon[32740]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:04:48.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:47 smithi086 ceph-mon[27783]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:04:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:47 smithi183 ceph-mon[32344]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:04:50.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:49 smithi194 ceph-mon[32740]: pgmap v59: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-25T19:04:50.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:49 smithi086 ceph-mon[27783]: pgmap v59: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-25T19:04:50.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:49 smithi183 ceph-mon[32344]: pgmap v59: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-25T19:04:52.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:51 smithi194 ceph-mon[32740]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:52.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:51 smithi086 ceph-mon[27783]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:52.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:51 smithi183 ceph-mon[32344]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:54.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:53 smithi194 ceph-mon[32740]: pgmap v61: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-25T19:04:54.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:53 smithi086 ceph-mon[27783]: pgmap v61: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-25T19:04:54.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:53 smithi183 ceph-mon[32344]: pgmap v61: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-25T19:04:56.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:55 smithi194 ceph-mon[32740]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:56.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:55 smithi086 ceph-mon[27783]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:56.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:55 smithi183 ceph-mon[32344]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:58.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:57 smithi086 ceph-mon[27783]: pgmap v63: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:58.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:57 smithi183 ceph-mon[32344]: pgmap v63: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:58.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:57 smithi194 ceph-mon[32740]: pgmap v63: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 -- 172.21.15.86:0/1086653455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 msgr2=0x7f6050102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 --2- 172.21.15.86:0/1086653455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f6050102580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6044009900 tx=0x7f604402a410 comp rx=0 tx=0).stop 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 -- 172.21.15.86:0/1086653455 shutdown_connections 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 --2- 172.21.15.86:0/1086653455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6050107270 0x7f605010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 --2- 172.21.15.86:0/1086653455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6050102b50 0x7f6050106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 --2- 172.21.15.86:0/1086653455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f6050102580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 -- 172.21.15.86:0/1086653455 >> 172.21.15.86:0/1086653455 conn(0x7f60500fda00 msgr2=0x7f60500ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 -- 172.21.15.86:0/1086653455 shutdown_connections 2024-04-25T19:04:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.852+0000 7f6058800640 1 -- 172.21.15.86:0/1086653455 wait complete. 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.853+0000 7f6058800640 1 Processor -- start 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.853+0000 7f6058800640 1 -- start start 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6058800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f60501a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6058800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6050107270 0x7f60501a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6058800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605019a710 0x7f605019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6058800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112200 con 0x7f60501021a0 2024-04-25T19:04:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6058800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112080 con 0x7f6050107270 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6058800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112380 con 0x7f605019a710 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6055e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f60501a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6055e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f60501a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36400/0 (socket says 172.21.15.86:36400) 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.854+0000 7f6055e00640 1 -- 172.21.15.86:0/2532322314 learned_addr learned my addr 172.21.15.86:0/2532322314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f6055e00640 1 -- 172.21.15.86:0/2532322314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605019a710 msgr2=0x7f605019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f6055e00640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605019a710 0x7f605019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f6055e00640 1 -- 172.21.15.86:0/2532322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6050107270 msgr2=0x7f60501a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f6055e00640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6050107270 0x7f60501a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f6055e00640 1 -- 172.21.15.86:0/2532322314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605019b310 con 0x7f60501021a0 2024-04-25T19:04:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f6055e00640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f60501a0650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6044005ab0 tx=0x7f6044003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.855+0000 7f604aa00640 1 -- 172.21.15.86:0/2532322314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604403f070 con 0x7f60501021a0 2024-04-25T19:04:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.856+0000 7f604aa00640 1 -- 172.21.15.86:0/2532322314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6044031b60 con 0x7f60501021a0 2024-04-25T19:04:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.856+0000 7f604aa00640 1 -- 172.21.15.86:0/2532322314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044035030 con 0x7f60501021a0 2024-04-25T19:04:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.856+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605019b5a0 con 0x7f60501021a0 2024-04-25T19:04:58.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.856+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a7690 con 0x7f60501021a0 2024-04-25T19:04:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f605b797740 1 Processor -- start 2024-04-25T19:04:58.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f605b797740 1 -- start start 2024-04-25T19:04:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f605b797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fa6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f604aa00640 1 -- 172.21.15.86:0/2532322314 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6044042070 con 0x7f60501021a0 2024-04-25T19:04:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f605b797740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f605b797740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a8fb3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.859+0000 7f605b797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5607a8f0c900 con 0x5607a8fa6820 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f605b797740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607a8eb2ef0 con 0x5607a8fb1a00 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f605b797740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5607a8f088c0 con 0x5607a8fa7140 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fa6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a8fb3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6056800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fa6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36408/0 (socket says 172.21.15.86:36408) 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a8fb3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:41470/0 (socket says 172.21.15.86:41470) 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6056800640 1 -- 172.21.15.86:0/4137188947 learned_addr learned my addr 172.21.15.86:0/4137188947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055e00640 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 -- 172.21.15.86:0/4137188947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 msgr2=0x5607a8fb14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 -- 172.21.15.86:0/4137188947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 msgr2=0x5607a8fa6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fa6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.860+0000 7f6055400640 1 -- 172.21.15.86:0/4137188947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a8c1f210 con 0x5607a8fb1a00 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.861+0000 7f6055400640 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a8fb3df0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6050069280 tx=0x7f6040009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1399237f876ed71c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.861+0000 7f603f000640 1 -- 172.21.15.86:0/4137188947 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000ecb0 con 0x5607a8fb1a00 2024-04-25T19:04:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.861+0000 7f603f000640 1 -- 172.21.15.86:0/4137188947 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604000ee50 con 0x5607a8fb1a00 2024-04-25T19:04:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.861+0000 7f603f000640 1 -- 172.21.15.86:0/4137188947 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040018840 con 0x5607a8fb1a00 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.861+0000 7f604aa00640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f60380a3e40 0x7f60380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f604aa00640 1 -- 172.21.15.86:0/2532322314 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f60440f0c50 con 0x7f60501021a0 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f6056800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f60380a3e40 0x7f60380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 -- 172.21.15.86:0/4137188947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 msgr2=0x5607a8fb3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a8fb3df0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6050069280 tx=0x7f6040009ef0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 -- 172.21.15.86:0/4137188947 shutdown_connections 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a8fb3df0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 --2- 172.21.15.86:0/4137188947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fa6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 -- 172.21.15.86:0/4137188947 >> 172.21.15.86:0/4137188947 conn(0x5607a8fa2d30 msgr2=0x5607a8fa3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 -- 172.21.15.86:0/4137188947 shutdown_connections 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f605b797740 1 -- 172.21.15.86:0/4137188947 wait complete. 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.862+0000 7f6056800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f60380a3e40 0x7f60380a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f604c0049d0 tx=0x7f604c009990 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 -- start start 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 Processor -- start 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa7140 0x5607a8faee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fb1a00 0x5607a8faf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5607a8fb4630 con 0x5607a8fa6820 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607a8ec6310 con 0x5607a8fa7140 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.863+0000 7f605b797740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5607a8fb47b0 con 0x5607a8fb1a00 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f605b797740 10 client.0 _ll_register_callbacks cb 0x5607a8e94cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36412/0 (socket says 172.21.15.86:36412) 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 -- 172.21.15.86:0/407979144 learned_addr learned my addr 172.21.15.86:0/407979144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055400640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa7140 0x5607a8faee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6056800640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fb1a00 0x5607a8faf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 -- 172.21.15.86:0/407979144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fb1a00 msgr2=0x5607a8faf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fb1a00 0x5607a8faf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 -- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa7140 msgr2=0x5607a8faee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa7140 0x5607a8faee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 -- 172.21.15.86:0/407979144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a8ceb620 con 0x5607a8fa6820 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6055e00640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fae920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6044005ec0 tx=0x7f60440bc4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7d9968d5d2ea88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.864+0000 7f6033400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f6033400640 1 -- 172.21.15.86:0/407979144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044049070 con 0x5607a8fa6820 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f6033400640 1 -- 172.21.15.86:0/407979144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f60440b91b0 con 0x5607a8fa6820 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604000ee50 con 0x5607a8fa6820 2024-04-25T19:04:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f6033400640 1 -- 172.21.15.86:0/407979144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60440c0ef0 con 0x5607a8fa6820 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f605b797740 10 client.15420 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607a8fc0870 con 0x5607a8fa6820 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f6033400640 1 -- 172.21.15.86:0/407979144 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604000ee50 con 0x5607a8fa6820 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.865+0000 7f6033400640 1 -- 172.21.15.86:0/407979144 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f604404c060 con 0x5607a8fa6820 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f605b797740 10 client.15420 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f605b797740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f605b797740 20 client.15420 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f605b797740 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a8fc1d20 0x5607a8fc41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f605b797740 4 client.15420 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607a8f208b0 con 0x5607a8fc1d20 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.866+0000 7f6055400640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a8fc1d20 0x5607a8fc41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.867+0000 7f6055400640 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a8fc1d20 0x5607a8fc41b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112948709600d546 server_cookie=2b691c6c55ee6cb1 in_seq=0 out_seq=0 2024-04-25T19:04:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.867+0000 7f6033400640 10 client.15420 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.867+0000 7f6033400640 1 -- 172.21.15.86:0/407979144 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607a8f208b0 con 0x5607a8fc1d20 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.867+0000 7f6033400640 10 client.15420 handle_command_reply: tid=0 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 client.15420 shutdown 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 msgr2=0x5607a8fae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fae920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6044005ec0 tx=0x7f60440bc4a0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 shutdown_connections 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a8fc1d20 0x5607a8fc41b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fb1a00 0x5607a8faf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa7140 0x5607a8faee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f6033400640 0 client.15420 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 --2- 172.21.15.86:0/407979144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fae920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 >> 172.21.15.86:0/407979144 conn(0x5607a8fa2d30 msgr2=0x5607a8fa8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 shutdown_connections 2024-04-25T19:04:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 1 -- 172.21.15.86:0/407979144 wait complete. 2024-04-25T19:04:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.868+0000 7f605b797740 20 client.15420 trim_cache size 0 max 16384 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.884+0000 7f605b797740 1 Processor -- start 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 -- start start 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa6820 0x5607a8fbd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fa1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fb1a00 0x5607a8fc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60501aff20 con 0x5607a8fb1a00 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b1a60 con 0x5607a8fa6820 2024-04-25T19:04:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f605b797740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b05f0 con 0x5607a8fa7140 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f6055400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fa1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f6055400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fa1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34836/0 (socket says 172.21.15.86:34836) 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f6055400640 1 -- 172.21.15.86:0/2184725148 learned_addr learned my addr 172.21.15.86:0/2184725148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f6056800640 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fb1a00 0x5607a8fc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.885+0000 7f6055e00640 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa6820 0x5607a8fbd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055400640 1 -- 172.21.15.86:0/2184725148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa6820 msgr2=0x5607a8fbd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055400640 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa6820 0x5607a8fbd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055400640 1 -- 172.21.15.86:0/2184725148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fb1a00 msgr2=0x5607a8fc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055400640 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fb1a00 0x5607a8fc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055400640 1 -- 172.21.15.86:0/2184725148 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a8fc4800 con 0x5607a8fa7140 2024-04-25T19:04:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055e00640 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa6820 0x5607a8fbd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f6055400640 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fa1d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f604001a040 tx=0x7f60400093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a258c5605aabe177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f603e600640 1 -- 172.21.15.86:0/2184725148 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000ec80 con 0x5607a8fa7140 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f603e600640 1 -- 172.21.15.86:0/2184725148 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60400044f0 con 0x5607a8fa7140 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.886+0000 7f603e600640 1 -- 172.21.15.86:0/2184725148 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400047a0 con 0x5607a8fa7140 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 -- 172.21.15.86:0/2184725148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 msgr2=0x5607a8fa1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fa1d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f604001a040 tx=0x7f60400093f0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 -- 172.21.15.86:0/2184725148 shutdown_connections 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fb1a00 0x5607a8fc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fa1d10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 --2- 172.21.15.86:0/2184725148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fa6820 0x5607a8fbd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 -- 172.21.15.86:0/2184725148 >> 172.21.15.86:0/2184725148 conn(0x5607a8fa2d30 msgr2=0x5607a8fa1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 -- 172.21.15.86:0/2184725148 shutdown_connections 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.887+0000 7f605b797740 1 -- 172.21.15.86:0/2184725148 wait complete. 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 -- start start 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 Processor -- start 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fb7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a90212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b1390 con 0x5607a8fa6820 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b0cc0 con 0x5607a8fb1a00 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60501adf00 con 0x5607a8fa7140 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.888+0000 7f605b797740 10 client.0 _ll_register_callbacks cb 0x7f60440bc9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fb7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fb7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36436/0 (socket says 172.21.15.86:36436) 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 -- 172.21.15.86:0/2352172525 learned_addr learned my addr 172.21.15.86:0/2352172525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055400640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6056800640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a90212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 -- 172.21.15.86:0/2352172525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 msgr2=0x5607a8fb81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 -- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 msgr2=0x5607a90212b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a90212b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 -- 172.21.15.86:0/2352172525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a90218a0 con 0x5607a8fa6820 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6055e00640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fb7c60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f60440ee7e0 tx=0x7f604403b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1811c1da0c19a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6033400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.889+0000 7f6033400640 1 -- 172.21.15.86:0/2352172525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044049040 con 0x5607a8fa6820 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f6033400640 1 -- 172.21.15.86:0/2352172525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f60440c5320 con 0x5607a8fa6820 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60400044f0 con 0x5607a8fa6820 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f6033400640 1 -- 172.21.15.86:0/2352172525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60440bd240 con 0x5607a8fa6820 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 10 client.15426 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607a9028c30 con 0x5607a8fa6820 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f6033400640 1 -- 172.21.15.86:0/2352172525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60400044f0 con 0x5607a8fa6820 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f6033400640 1 -- 172.21.15.86:0/2352172525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f60440bbf90 con 0x5607a8fa6820 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 20 client.15426 populate_metadata read hostname 'smithi086' 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a9029210 0x5607a902b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 4 client.15426 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.890+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607a8f1a320 con 0x5607a9029210 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.891+0000 7f6055400640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a9029210 0x5607a902b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.891+0000 7f6055400640 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a9029210 0x5607a902b600 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61557af160a088bf server_cookie=6e464b05d8b77b18 in_seq=0 out_seq=0 2024-04-25T19:04:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.891+0000 7f6033400640 10 client.15426 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f6033400640 1 -- 172.21.15.86:0/2352172525 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607a8f1a320 con 0x5607a9029210 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f6033400640 10 client.15426 handle_command_reply: tid=0 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 client.15426 shutdown 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 msgr2=0x5607a8fb7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fb7c60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f60440ee7e0 tx=0x7f604403b800 comp rx=0 tx=0).stop 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 shutdown_connections 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5607a9029210 0x5607a902b600 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607a8fb1a00 0x5607a90212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5607a8fa7140 0x5607a8fb81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f6033400640 0 client.15426 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 --2- 172.21.15.86:0/2352172525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5607a8fa6820 0x5607a8fb7c60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.904+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 >> 172.21.15.86:0/2352172525 conn(0x5607a8fa2d30 msgr2=0x5607a8fa1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.905+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 shutdown_connections 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.905+0000 7f605b797740 1 -- 172.21.15.86:0/2352172525 wait complete. 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.905+0000 7f605b797740 20 client.15426 trim_cache size 0 max 16384 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:04:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:04:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:04:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:04:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f60380a3e40 msgr2=0x7f60380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f60380a3e40 0x7f60380a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f604c0049d0 tx=0x7f604c009990 comp rx=0 tx=0).stop 2024-04-25T19:04:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 msgr2=0x7f60501a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f60501a0650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6044005ab0 tx=0x7f6044003c10 comp rx=0 tx=0).stop 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 shutdown_connections 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f60380a3e40 0x7f60380a6300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605019a710 0x7f605019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6050107270 0x7f60501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 --2- 172.21.15.86:0/2532322314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60501021a0 0x7f60501a0650 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 >> 172.21.15.86:0/2532322314 conn(0x7f60500fda00 msgr2=0x7f60500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 shutdown_connections 2024-04-25T19:04:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:04:58.906+0000 7f6058800640 1 -- 172.21.15.86:0/2532322314 wait complete. 2024-04-25T19:05:00.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:04:59 smithi086 ceph-mon[27783]: pgmap v64: 97 pgs: 97 active+clean; 1.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-25T19:05:00.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:04:59 smithi183 ceph-mon[32344]: pgmap v64: 97 pgs: 97 active+clean; 1.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-25T19:05:00.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:04:59 smithi194 ceph-mon[32740]: pgmap v64: 97 pgs: 97 active+clean; 1.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-25T19:05:02.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:02 smithi086 ceph-mon[27783]: pgmap v65: 97 pgs: 97 active+clean; 1.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-25T19:05:02.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:02 smithi183 ceph-mon[32344]: pgmap v65: 97 pgs: 97 active+clean; 1.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-25T19:05:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:02 smithi194 ceph-mon[32740]: pgmap v65: 97 pgs: 97 active+clean; 1.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-25T19:05:03.109 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T19:05:03.109 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-25T19:05:03.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:05:03.576 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:05:04.242 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:04 smithi086 ceph-mon[27783]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 36 op/s 2024-04-25T19:05:04.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:04 smithi183 ceph-mon[32344]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 36 op/s 2024-04-25T19:05:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:04 smithi194 ceph-mon[32740]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 36 op/s 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.961+0000 7f7439800700 1 -- 172.21.15.86:0/3614127049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 msgr2=0x7f7434115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.961+0000 7f7439800700 1 --2- 172.21.15.86:0/3614127049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f7434115f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f742c009fc0 tx=0x7f742c00b330 comp rx=0 tx=0).stop 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 -- 172.21.15.86:0/3614127049 shutdown_connections 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 --2- 172.21.15.86:0/3614127049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f7434115f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 --2- 172.21.15.86:0/3614127049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 0x7f74341096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 --2- 172.21.15.86:0/3614127049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 0x7f743410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 -- 172.21.15.86:0/3614127049 >> 172.21.15.86:0/3614127049 conn(0x7f7434076210 msgr2=0x7f7434078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 -- 172.21.15.86:0/3614127049 shutdown_connections 2024-04-25T19:05:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.962+0000 7f7439800700 1 -- 172.21.15.86:0/3614127049 wait complete. 2024-04-25T19:05:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.963+0000 7f7439800700 1 Processor -- start 2024-04-25T19:05:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.963+0000 7f7439800700 1 -- start start 2024-04-25T19:05:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7439800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 0x7f743419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7439800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 0x7f743419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7439800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f74341a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7439800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74341192b0 con 0x7f7434109cc0 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7439800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7434119130 con 0x7f7434109310 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7439800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7434119430 con 0x7f743410e3a0 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7432000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 0x7f743419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f74341a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7432a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 0x7f743419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7432000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 0x7f743419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37280/0 (socket says 172.21.15.86:37280) 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7432000700 1 -- 172.21.15.86:0/2151287061 learned_addr learned my addr 172.21.15.86:0/2151287061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f74341a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46560/0 (socket says 172.21.15.86:46560) 2024-04-25T19:05:05.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 -- 172.21.15.86:0/2151287061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 msgr2=0x7f743419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:05.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 0x7f743419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:05.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 -- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 msgr2=0x7f743419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 0x7f743419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7433400700 1 -- 172.21.15.86:0/2151287061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74341a4730 con 0x7f743410e3a0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.964+0000 7f7432a00700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 0x7f743419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.965+0000 7f7433400700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f74341a3ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f742c00b6c0 tx=0x7f742c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.965+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c021070 con 0x7f743410e3a0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.965+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f742c007ac0 con 0x7f743410e3a0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.965+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74341a49c0 con 0x7f743410e3a0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.965+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c005040 con 0x7f743410e3a0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.965+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74341ac2a0 con 0x7f743410e3a0 2024-04-25T19:05:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.967+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7434069730 con 0x7f743410e3a0 2024-04-25T19:05:05.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.969+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f742c024070 con 0x7f743410e3a0 2024-04-25T19:05:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.970+0000 7f742b400700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f74180a3e30 0x7f74180a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.970+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f742c00f020 con 0x7f743410e3a0 2024-04-25T19:05:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.970+0000 7f7432a00700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f74180a3e30 0x7f74180a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.971+0000 7f7432a00700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f74180a3e30 0x7f74180a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7424011490 tx=0x7f7424009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:05.974+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f742c02b020 con 0x7f743410e3a0 2024-04-25T19:05:06.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:06 smithi086 ceph-mon[27783]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-25T19:05:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.253+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f743402df80 con 0x7f743410e3a0 2024-04-25T19:05:06.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.254+0000 7f742b400700 1 -- 172.21.15.86:0/2151287061 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f742c09e140 con 0x7f743410e3a0 2024-04-25T19:05:06.256 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:05:06.256 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f74180a3e30 msgr2=0x7f74180a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f74180a3e30 0x7f74180a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7424011490 tx=0x7f7424009260 comp rx=0 tx=0).stop 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 msgr2=0x7f74341a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f74341a3ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f742c00b6c0 tx=0x7f742c00cc30 comp rx=0 tx=0).stop 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 shutdown_connections 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f74180a3e30 0x7f74180a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f743410e3a0 0x7f74341a3ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7434109310 0x7f743419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 --2- 172.21.15.86:0/2151287061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7434109cc0 0x7f743419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 >> 172.21.15.86:0/2151287061 conn(0x7f7434076210 msgr2=0x7f7434078530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 shutdown_connections 2024-04-25T19:05:06.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:06.257+0000 7f7439800700 1 -- 172.21.15.86:0/2151287061 wait complete. 2024-04-25T19:05:06.260 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:05:06.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:06 smithi183 ceph-mon[32344]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-25T19:05:06.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:06 smithi194 ceph-mon[32740]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-25T19:05:07.140 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-25T19:05:07.140 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-25T19:05:07.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:07 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2151287061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:07.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:07 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:05:07.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:07 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2151287061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:07.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:07 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:05:07.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:07 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2151287061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:07.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:07 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:05:07.580 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:05:08.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:08 smithi086 ceph-mon[27783]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-25T19:05:08.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:08 smithi183 ceph-mon[32344]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-25T19:05:08.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:08 smithi194 ceph-mon[32740]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 30 op/s 2024-04-25T19:05:09.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:09 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:05:09.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:09 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:05:09.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:09 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:05:09.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:09 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:05:09.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:09 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:05:09.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:09 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:05:09.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:09 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:05:09.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:09 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:05:09.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:09 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:05:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 -- 172.21.15.86:0/2111625652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00410a010 msgr2=0x7fe004111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 --2- 172.21.15.86:0/2111625652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00410a010 0x7fe004111be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b0d0 tx=0x7fdffc00b3e0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 -- 172.21.15.86:0/2111625652 shutdown_connections 2024-04-25T19:05:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 --2- 172.21.15.86:0/2111625652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00410a010 0x7fe004111be0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 --2- 172.21.15.86:0/2111625652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe004104f30 0x7fe004105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 --2- 172.21.15.86:0/2111625652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0041058e0 0x7fe004109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 -- 172.21.15.86:0/2111625652 >> 172.21.15.86:0/2111625652 conn(0x7fe004100800 msgr2=0x7fe004102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 -- 172.21.15.86:0/2111625652 shutdown_connections 2024-04-25T19:05:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.885+0000 7fe008c00700 1 -- 172.21.15.86:0/2111625652 wait complete. 2024-04-25T19:05:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.886+0000 7fe008c00700 1 Processor -- start 2024-04-25T19:05:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 -- start start 2024-04-25T19:05:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe004104f30 0x7fe00419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0041058e0 0x7fe00419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe00410a010 0x7fe00419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe004114ec0 con 0x7fe0041058e0 2024-04-25T19:05:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe004114d40 con 0x7fe004104f30 2024-04-25T19:05:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe008c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe004115040 con 0x7fe00410a010 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.887+0000 7fe001400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0041058e0 0x7fe00419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe004104f30 0x7fe00419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0041058e0 0x7fe00419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38226/0 (socket says 172.21.15.86:38226) 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001400700 1 -- 172.21.15.86:0/1767590719 learned_addr learned my addr 172.21.15.86:0/1767590719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe002800700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe00410a010 0x7fe00419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 -- 172.21.15.86:0/1767590719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe00410a010 msgr2=0x7fe00419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe00410a010 0x7fe00419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 -- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0041058e0 msgr2=0x7fe00419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0041058e0 0x7fe00419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 -- 172.21.15.86:0/1767590719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0041a0280 con 0x7fe004104f30 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.888+0000 7fe001e00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe004104f30 0x7fe00419b270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9f0 tx=0x7fdff400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.889+0000 7fdffaa00700 1 -- 172.21.15.86:0/1767590719 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4004220 con 0x7fe004104f30 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.889+0000 7fdffaa00700 1 -- 172.21.15.86:0/1767590719 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff40043c0 con 0x7fe004104f30 2024-04-25T19:05:09.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.889+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0041a0570 con 0x7fe004104f30 2024-04-25T19:05:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.889+0000 7fdffaa00700 1 -- 172.21.15.86:0/1767590719 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4005020 con 0x7fe004104f30 2024-04-25T19:05:09.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.889+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0041a7df0 con 0x7fe004104f30 2024-04-25T19:05:09.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.892+0000 7fdffaa00700 1 -- 172.21.15.86:0/1767590719 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdff4004750 con 0x7fe004104f30 2024-04-25T19:05:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 Processor -- start 2024-04-25T19:05:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 -- start start 2024-04-25T19:05:09.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fdffaa00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdfe80a3ed0 0x7fdfe80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fdffaa00700 1 -- 172.21.15.86:0/1767590719 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fdff40d0710 con 0x7fe004104f30 2024-04-25T19:05:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e58201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e582b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e582d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d6e5773020 con 0x55d6e58206f0 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d6e57198c0 con 0x55d6e582b560 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe01357cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d6e576efe0 con 0x55d6e581fdd0 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.894+0000 7fe002800700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdfe80a3ed0 0x7fdfe80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e582b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e58201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e582b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38238/0 (socket says 172.21.15.86:38238) 2024-04-25T19:05:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001400700 1 -- 172.21.15.86:0/3479842356 learned_addr learned my addr 172.21.15.86:0/3479842356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:09.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001e00700 1 -- 172.21.15.86:0/3479842356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 msgr2=0x55d6e582d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001e00700 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e582d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001e00700 1 -- 172.21.15.86:0/3479842356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 msgr2=0x55d6e582b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001e00700 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e582b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001e00700 1 -- 172.21.15.86:0/3479842356 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e570f810 con 0x55d6e581fdd0 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe001400700 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e582b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.895+0000 7fe002800700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdfe80a3ed0 0x7fdfe80a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009fd0 tx=0x7fdffc009c80 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.896+0000 7fe001e00700 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e58201b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdff40cde10 tx=0x7fdff409bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecfc441097c3e37e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.896+0000 7fdfef400700 1 -- 172.21.15.86:0/3479842356 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff401c040 con 0x55d6e581fdd0 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.896+0000 7fdfef400700 1 -- 172.21.15.86:0/3479842356 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff409c660 con 0x55d6e581fdd0 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.896+0000 7fdfef400700 1 -- 172.21.15.86:0/3479842356 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff409c8c0 con 0x55d6e581fdd0 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 -- 172.21.15.86:0/3479842356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 msgr2=0x55d6e58201b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e58201b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdff40cde10 tx=0x7fdff409bfc0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 -- 172.21.15.86:0/3479842356 shutdown_connections 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e582d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e58201b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3479842356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e582b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 -- 172.21.15.86:0/3479842356 >> 172.21.15.86:0/3479842356 conn(0x55d6e574f0e0 msgr2=0x55d6e581c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 -- 172.21.15.86:0/3479842356 shutdown_connections 2024-04-25T19:05:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 -- 172.21.15.86:0/3479842356 wait complete. 2024-04-25T19:05:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.897+0000 7fe01357cb80 1 -- start start 2024-04-25T19:05:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 Processor -- start 2024-04-25T19:05:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e582ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e5828010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5828550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d6e582e050 con 0x55d6e58206f0 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d6e572c9f0 con 0x55d6e582b560 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d6e582e1d0 con 0x55d6e581fdd0 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.898+0000 7fe01357cb80 10 client.0 _ll_register_callbacks cb 0x55d6e570b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e582ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e582ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:43940/0 (socket says 172.21.15.86:43940) 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 -- 172.21.15.86:0/1607555304 learned_addr learned my addr 172.21.15.86:0/1607555304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001400700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e5828010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe002800700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5828550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 -- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 msgr2=0x55d6e5828550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5828550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 -- 172.21.15.86:0/1607555304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 msgr2=0x55d6e5828010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e5828010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 -- 172.21.15.86:0/1607555304 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e56c0a60 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fe001e00700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e582ad10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9c0 tx=0x7fdff40a0a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b77ccf6de845449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.899+0000 7fdfe3e00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fdfe3e00700 1 -- 172.21.15.86:0/1607555304 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40b9070 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fdfe3e00700 1 -- 172.21.15.86:0/1607555304 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fdff409c8c0 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff409c660 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fdfe3e00700 1 -- 172.21.15.86:0/1607555304 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdff401c030 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 10 client.25546 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d6e5709c80 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fdfe3e00700 1 -- 172.21.15.86:0/1607555304 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff409c660 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fdfe3e00700 1 -- 172.21.15.86:0/1607555304 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdff40bb2c0 con 0x55d6e581fdd0 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 10 client.25546 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 10 client.25546 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 20 client.25546 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e583acd0 0x55d6e583d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 4 client.25546 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.900+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6e5780810 con 0x55d6e583acd0 2024-04-25T19:05:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.901+0000 7fe001400700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e583acd0 0x55d6e583d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.901+0000 7fe001400700 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e583acd0 0x55d6e583d0c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80bfd70b62b942d server_cookie=1a65c5e335aefafe in_seq=0 out_seq=0 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.901+0000 7fdfe3e00700 10 client.25546 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.902+0000 7fdfe3e00700 1 -- 172.21.15.86:0/1607555304 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6e5780810 con 0x55d6e583acd0 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.902+0000 7fdfe3e00700 10 client.25546 handle_command_reply: tid=0 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 client.25546 shutdown 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 msgr2=0x55d6e582ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e582ad10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9c0 tx=0x7fdff40a0a80 comp rx=0 tx=0).stop 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 shutdown_connections 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e583acd0 0x55d6e583d0c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5828550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e582ad10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 --2- 172.21.15.86:0/1607555304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e5828010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fdfe3e00700 0 client.25546 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 >> 172.21.15.86:0/1607555304 conn(0x55d6e574f0e0 msgr2=0x55d6e581bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 shutdown_connections 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 1 -- 172.21.15.86:0/1607555304 wait complete. 2024-04-25T19:05:09.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.903+0000 7fe01357cb80 20 client.25546 trim_cache size 0 max 16384 2024-04-25T19:05:09.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 Processor -- start 2024-04-25T19:05:09.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 -- start start 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e581fdd0 0x55d6e5828640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 0x55d6e58400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e582b560 0x55d6e5840630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041af600 con 0x55d6e582b560 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041b1140 con 0x55d6e581fdd0 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.925+0000 7fe01357cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041afcd0 con 0x55d6e58206f0 2024-04-25T19:05:09.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe002800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e582b560 0x55d6e5840630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 0x55d6e58400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 0x55d6e58400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:43952/0 (socket says 172.21.15.86:43952) 2024-04-25T19:05:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 -- 172.21.15.86:0/3585544494 learned_addr learned my addr 172.21.15.86:0/3585544494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:09.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001e00700 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e581fdd0 0x55d6e5828640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 -- 172.21.15.86:0/3585544494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e581fdd0 msgr2=0x55d6e5828640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e581fdd0 0x55d6e5828640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 -- 172.21.15.86:0/3585544494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e582b560 msgr2=0x55d6e5840630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e582b560 0x55d6e5840630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 -- 172.21.15.86:0/3585544494 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e581cdb0 con 0x55d6e58206f0 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.926+0000 7fe001400700 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 0x55d6e58400f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55d6e58295e0 tx=0x7fdff0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e3e6e9ff0e3989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fdfeea00700 1 -- 172.21.15.86:0/3585544494 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0003ca0 con 0x55d6e58206f0 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fdfeea00700 1 -- 172.21.15.86:0/3585544494 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff0003e40 con 0x55d6e58206f0 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fdfeea00700 1 -- 172.21.15.86:0/3585544494 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0005700 con 0x55d6e58206f0 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 -- 172.21.15.86:0/3585544494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 msgr2=0x55d6e58400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 0x55d6e58400f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55d6e58295e0 tx=0x7fdff0009260 comp rx=0 tx=0).stop 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 -- 172.21.15.86:0/3585544494 shutdown_connections 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e582b560 0x55d6e5840630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e581fdd0 0x55d6e5828640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 --2- 172.21.15.86:0/3585544494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e58206f0 0x55d6e58400f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.927+0000 7fe01357cb80 1 -- 172.21.15.86:0/3585544494 >> 172.21.15.86:0/3585544494 conn(0x55d6e574f0e0 msgr2=0x55d6e582a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.928+0000 7fe01357cb80 1 -- 172.21.15.86:0/3585544494 shutdown_connections 2024-04-25T19:05:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.928+0000 7fe01357cb80 1 -- 172.21.15.86:0/3585544494 wait complete. 2024-04-25T19:05:09.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.928+0000 7fe01357cb80 1 -- start start 2024-04-25T19:05:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.928+0000 7fe01357cb80 1 Processor -- start 2024-04-25T19:05:09.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e589b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e589b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5895510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041b0a70 con 0x55d6e58206f0 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041b03a0 con 0x55d6e582b560 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041ad5a0 con 0x55d6e581fdd0 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe01357cb80 10 client.0 _ll_register_callbacks cb 0x55d6e570b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe001e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e589b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe001e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e589b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:43966/0 (socket says 172.21.15.86:43966) 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe001e00700 1 -- 172.21.15.86:0/262608782 learned_addr learned my addr 172.21.15.86:0/262608782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe001400700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e589b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.929+0000 7fe002800700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5895510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe001e00700 1 -- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 msgr2=0x55d6e5895510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe001e00700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5895510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe001e00700 1 -- 172.21.15.86:0/262608782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 msgr2=0x55d6e589b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe001e00700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e589b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe001e00700 1 -- 172.21.15.86:0/262608782 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e5895c10 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe001e00700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e589b0f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdff4098fe0 tx=0x7fdff400c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9beb4015f3ce33d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fdfe3e00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fdfe3e00700 1 -- 172.21.15.86:0/262608782 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40b9070 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fdfe3e00700 1 -- 172.21.15.86:0/262608782 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fdff40a4e40 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.930+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff0003e40 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fdfe3e00700 1 -- 172.21.15.86:0/262608782 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdff4098710 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 10 client.25552 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d6e5895c10 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fdfe3e00700 1 -- 172.21.15.86:0/262608782 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff0003e40 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fdfe3e00700 1 -- 172.21.15.86:0/262608782 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdff409c8c0 con 0x55d6e581fdd0 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 10 client.25552 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:09.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 10 client.25552 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 20 client.25552 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e58a2bb0 0x55d6e58a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 4 client.25552 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55d6e577a8b0 con 0x55d6e58a2bb0 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.931+0000 7fe001400700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e58a2bb0 0x55d6e58a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.932+0000 7fe001400700 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e58a2bb0 0x55d6e58a4fa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea376ccc90b5daff server_cookie=4b2823d343f0a7ae in_seq=0 out_seq=0 2024-04-25T19:05:09.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.932+0000 7fdfe3e00700 10 client.25552 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fdfe3e00700 1 -- 172.21.15.86:0/262608782 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55d6e577a8b0 con 0x55d6e58a2bb0 2024-04-25T19:05:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fdfe3e00700 10 client.25552 handle_command_reply: tid=0 2024-04-25T19:05:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 client.25552 shutdown 2024-04-25T19:05:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 msgr2=0x55d6e589b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e589b0f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdff4098fe0 tx=0x7fdff400c960 comp rx=0 tx=0).stop 2024-04-25T19:05:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 shutdown_connections 2024-04-25T19:05:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d6e58a2bb0 0x55d6e58a4fa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d6e582b560 0x55d6e5895510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6e581fdd0 0x55d6e589b0f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 --2- 172.21.15.86:0/262608782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6e58206f0 0x55d6e589b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fdfe3e00700 0 client.25552 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 >> 172.21.15.86:0/262608782 conn(0x55d6e574f0e0 msgr2=0x55d6e5836460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 shutdown_connections 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.952+0000 7fe01357cb80 1 -- 172.21.15.86:0/262608782 wait complete. 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.953+0000 7fe01357cb80 20 client.25552 trim_cache size 0 max 16384 2024-04-25T19:05:09.956 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 6, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "57c7c7b5": { 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 6, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 90.0, 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:05:09.957 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:09.958 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdfe80a3ed0 msgr2=0x7fdfe80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdfe80a3ed0 0x7fdfe80a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009fd0 tx=0x7fdffc009c80 comp rx=0 tx=0).stop 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe004104f30 msgr2=0x7fe00419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe004104f30 0x7fe00419b270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9f0 tx=0x7fdff400ed00 comp rx=0 tx=0).stop 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 shutdown_connections 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdfe80a3ed0 0x7fdfe80a6390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe00410a010 0x7fe00419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe004104f30 0x7fe00419b270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 --2- 172.21.15.86:0/1767590719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0041058e0 0x7fe00419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 >> 172.21.15.86:0/1767590719 conn(0x7fe004100800 msgr2=0x7fe004101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 shutdown_connections 2024-04-25T19:05:09.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:09.954+0000 7fe008c00700 1 -- 172.21.15.86:0/1767590719 wait complete. 2024-04-25T19:05:10.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:10 smithi086 ceph-mon[27783]: pgmap v69: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-25T19:05:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:10 smithi194 ceph-mon[32740]: pgmap v69: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-25T19:05:10.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:10 smithi183 ceph-mon[32344]: pgmap v69: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-25T19:05:10.935 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '57c7c7b5', quiesce took 7.8 seconds. Will release after: 30.0 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 -- 172.21.15.86:0/2268080533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff428108560 msgr2=0x7ff4281089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 --2- 172.21.15.86:0/2268080533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff428108560 0x7ff4281089e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff41402a400 comp rx=0 tx=0).stop 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 -- 172.21.15.86:0/2268080533 shutdown_connections 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 --2- 172.21.15.86:0/2268080533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108f20 0x7ff4281153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 --2- 172.21.15.86:0/2268080533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff428108560 0x7ff4281089e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 --2- 172.21.15.86:0/2268080533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff42810e620 0x7ff42810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 -- 172.21.15.86:0/2268080533 >> 172.21.15.86:0/2268080533 conn(0x7ff4280764e0 msgr2=0x7ff428078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 -- 172.21.15.86:0/2268080533 shutdown_connections 2024-04-25T19:05:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.049+0000 7ff430000640 1 -- 172.21.15.86:0/2268080533 wait complete. 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.050+0000 7ff430000640 1 Processor -- start 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.050+0000 7ff430000640 1 -- start start 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff430000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 0x7ff4281a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff430000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108f20 0x7ff4281a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff430000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 0x7ff42819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff430000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281187e0 con 0x7ff42810e620 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff430000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff428118660 con 0x7ff428108f20 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff430000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff428118960 con 0x7ff428108560 2024-04-25T19:05:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 0x7ff4281a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 0x7ff42819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 0x7ff4281a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:43988/0 (socket says 172.21.15.86:43988) 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 -- 172.21.15.86:0/790522573 learned_addr learned my addr 172.21.15.86:0/790522573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 0x7ff42819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38278/0 (socket says 172.21.15.86:38278) 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42cc00640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108f20 0x7ff4281a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 -- 172.21.15.86:0/790522573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108f20 msgr2=0x7ff4281a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108f20 0x7ff4281a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 -- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 msgr2=0x7ff42819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 0x7ff42819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42d600640 1 -- 172.21.15.86:0/790522573 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42819f240 con 0x7ff428108560 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.051+0000 7ff42e000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 0x7ff42819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.052+0000 7ff42d600640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 0x7ff4281a48c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9f0 tx=0x7ff41c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.052+0000 7ff426000640 1 -- 172.21.15.86:0/790522573 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cd80 con 0x7ff428108560 2024-04-25T19:05:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.052+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42819f530 con 0x7ff428108560 2024-04-25T19:05:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.052+0000 7ff426000640 1 -- 172.21.15.86:0/790522573 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c0108e0 con 0x7ff428108560 2024-04-25T19:05:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.052+0000 7ff426000640 1 -- 172.21.15.86:0/790522573 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c018980 con 0x7ff428108560 2024-04-25T19:05:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.052+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4281ab780 con 0x7ff428108560 2024-04-25T19:05:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.055+0000 7ff432d7f740 1 Processor -- start 2024-04-25T19:05:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.055+0000 7ff432d7f740 1 -- start start 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff426000640 1 -- 172.21.15.86:0/790522573 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff41c010430 con 0x7ff428108560 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 0x55757d5b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b4140 0x55757d5be4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d5c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff432d7f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55757d519900 con 0x55757d5bea00 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff432d7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55757d4bfef0 con 0x55757d5b4140 2024-04-25T19:05:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff432d7f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55757d5158c0 con 0x55757d5b3820 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff42cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d5c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff42e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 0x55757d5b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff42d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b4140 0x55757d5be4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff42cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d5c0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38282/0 (socket says 172.21.15.86:38282) 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff42cc00640 1 -- 172.21.15.86:0/3958412615 learned_addr learned my addr 172.21.15.86:0/3958412615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.056+0000 7ff42e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 0x55757d5b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:43992/0 (socket says 172.21.15.86:43992) 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff42e000640 1 -- 172.21.15.86:0/3958412615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b4140 msgr2=0x55757d5be4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff42e000640 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b4140 0x55757d5be4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff42e000640 1 -- 172.21.15.86:0/3958412615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 msgr2=0x55757d5c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff42e000640 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d5c0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff42e000640 1 -- 172.21.15.86:0/3958412615 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757d22c210 con 0x55757d5b3820 2024-04-25T19:05:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff42e000640 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 0x55757d5b3c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff420010690 tx=0x7ff420010b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fedff7dcc8979475 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff41a600640 1 -- 172.21.15.86:0/3958412615 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420004b20 con 0x55757d5b3820 2024-04-25T19:05:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff426000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff4100a3dc0 0x7ff4100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff41a600640 1 -- 172.21.15.86:0/3958412615 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42000ed80 con 0x55757d5b3820 2024-04-25T19:05:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.057+0000 7ff41a600640 1 -- 172.21.15.86:0/3958412615 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420005780 con 0x55757d5b3820 2024-04-25T19:05:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff426000640 1 -- 172.21.15.86:0/790522573 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ff41c0d0b90 con 0x7ff428108560 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff42d600640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff4100a3dc0 0x7ff4100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 -- 172.21.15.86:0/3958412615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 msgr2=0x55757d5b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 0x55757d5b3c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff420010690 tx=0x7ff420010b50 comp rx=0 tx=0).stop 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 -- 172.21.15.86:0/3958412615 shutdown_connections 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d5c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b4140 0x55757d5be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 --2- 172.21.15.86:0/3958412615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b3820 0x55757d5b3c00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 -- 172.21.15.86:0/3958412615 >> 172.21.15.86:0/3958412615 conn(0x55757d5afd30 msgr2=0x55757d5b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 -- 172.21.15.86:0/3958412615 shutdown_connections 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.058+0000 7ff432d7f740 1 -- 172.21.15.86:0/3958412615 wait complete. 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.059+0000 7ff42d600640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff4100a3dc0 0x7ff4100a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55757d5b4dc0 tx=0x7ff41c0d4640 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.059+0000 7ff432d7f740 1 -- start start 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.059+0000 7ff432d7f740 1 Processor -- start 2024-04-25T19:05:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.059+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff432d7f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55757d5c1630 con 0x55757d5b3820 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff432d7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55757d4d3310 con 0x55757d5bea00 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff432d7f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55757d5c17b0 con 0x55757d5b4140 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff432d7f740 10 client.0 _ll_register_callbacks cb 0x55757d4a1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5bb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38290/0 (socket says 172.21.15.86:38290) 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 -- 172.21.15.86:0/104629200 learned_addr learned my addr 172.21.15.86:0/104629200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 -- 172.21.15.86:0/104629200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 msgr2=0x55757d5bbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 -- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 msgr2=0x55757d5bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42e000640 1 -- 172.21.15.86:0/104629200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757d2f8620 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.060+0000 7ff42cc00640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff42e000640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5bb9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff420010e20 tx=0x7ff420005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52e2e4a610cfc9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff40ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff40ea00640 1 -- 172.21.15.86:0/104629200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420004a40 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff40ea00640 1 -- 172.21.15.86:0/104629200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ff420004be0 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42000ed80 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff40ea00640 1 -- 172.21.15.86:0/104629200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff420004de0 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff432d7f740 10 client.15462 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55757d5cd8f0 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.061+0000 7ff40ea00640 1 -- 172.21.15.86:0/104629200 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42000ed80 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff40ea00640 1 -- 172.21.15.86:0/104629200 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff42000f040 con 0x55757d5b3820 2024-04-25T19:05:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff432d7f740 10 client.15462 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff432d7f740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff432d7f740 20 client.15462 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff432d7f740 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d5ceca0 0x55757d5d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff432d7f740 4 client.15462 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757d52d8b0 con 0x55757d5ceca0 2024-04-25T19:05:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.062+0000 7ff42cc00640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d5ceca0 0x55757d5d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.063+0000 7ff42cc00640 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d5ceca0 0x55757d5d1090 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea181bff014f56f8 server_cookie=e3266f46d410f903 in_seq=0 out_seq=0 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.063+0000 7ff40ea00640 10 client.15462 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff40ea00640 1 -- 172.21.15.86:0/104629200 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55757d52d8b0 con 0x55757d5ceca0 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff40ea00640 10 client.15462 handle_command_reply: tid=0 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 client.15462 shutdown 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 msgr2=0x55757d5bb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5bb9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff420010e20 tx=0x7ff420005380 comp rx=0 tx=0).stop 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 shutdown_connections 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d5ceca0 0x55757d5d1090 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff40ea00640 0 client.15462 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 --2- 172.21.15.86:0/104629200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5bb9f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 >> 172.21.15.86:0/104629200 conn(0x55757d5afd30 msgr2=0x55757d5b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 shutdown_connections 2024-04-25T19:05:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 1 -- 172.21.15.86:0/104629200 wait complete. 2024-04-25T19:05:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.064+0000 7ff432d7f740 20 client.15462 trim_cache size 0 max 16384 2024-04-25T19:05:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.080+0000 7ff432d7f740 1 Processor -- start 2024-04-25T19:05:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 -- start start 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b42d0 con 0x55757d5b3820 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b5e10 con 0x55757d5bea00 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff432d7f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b49a0 con 0x55757d5b4140 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38304/0 (socket says 172.21.15.86:38304) 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff42e000640 1 -- 172.21.15.86:0/1714838841 learned_addr learned my addr 172.21.15.86:0/1714838841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff42cc00640 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.081+0000 7ff42e000640 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42cc00640 1 -- 172.21.15.86:0/1714838841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 msgr2=0x55757d5d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42cc00640 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42cc00640 1 -- 172.21.15.86:0/1714838841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 msgr2=0x55757d5ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42cc00640 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42cc00640 1 -- 172.21.15.86:0/1714838841 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757d5ab440 con 0x55757d5b4140 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42e000640 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff42cc00640 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bb1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55757d5b1150 tx=0x7ff414005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40e87df1bbdbd0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff419c00640 1 -- 172.21.15.86:0/1714838841 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414035040 con 0x55757d5b4140 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff419c00640 1 -- 172.21.15.86:0/1714838841 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff414046830 con 0x55757d5b4140 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.082+0000 7ff419c00640 1 -- 172.21.15.86:0/1714838841 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414005580 con 0x55757d5b4140 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 -- 172.21.15.86:0/1714838841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 msgr2=0x55757d5bb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bb1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55757d5b1150 tx=0x7ff414005e80 comp rx=0 tx=0).stop 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 -- 172.21.15.86:0/1714838841 shutdown_connections 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5bea00 0x55757d5d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5bb1a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 --2- 172.21.15.86:0/1714838841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5b3820 0x55757d5ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 -- 172.21.15.86:0/1714838841 >> 172.21.15.86:0/1714838841 conn(0x55757d5afd30 msgr2=0x55757d5aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 -- 172.21.15.86:0/1714838841 shutdown_connections 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 -- 172.21.15.86:0/1714838841 wait complete. 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.083+0000 7ff432d7f740 1 -- start start 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 Processor -- start 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b3820 0x55757d5c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d62bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b5740 con 0x55757d5bea00 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b5070 con 0x55757d5b3820 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b22b0 con 0x55757d5b4140 2024-04-25T19:05:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff432d7f740 10 client.0 _ll_register_callbacks cb 0x7ff41c0d4510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d62bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d62bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38312/0 (socket says 172.21.15.86:38312) 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff42e000640 1 -- 172.21.15.86:0/929281510 learned_addr learned my addr 172.21.15.86:0/929281510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.084+0000 7ff42cc00640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b3820 0x55757d5c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 -- 172.21.15.86:0/929281510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 msgr2=0x55757d5c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 -- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b3820 msgr2=0x55757d5c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b3820 0x55757d5c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 -- 172.21.15.86:0/929281510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757d62c530 con 0x55757d5bea00 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff42e000640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d62bec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff420035fd0 tx=0x7ff42001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc430962b98f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff40ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff40ea00640 1 -- 172.21.15.86:0/929281510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420005b30 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff40ea00640 1 -- 172.21.15.86:0/929281510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ff42002e0d0 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.085+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414046830 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff40ea00640 1 -- 172.21.15.86:0/929281510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff420047050 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 10 client.15468 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55757d634ae0 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff40ea00640 1 -- 172.21.15.86:0/929281510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff414046830 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff40ea00640 1 -- 172.21.15.86:0/929281510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff42000ece0 con 0x55757d5bea00 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 20 client.15468 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d635090 0x55757d637480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 4 client.15468 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.086+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55757d527320 con 0x55757d635090 2024-04-25T19:05:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.087+0000 7ff42cc00640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d635090 0x55757d637480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.087+0000 7ff42cc00640 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d635090 0x55757d637480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d557ff9cb340a5b6 server_cookie=89a47e12f6128339 in_seq=0 out_seq=0 2024-04-25T19:05:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.087+0000 7ff40ea00640 10 client.15468 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.100+0000 7ff40ea00640 1 -- 172.21.15.86:0/929281510 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55757d527320 con 0x55757d635090 2024-04-25T19:05:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.100+0000 7ff40ea00640 10 client.15468 handle_command_reply: tid=0 2024-04-25T19:05:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.100+0000 7ff432d7f740 1 client.15468 shutdown 2024-04-25T19:05:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 msgr2=0x55757d62bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d62bec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff420035fd0 tx=0x7ff42001f040 comp rx=0 tx=0).stop 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 shutdown_connections 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55757d635090 0x55757d637480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757d5bea00 0x55757d62bec0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55757d5b4140 0x55757d5c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 --2- 172.21.15.86:0/929281510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55757d5b3820 0x55757d5c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff40ea00640 0 client.15468 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 >> 172.21.15.86:0/929281510 conn(0x55757d5afd30 msgr2=0x55757d5ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 shutdown_connections 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 1 -- 172.21.15.86:0/929281510 wait complete. 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.101+0000 7ff432d7f740 20 client.15468 trim_cache size 0 max 16384 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:05:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:05:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff4100a3dc0 msgr2=0x7ff4100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff4100a3dc0 0x7ff4100a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55757d5b4dc0 tx=0x7ff41c0d4640 comp rx=0 tx=0).stop 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 msgr2=0x7ff4281a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 0x7ff4281a48c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9f0 tx=0x7ff41c00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 shutdown_connections 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff4100a3dc0 0x7ff4100a6280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42810e620 0x7ff42819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108f20 0x7ff4281a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 --2- 172.21.15.86:0/790522573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff428108560 0x7ff4281a48c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 >> 172.21.15.86:0/790522573 conn(0x7ff4280764e0 msgr2=0x7ff428077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 shutdown_connections 2024-04-25T19:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:12.102+0000 7ff430000640 1 -- 172.21.15.86:0/790522573 wait complete. 2024-04-25T19:05:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:12 smithi194 ceph-mon[32740]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-25T19:05:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:12 smithi086 ceph-mon[27783]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-25T19:05:12.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:12 smithi183 ceph-mon[32344]: pgmap v70: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-25T19:05:14.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:14 smithi194 ceph-mon[32740]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 53 KiB/s wr, 38 op/s 2024-04-25T19:05:14.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:14 smithi086 ceph-mon[27783]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 53 KiB/s wr, 38 op/s 2024-04-25T19:05:14.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:14 smithi183 ceph-mon[32344]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 53 KiB/s wr, 38 op/s 2024-04-25T19:05:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:16 smithi194 ceph-mon[32740]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T19:05:16.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:16 smithi086 ceph-mon[27783]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T19:05:16.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:16 smithi183 ceph-mon[32344]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T19:05:18.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:18 smithi086 ceph-mon[27783]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T19:05:18.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:18 smithi194 ceph-mon[32740]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T19:05:18.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:18 smithi183 ceph-mon[32344]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T19:05:20.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:20 smithi194 ceph-mon[32740]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-25T19:05:20.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:20 smithi086 ceph-mon[27783]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-25T19:05:20.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:20 smithi183 ceph-mon[32344]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-25T19:05:22.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:22 smithi194 ceph-mon[32740]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-04-25T19:05:22.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:22 smithi086 ceph-mon[27783]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-04-25T19:05:22.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:22 smithi183 ceph-mon[32344]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-04-25T19:05:24.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:24 smithi194 ceph-mon[32740]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-04-25T19:05:24.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:24 smithi086 ceph-mon[27783]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-04-25T19:05:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:24 smithi183 ceph-mon[32344]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-04-25T19:05:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 -- 172.21.15.86:0/4119900843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0104bb0 msgr2=0x7f3ed0108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 --2- 172.21.15.86:0/4119900843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0104bb0 0x7f3ed0108bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009900 tx=0x7f3ebc02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 -- 172.21.15.86:0/4119900843 shutdown_connections 2024-04-25T19:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 --2- 172.21.15.86:0/4119900843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ed01092d0 0x7f3ed0110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 --2- 172.21.15.86:0/4119900843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed0104bb0 0x7f3ed0108bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 --2- 172.21.15.86:0/4119900843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 -- 172.21.15.86:0/4119900843 >> 172.21.15.86:0/4119900843 conn(0x7f3ed00ffa60 msgr2=0x7f3ed0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.250+0000 7f3ed6800640 1 -- 172.21.15.86:0/4119900843 shutdown_connections 2024-04-25T19:05:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.251+0000 7f3ed6800640 1 -- 172.21.15.86:0/4119900843 wait complete. 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.251+0000 7f3ed6800640 1 Processor -- start 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 -- start start 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ed01092d0 0x7f3ed01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed019a5e0 0x7f3ed019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0114240 con 0x7f3ed019a5e0 2024-04-25T19:05:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01140c0 con 0x7f3ed01092d0 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed6800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01143c0 con 0x7f3ed0104200 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed019a5e0 0x7f3ed019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:45328/0 (socket says 172.21.15.86:45328) 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ed01092d0 0x7f3ed01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.252+0000 7f3ed5400640 1 -- 172.21.15.86:0/1186536629 learned_addr learned my addr 172.21.15.86:0/1186536629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5e00640 1 -- 172.21.15.86:0/1186536629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 msgr2=0x7f3ed01a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5e00640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5e00640 1 -- 172.21.15.86:0/1186536629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ed01092d0 msgr2=0x7f3ed01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5e00640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ed01092d0 0x7f3ed01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5e00640 1 -- 172.21.15.86:0/1186536629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed019b100 con 0x7f3ed019a5e0 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5400640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed5e00640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed019a5e0 0x7f3ed019aa90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009fd0 tx=0x7f3ec800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ece000640 1 -- 172.21.15.86:0/1186536629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800ce20 con 0x7f3ed019a5e0 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ece000640 1 -- 172.21.15.86:0/1186536629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec8005020 con 0x7f3ed019a5e0 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ece000640 1 -- 172.21.15.86:0/1186536629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8010690 con 0x7f3ed019a5e0 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.253+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed019b3f0 con 0x7f3ed019a5e0 2024-04-25T19:05:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.254+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01a75f0 con 0x7f3ed019a5e0 2024-04-25T19:05:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.257+0000 7f3ece000640 1 -- 172.21.15.86:0/1186536629 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ec80054d0 con 0x7f3ed019a5e0 2024-04-25T19:05:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.257+0000 7f3edabfd740 1 Processor -- start 2024-04-25T19:05:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.257+0000 7f3edabfd740 1 -- start start 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddab9ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddababf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3edabfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ddab04900 con 0x561ddab9e9b0 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3edabfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ddaaaaef0 con 0x561ddab9f2d0 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3edabfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561ddab008c0 con 0x561ddaba9b90 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddab9ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddab9ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54608/0 (socket says 172.21.15.86:54608) 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3516727950 learned_addr learned my addr 172.21.15.86:0/3516727950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed5400640 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddababf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3516727950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 msgr2=0x561ddababf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddababf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3516727950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 msgr2=0x561ddaba9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.258+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3516727950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dda817210 con 0x561ddab9e9b0 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddab9ed90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc006010 tx=0x7f3ebc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7668b575cf2197d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ece000640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3eb80a3d90 0x7f3eb80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ec2600640 1 -- 172.21.15.86:0/3516727950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc044020 con 0x561ddab9e9b0 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ec2600640 1 -- 172.21.15.86:0/3516727950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ebc004750 con 0x561ddab9e9b0 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ec2600640 1 -- 172.21.15.86:0/3516727950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035040 con 0x561ddab9e9b0 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ece000640 1 -- 172.21.15.86:0/1186536629 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f3ec80d0ea0 con 0x7f3ed019a5e0 2024-04-25T19:05:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.259+0000 7f3ed5400640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3eb80a3d90 0x7f3eb80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 -- 172.21.15.86:0/3516727950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 msgr2=0x561ddab9ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddab9ed90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc006010 tx=0x7f3ebc005e50 comp rx=0 tx=0).stop 2024-04-25T19:05:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3ed5400640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3eb80a3d90 0x7f3eb80a6250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009620 tx=0x7f3ec4005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 -- 172.21.15.86:0/3516727950 shutdown_connections 2024-04-25T19:05:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddababf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3516727950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddab9ed90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 -- 172.21.15.86:0/3516727950 >> 172.21.15.86:0/3516727950 conn(0x561ddab9aec0 msgr2=0x561ddab9b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 -- 172.21.15.86:0/3516727950 shutdown_connections 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.260+0000 7f3edabfd740 1 -- 172.21.15.86:0/3516727950 wait complete. 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 -- start start 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 Processor -- start 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddaba6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddaba7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddaba75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ddabac760 con 0x561ddaba9b90 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ddaabe310 con 0x561ddab9e9b0 2024-04-25T19:05:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561ddabac8e0 con 0x561ddab9f2d0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.261+0000 7f3edabfd740 10 client.0 _ll_register_callbacks cb 0x561ddaa8ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddaba7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddaba75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddaba7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:45346/0 (socket says 172.21.15.86:45346) 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3050221688 learned_addr learned my addr 172.21.15.86:0/3050221688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed5400640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddaba6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 msgr2=0x561ddaba6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddaba6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3050221688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 msgr2=0x561ddaba75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddaba75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 -- 172.21.15.86:0/3050221688 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dda8e3620 con 0x561ddab9f2d0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed5e00640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddaba75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddaba7090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc004b90 tx=0x7f3ebc004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a72048add4acabc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.262+0000 7f3eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3eb6a00640 1 -- 172.21.15.86:0/3050221688 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc044030 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ebc004750 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3eb6a00640 1 -- 172.21.15.86:0/3050221688 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f3ebc005410 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3eb6a00640 1 -- 172.21.15.86:0/3050221688 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc056050 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3edabfd740 10 client.25582 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ddabac760 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.263+0000 7f3eb6a00640 1 -- 172.21.15.86:0/3050221688 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ebc004750 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3eb6a00640 1 -- 172.21.15.86:0/3050221688 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3ebc051030 con 0x561ddab9f2d0 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3edabfd740 10 client.25582 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3edabfd740 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3edabfd740 20 client.25582 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddabb9b70 0x561ddabbbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3edabfd740 4 client.25582 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ddab188b0 con 0x561ddabb9b70 2024-04-25T19:05:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.264+0000 7f3ed5400640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddabb9b70 0x561ddabbbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.265+0000 7f3ed5400640 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddabb9b70 0x561ddabbbf60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2a9fb82a0df25a server_cookie=214d7635c3e2316c in_seq=0 out_seq=0 2024-04-25T19:05:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.265+0000 7f3eb6a00640 10 client.25582 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3eb6a00640 1 -- 172.21.15.86:0/3050221688 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ddab188b0 con 0x561ddabb9b70 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3eb6a00640 10 client.25582 handle_command_reply: tid=0 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 client.25582 shutdown 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 msgr2=0x561ddaba7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddaba7090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc004b90 tx=0x7f3ebc004bc0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 shutdown_connections 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddabb9b70 0x561ddabbbf60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddaba75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddaba7090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 --2- 172.21.15.86:0/3050221688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddaba6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3eb6a00640 0 client.25582 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 >> 172.21.15.86:0/3050221688 conn(0x561ddab9aec0 msgr2=0x561ddaba0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 shutdown_connections 2024-04-25T19:05:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 1 -- 172.21.15.86:0/3050221688 wait complete. 2024-04-25T19:05:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.266+0000 7f3edabfd740 20 client.25582 trim_cache size 0 max 16384 2024-04-25T19:05:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.282+0000 7f3edabfd740 1 Processor -- start 2024-04-25T19:05:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 -- start start 2024-04-25T19:05:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddaba1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddabc1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01afbe0 con 0x561ddab9e9b0 2024-04-25T19:05:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01b1720 con 0x561ddab9f2d0 2024-04-25T19:05:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3edabfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01b02b0 con 0x561ddaba9b90 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3ed5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddaba1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3ed5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddaba1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54626/0 (socket says 172.21.15.86:54626) 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3ed5400640 1 -- 172.21.15.86:0/2277840533 learned_addr learned my addr 172.21.15.86:0/2277840533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.283+0000 7f3ed5e00640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddabc1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5400640 1 -- 172.21.15.86:0/2277840533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 msgr2=0x561ddabc1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5400640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddabc1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5400640 1 -- 172.21.15.86:0/2277840533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 msgr2=0x561ddaba73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5400640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5400640 1 -- 172.21.15.86:0/2277840533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ddab96820 con 0x561ddab9e9b0 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5e00640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddabc1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ed5400640 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddaba1370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x561ddabbc6e0 tx=0x7f3ec401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59919e67682b786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ec1c00640 1 -- 172.21.15.86:0/2277840533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400fd80 con 0x561ddab9e9b0 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ec1c00640 1 -- 172.21.15.86:0/2277840533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec402b420 con 0x561ddab9e9b0 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.284+0000 7f3ec1c00640 1 -- 172.21.15.86:0/2277840533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4005200 con 0x561ddab9e9b0 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 -- 172.21.15.86:0/2277840533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 msgr2=0x561ddaba1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddaba1370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x561ddabbc6e0 tx=0x7f3ec401a000 comp rx=0 tx=0).stop 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 -- 172.21.15.86:0/2277840533 shutdown_connections 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddaba9b90 0x561ddabc1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9f2d0 0x561ddaba73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 --2- 172.21.15.86:0/2277840533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddab9e9b0 0x561ddaba1370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 -- 172.21.15.86:0/2277840533 >> 172.21.15.86:0/2277840533 conn(0x561ddab9aec0 msgr2=0x561ddab99790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 -- 172.21.15.86:0/2277840533 shutdown_connections 2024-04-25T19:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 -- 172.21.15.86:0/2277840533 wait complete. 2024-04-25T19:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.285+0000 7f3edabfd740 1 -- start start 2024-04-25T19:05:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 Processor -- start 2024-04-25T19:05:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddabaedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddabaf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddac20790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01b1050 con 0x561ddaba9b90 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01b0980 con 0x561ddab9e9b0 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01adbc0 con 0x561ddab9f2d0 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.286+0000 7f3edabfd740 10 client.0 _ll_register_callbacks cb 0x7f3ebc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddac20790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddabaedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddac20790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54638/0 (socket says 172.21.15.86:54638) 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddabaedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38206/0 (socket says 172.21.15.86:38206) 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 -- 172.21.15.86:0/575303796 learned_addr learned my addr 172.21.15.86:0/575303796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddabaf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 -- 172.21.15.86:0/575303796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 msgr2=0x561ddabaf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddabaf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 -- 172.21.15.86:0/575303796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 msgr2=0x561ddac20790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddac20790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 -- 172.21.15.86:0/575303796 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ddac20dd0 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3ed5400640 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddabaedd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400ff60 tx=0x7f3ec400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04ebcdc2169dd38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.287+0000 7f3eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3eb6a00640 1 -- 172.21.15.86:0/575303796 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec402b840 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3eb6a00640 1 -- 172.21.15.86:0/575303796 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f3ec4034470 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec402b420 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3eb6a00640 1 -- 172.21.15.86:0/575303796 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ec4042050 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3edabfd740 10 client.25595 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ddac280c0 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3eb6a00640 1 -- 172.21.15.86:0/575303796 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec402b420 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.288+0000 7f3eb6a00640 1 -- 172.21.15.86:0/575303796 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec401f830 con 0x561ddab9e9b0 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3edabfd740 10 client.25595 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3edabfd740 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3edabfd740 20 client.25595 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3edabfd740 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddac287d0 0x561ddac2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3edabfd740 4 client.25595 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ddab12320 con 0x561ddac287d0 2024-04-25T19:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddac287d0 0x561ddac2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3ed4a00640 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddac287d0 0x561ddac2abf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbe5dbd36ff83e7 server_cookie=9035348e69487c28 in_seq=0 out_seq=0 2024-04-25T19:05:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.289+0000 7f3eb6a00640 10 client.25595 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3eb6a00640 1 -- 172.21.15.86:0/575303796 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ddab12320 con 0x561ddac287d0 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3eb6a00640 10 client.25595 handle_command_reply: tid=0 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 client.25595 shutdown 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 msgr2=0x561ddabaedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddabaedd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400ff60 tx=0x7f3ec400ff90 comp rx=0 tx=0).stop 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 shutdown_connections 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561ddac287d0 0x561ddac2abf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ddaba9b90 0x561ddac20790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ddab9f2d0 0x561ddabaf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 --2- 172.21.15.86:0/575303796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ddab9e9b0 0x561ddabaedd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3eb6a00640 0 client.25595 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 >> 172.21.15.86:0/575303796 conn(0x561ddab9aec0 msgr2=0x561ddab9a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 shutdown_connections 2024-04-25T19:05:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 1 -- 172.21.15.86:0/575303796 wait complete. 2024-04-25T19:05:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.301+0000 7f3edabfd740 20 client.25595 trim_cache size 0 max 16384 2024-04-25T19:05:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:05:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:05:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:05:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3eb80a3d90 msgr2=0x7f3eb80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3eb80a3d90 0x7f3eb80a6250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009620 tx=0x7f3ec4005f90 comp rx=0 tx=0).stop 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed019a5e0 msgr2=0x7f3ed019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed019a5e0 0x7f3ed019aa90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009fd0 tx=0x7f3ec800ef10 comp rx=0 tx=0).stop 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 shutdown_connections 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3eb80a3d90 0x7f3eb80a6250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed019a5e0 0x7f3ed019aa90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ed01092d0 0x7f3ed01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 --2- 172.21.15.86:0/1186536629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0104200 0x7f3ed01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 >> 172.21.15.86:0/1186536629 conn(0x7f3ed00ffa60 msgr2=0x7f3ed0100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 shutdown_connections 2024-04-25T19:05:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:25.302+0000 7f3ed6800640 1 -- 172.21.15.86:0/1186536629 wait complete. 2024-04-25T19:05:26.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:26 smithi194 ceph-mon[32740]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T19:05:26.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:26 smithi086 ceph-mon[27783]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T19:05:26.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:26 smithi183 ceph-mon[32344]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T19:05:28.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:28 smithi086 ceph-mon[27783]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T19:05:28.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:28 smithi183 ceph-mon[32344]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T19:05:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:28 smithi194 ceph-mon[32740]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T19:05:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:30 smithi086 ceph-mon[27783]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T19:05:30.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:30 smithi183 ceph-mon[32344]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T19:05:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:30 smithi194 ceph-mon[32740]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T19:05:32.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:32 smithi086 ceph-mon[27783]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T19:05:32.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:32 smithi183 ceph-mon[32344]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T19:05:32.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:32 smithi194 ceph-mon[32740]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T19:05:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:34 smithi086 ceph-mon[27783]: pgmap v81: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T19:05:34.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:34 smithi183 ceph-mon[32344]: pgmap v81: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T19:05:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:34 smithi194 ceph-mon[32740]: pgmap v81: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T19:05:36.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:36 smithi086 ceph-mon[27783]: pgmap v82: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-04-25T19:05:36.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:36 smithi183 ceph-mon[32344]: pgmap v82: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-04-25T19:05:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:36 smithi194 ceph-mon[32740]: pgmap v82: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-04-25T19:05:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 -- 172.21.15.86:0/3781109782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e010e420 msgr2=0x7f37e010e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 --2- 172.21.15.86:0/3781109782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e010e420 0x7f37e010e800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009900 tx=0x7f37d002a410 comp rx=0 tx=0).stop 2024-04-25T19:05:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 -- 172.21.15.86:0/3781109782 shutdown_connections 2024-04-25T19:05:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 --2- 172.21.15.86:0/3781109782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37e0108e10 0x7f37e0115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 --2- 172.21.15.86:0/3781109782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e0108490 0x7f37e01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 --2- 172.21.15.86:0/3781109782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e010e420 0x7f37e010e800 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 -- 172.21.15.86:0/3781109782 >> 172.21.15.86:0/3781109782 conn(0x7f37e0077d30 msgr2=0x7f37e0106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 -- 172.21.15.86:0/3781109782 shutdown_connections 2024-04-25T19:05:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.447+0000 7f37e4a00640 1 -- 172.21.15.86:0/3781109782 wait complete. 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 Processor -- start 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 -- start start 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e0108490 0x7f37e01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37e0108e10 0x7f37e01a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 0x7f37e019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0118600 con 0x7f37e010e420 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0118480 con 0x7f37e0108490 2024-04-25T19:05:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37e4a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0118780 con 0x7f37e0108e10 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.449+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 0x7f37e019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 0x7f37e019e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57646/0 (socket says 172.21.15.86:57646) 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 -- 172.21.15.86:0/1752531085 learned_addr learned my addr 172.21.15.86:0/1752531085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37df400640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e0108490 0x7f37e01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37d7e00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37e0108e10 0x7f37e01a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 -- 172.21.15.86:0/1752531085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37e0108e10 msgr2=0x7f37e01a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37e0108e10 0x7f37e01a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 -- 172.21.15.86:0/1752531085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e0108490 msgr2=0x7f37e01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e0108490 0x7f37e01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 -- 172.21.15.86:0/1752531085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e019f100 con 0x7f37e010e420 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dfe00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 0x7f37e019e980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37d800eb10 tx=0x7f37d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.450+0000 7f37dcc00640 1 -- 172.21.15.86:0/1752531085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004b20 con 0x7f37e010e420 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.451+0000 7f37dcc00640 1 -- 172.21.15.86:0/1752531085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d8004cc0 con 0x7f37e010e420 2024-04-25T19:05:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.451+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e019f3f0 con 0x7f37e010e420 2024-04-25T19:05:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.451+0000 7f37dcc00640 1 -- 172.21.15.86:0/1752531085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8010690 con 0x7f37e010e420 2024-04-25T19:05:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.451+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01ab690 con 0x7f37e010e420 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.454+0000 7f37e8f76740 1 Processor -- start 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.454+0000 7f37dcc00640 1 -- 172.21.15.86:0/1752531085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37d8005020 con 0x7f37e010e420 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.454+0000 7f37e8f76740 1 -- start start 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.454+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c87d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.454+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c92650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c94f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37e8f76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b07bed900 con 0x559b07c92b90 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37e8f76740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b07b93ef0 con 0x559b07c882d0 2024-04-25T19:05:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37e8f76740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559b07be98c0 con 0x559b07c879b0 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c94f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37df400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c92650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c94f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57652/0 (socket says 172.21.15.86:57652) 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37d7e00640 1 -- 172.21.15.86:0/1825942143 learned_addr learned my addr 172.21.15.86:0/1825942143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37d7e00640 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c87d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37dcc00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f37c40a3d10 0x7f37c40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37df400640 1 -- 172.21.15.86:0/1825942143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 msgr2=0x559b07c87d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37df400640 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c87d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37df400640 1 -- 172.21.15.86:0/1825942143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 msgr2=0x559b07c94f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37df400640 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c94f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37df400640 1 -- 172.21.15.86:0/1825942143 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b07900210 con 0x559b07c882d0 2024-04-25T19:05:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.455+0000 7f37d7e00640 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c87d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.456+0000 7f37dfe00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f37c40a3d10 0x7f37c40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.456+0000 7f37dcc00640 1 -- 172.21.15.86:0/1752531085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f37d80d0d90 con 0x7f37e010e420 2024-04-25T19:05:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.456+0000 7f37df400640 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c92650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f37d0005940 tx=0x7f37d0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9bbda61711750a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.456+0000 7f37cfe00640 1 -- 172.21.15.86:0/1825942143 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0040070 con 0x559b07c882d0 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.456+0000 7f37cfe00640 1 -- 172.21.15.86:0/1825942143 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d0005020 con 0x559b07c882d0 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.456+0000 7f37cfe00640 1 -- 172.21.15.86:0/1825942143 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0035040 con 0x559b07c882d0 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 -- 172.21.15.86:0/1825942143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 msgr2=0x559b07c92650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c92650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f37d0005940 tx=0x7f37d0005bd0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37dfe00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f37c40a3d10 0x7f37c40a61d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009480 tx=0x7f37d800cf70 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 -- 172.21.15.86:0/1825942143 shutdown_connections 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c94f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c92650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1825942143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c87d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 -- 172.21.15.86:0/1825942143 >> 172.21.15.86:0/1825942143 conn(0x559b07c83ec0 msgr2=0x559b07c842a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 -- 172.21.15.86:0/1825942143 shutdown_connections 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.457+0000 7f37e8f76740 1 -- 172.21.15.86:0/1825942143 wait complete. 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 -- start start 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 Processor -- start 2024-04-25T19:05:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 0x559b07c8fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c882d0 0x559b07c90060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07c905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b07c95760 con 0x559b07c879b0 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b07ba7310 con 0x559b07c92b90 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559b07c958e0 con 0x559b07c882d0 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.458+0000 7f37e8f76740 10 client.0 _ll_register_callbacks cb 0x559b07b75cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 0x559b07c8fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 0x559b07c8fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57662/0 (socket says 172.21.15.86:57662) 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 -- 172.21.15.86:0/4207663262 learned_addr learned my addr 172.21.15.86:0/4207663262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37df400640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c882d0 0x559b07c90060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07c905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 -- 172.21.15.86:0/4207663262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c882d0 msgr2=0x559b07c90060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c882d0 0x559b07c90060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 -- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 msgr2=0x559b07c905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07c905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 -- 172.21.15.86:0/4207663262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b079cc620 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37d7e00640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 0x559b07c8fb20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009fa0 tx=0x7f37c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48efaba33a1d5787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.459+0000 7f37ccc00640 1 -- 172.21.15.86:0/4207663262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8004030 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37ccc00640 1 -- 172.21.15.86:0/4207663262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f37c8009040 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d0005020 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37ccc00640 1 -- 172.21.15.86:0/4207663262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37c80041d0 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37e8f76740 10 client.15498 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b07ca1a60 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37ccc00640 1 -- 172.21.15.86:0/4207663262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d0005020 con 0x559b07c879b0 2024-04-25T19:05:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37ccc00640 1 -- 172.21.15.86:0/4207663262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f37c8020420 con 0x559b07c879b0 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37e8f76740 10 client.15498 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37e8f76740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.460+0000 7f37e8f76740 20 client.15498 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.461+0000 7f37e8f76740 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07ca2ed0 0x559b07ca52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.461+0000 7f37e8f76740 4 client.15498 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.461+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b07c018b0 con 0x559b07ca2ed0 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.461+0000 7f37df400640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07ca2ed0 0x559b07ca52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.461+0000 7f37df400640 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07ca2ed0 0x559b07ca52c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadc8034195b5daa server_cookie=e85f5727752c2869 in_seq=0 out_seq=0 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.461+0000 7f37ccc00640 10 client.15498 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.462+0000 7f37ccc00640 1 -- 172.21.15.86:0/4207663262 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b07c018b0 con 0x559b07ca2ed0 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.462+0000 7f37ccc00640 10 client.15498 handle_command_reply: tid=0 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.462+0000 7f37e8f76740 1 client.15498 shutdown 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.462+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 msgr2=0x559b07c8fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.462+0000 7f37e8f76740 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 0x559b07c8fb20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009fa0 tx=0x7f37c8004730 comp rx=0 tx=0).stop 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 shutdown_connections 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07ca2ed0 0x559b07ca52c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07c905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c882d0 0x559b07c90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 --2- 172.21.15.86:0/4207663262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c879b0 0x559b07c8fb20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37ccc00640 0 client.15498 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 >> 172.21.15.86:0/4207663262 conn(0x559b07c83ec0 msgr2=0x559b07c89bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 shutdown_connections 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 1 -- 172.21.15.86:0/4207663262 wait complete. 2024-04-25T19:05:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.463+0000 7f37e8f76740 20 client.15498 trim_cache size 0 max 16384 2024-04-25T19:05:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 Processor -- start 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 -- start start 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 0x559b07caabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07cab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01b4180 con 0x559b07c882d0 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01b5cc0 con 0x559b07c92b90 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.479+0000 7f37e8f76740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01b4850 con 0x559b07c879b0 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 0x559b07caabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 0x559b07caabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57666/0 (socket says 172.21.15.86:57666) 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 -- 172.21.15.86:0/1084261370 learned_addr learned my addr 172.21.15.86:0/1084261370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37d7e00640 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37d7e00640 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07cab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 -- 172.21.15.86:0/1084261370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 msgr2=0x559b07c9e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 -- 172.21.15.86:0/1084261370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 msgr2=0x559b07cab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07cab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 -- 172.21.15.86:0/1084261370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b07ca5630 con 0x559b07c882d0 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37d7e00640 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07cab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:05:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.480+0000 7f37df400640 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 0x559b07caabd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37d0005940 tx=0x7f37d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cba1862c8467735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37cf400640 1 -- 172.21.15.86:0/1084261370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0035040 con 0x559b07c882d0 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37cf400640 1 -- 172.21.15.86:0/1084261370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d0004240 con 0x559b07c882d0 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37cf400640 1 -- 172.21.15.86:0/1084261370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0040040 con 0x559b07c882d0 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 -- 172.21.15.86:0/1084261370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 msgr2=0x559b07caabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 0x559b07caabd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37d0005940 tx=0x7f37d002a410 comp rx=0 tx=0).stop 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 -- 172.21.15.86:0/1084261370 shutdown_connections 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c92b90 0x559b07cab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c882d0 0x559b07caabd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 --2- 172.21.15.86:0/1084261370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.481+0000 7f37e8f76740 1 -- 172.21.15.86:0/1084261370 >> 172.21.15.86:0/1084261370 conn(0x559b07c83ec0 msgr2=0x559b07c8ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.482+0000 7f37e8f76740 1 -- 172.21.15.86:0/1084261370 shutdown_connections 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.482+0000 7f37e8f76740 1 -- 172.21.15.86:0/1084261370 wait complete. 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.482+0000 7f37e8f76740 1 -- start start 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.482+0000 7f37e8f76740 1 Processor -- start 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c98060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01b55f0 con 0x559b07c92b90 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01b4f20 con 0x559b07c882d0 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01b2160 con 0x559b07c879b0 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37e8f76740 10 client.0 _ll_register_callbacks cb 0x7f37c8011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37df400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c98060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37df400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c98060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46144/0 (socket says 172.21.15.86:46144) 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37df400640 1 -- 172.21.15.86:0/3292609729 learned_addr learned my addr 172.21.15.86:0/3292609729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.483+0000 7f37d7e00640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37df400640 1 -- 172.21.15.86:0/3292609729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 msgr2=0x559b07c9aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37df400640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37df400640 1 -- 172.21.15.86:0/3292609729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 msgr2=0x559b07c985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37df400640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37df400640 1 -- 172.21.15.86:0/3292609729 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b07c98ae0 con 0x559b07c882d0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37d7e00640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37df400640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c98060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37d002a930 tx=0x7f37d002ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2d6947f600a76b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37ccc00640 1 -- 172.21.15.86:0/3292609729 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d005f060 con 0x559b07c882d0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d0004240 con 0x559b07c882d0 2024-04-25T19:05:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.484+0000 7f37ccc00640 1 -- 172.21.15.86:0/3292609729 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f37d0005240 con 0x559b07c882d0 2024-04-25T19:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37ccc00640 1 -- 172.21.15.86:0/3292609729 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37d0035070 con 0x559b07c882d0 2024-04-25T19:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 10 client.25610 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b07d090c0 con 0x559b07c882d0 2024-04-25T19:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37ccc00640 1 -- 172.21.15.86:0/3292609729 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d0004240 con 0x559b07c882d0 2024-04-25T19:05:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37ccc00640 1 -- 172.21.15.86:0/3292609729 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f37d004d040 con 0x559b07c882d0 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 10 client.25610 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 20 client.25610 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07d09400 0x559b07d0b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 4 client.25610 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b07bfb320 con 0x559b07d09400 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.485+0000 7f37d7e00640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07d09400 0x559b07d0b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.486+0000 7f37d7e00640 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07d09400 0x559b07d0b7f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53ef053192fbeb0 server_cookie=e82314998db2411f in_seq=0 out_seq=0 2024-04-25T19:05:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.486+0000 7f37ccc00640 10 client.25610 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37ccc00640 1 -- 172.21.15.86:0/3292609729 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b07bfb320 con 0x559b07d09400 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37ccc00640 10 client.25610 handle_command_reply: tid=0 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 client.25610 shutdown 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 msgr2=0x559b07c98060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c98060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37d002a930 tx=0x7f37d002ab30 comp rx=0 tx=0).stop 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 shutdown_connections 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b07d09400 0x559b07d0b7f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b07c92b90 0x559b07c985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b07c882d0 0x559b07c98060 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 --2- 172.21.15.86:0/3292609729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b07c879b0 0x559b07c9aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 >> 172.21.15.86:0/3292609729 conn(0x559b07c83ec0 msgr2=0x559b07c8fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37ccc00640 0 client.25610 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 shutdown_connections 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 1 -- 172.21.15.86:0/3292609729 wait complete. 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.496+0000 7f37e8f76740 20 client.25610 trim_cache size 0 max 16384 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:05:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f37c40a3d10 msgr2=0x7f37c40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f37c40a3d10 0x7f37c40a61d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009480 tx=0x7f37d800cf70 comp rx=0 tx=0).stop 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 msgr2=0x7f37e019e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 0x7f37e019e980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37d800eb10 tx=0x7f37d800c2d0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 shutdown_connections 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f37c40a3d10 0x7f37c40a61d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37e010e420 0x7f37e019e980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37e0108e10 0x7f37e01a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 --2- 172.21.15.86:0/1752531085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37e0108490 0x7f37e01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.497+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 >> 172.21.15.86:0/1752531085 conn(0x7f37e0077d30 msgr2=0x7f37e0078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.498+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 shutdown_connections 2024-04-25T19:05:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:38.498+0000 7f37e4a00640 1 -- 172.21.15.86:0/1752531085 wait complete. 2024-04-25T19:05:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:38 smithi183 ceph-mon[32344]: pgmap v83: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-04-25T19:05:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:38 smithi194 ceph-mon[32740]: pgmap v83: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-04-25T19:05:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:38 smithi086 ceph-mon[27783]: pgmap v83: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-04-25T19:05:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:40 smithi183 ceph-mon[32344]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-04-25T19:05:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:40 smithi194 ceph-mon[32740]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-04-25T19:05:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:40 smithi086 ceph-mon[27783]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-04-25T19:05:40.936 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:05:41.395 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:05:42.591 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:42 smithi086 ceph-mon[27783]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:05:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:42 smithi183 ceph-mon[32344]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:05:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:42 smithi194 ceph-mon[32740]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 -- 172.21.15.86:0/407924817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec074c70 msgr2=0x7efdec075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 --2- 172.21.15.86:0/407924817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec074c70 0x7efdec075050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efde0009990 tx=0x7efde0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 -- 172.21.15.86:0/407924817 shutdown_connections 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 --2- 172.21.15.86:0/407924817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec113c10 0x7efdec116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 --2- 172.21.15.86:0/407924817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec074c70 0x7efdec075050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 --2- 172.21.15.86:0/407924817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec075590 0x7efdec1136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 -- 172.21.15.86:0/407924817 >> 172.21.15.86:0/407924817 conn(0x7efdec079ba0 msgr2=0x7efdec079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:43.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 -- 172.21.15.86:0/407924817 shutdown_connections 2024-04-25T19:05:43.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.668+0000 7efdf4800700 1 -- 172.21.15.86:0/407924817 wait complete. 2024-04-25T19:05:43.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.669+0000 7efdf4800700 1 Processor -- start 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.669+0000 7efdf4800700 1 -- start start 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.669+0000 7efdf4800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec074c70 0x7efdec1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.669+0000 7efdf4800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec075590 0x7efdec1a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf4800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 0x7efdec19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf4800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdec119260 con 0x7efdec113c10 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf4800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1190e0 con 0x7efdec074c70 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf4800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1193e0 con 0x7efdec075590 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 0x7efdec19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 0x7efdec19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57508/0 (socket says 172.21.15.86:57508) 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 -- 172.21.15.86:0/434136289 learned_addr learned my addr 172.21.15.86:0/434136289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf1400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec075590 0x7efdec1a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf1e00700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec074c70 0x7efdec1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 -- 172.21.15.86:0/434136289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec075590 msgr2=0x7efdec1a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec075590 0x7efdec1a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 -- 172.21.15.86:0/434136289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec074c70 msgr2=0x7efdec1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec074c70 0x7efdec1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.670+0000 7efdf2800700 1 -- 172.21.15.86:0/434136289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec19ff20 con 0x7efdec113c10 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.671+0000 7efdf2800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 0x7efdec19f640 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efde80094f0 tx=0x7efde800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.671+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8004db0 con 0x7efdec113c10 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.671+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8009e20 con 0x7efdec113c10 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.671+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8015400 con 0x7efdec113c10 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.671+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec1a0210 con 0x7efdec113c10 2024-04-25T19:05:43.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.671+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec1ac4f0 con 0x7efdec113c10 2024-04-25T19:05:43.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.673+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec076360 con 0x7efdec113c10 2024-04-25T19:05:43.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.675+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efde8005260 con 0x7efdec113c10 2024-04-25T19:05:43.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.677+0000 7efde6a00700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7efdd40a3db0 0x7efdd40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:43.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.677+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7efde80d01c0 con 0x7efdec113c10 2024-04-25T19:05:43.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.677+0000 7efdf1e00700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7efdd40a3db0 0x7efdd40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:43.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.678+0000 7efdf1e00700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7efdd40a3db0 0x7efdd40a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efde0006010 tx=0x7efde000b260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:43.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.682+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efde8010070 con 0x7efdec113c10 2024-04-25T19:05:43.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.963+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdec1ac6c0 con 0x7efdec113c10 2024-04-25T19:05:43.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.964+0000 7efde6a00700 1 -- 172.21.15.86:0/434136289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7efde809c810 con 0x7efdec113c10 2024-04-25T19:05:43.966 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:05:43.966 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:05:43.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7efdd40a3db0 msgr2=0x7efdd40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:43.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7efdd40a3db0 0x7efdd40a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efde0006010 tx=0x7efde000b260 comp rx=0 tx=0).stop 2024-04-25T19:05:43.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 msgr2=0x7efdec19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:43.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 0x7efdec19f640 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efde80094f0 tx=0x7efde800d3b0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 shutdown_connections 2024-04-25T19:05:43.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdec113c10 0x7efdec19f640 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec074c70 0x7efdec1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7efdd40a3db0 0x7efdd40a6270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 --2- 172.21.15.86:0/434136289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec075590 0x7efdec1a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:43.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.967+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 >> 172.21.15.86:0/434136289 conn(0x7efdec079ba0 msgr2=0x7efdec112a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:43.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.968+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 shutdown_connections 2024-04-25T19:05:43.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:43.968+0000 7efdf4800700 1 -- 172.21.15.86:0/434136289 wait complete. 2024-04-25T19:05:43.971 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:05:44.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:44 smithi183 ceph-mon[32344]: pgmap v86: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-25T19:05:44.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:44 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/434136289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:44 smithi194 ceph-mon[32740]: pgmap v86: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-25T19:05:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:44 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/434136289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:44.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:44 smithi086 ceph-mon[27783]: pgmap v86: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-04-25T19:05:44.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:44 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/434136289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:44.839 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 57c7c7b5 --release --await' 2024-04-25T19:05:44.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 57c7c7b5 --release --await 2024-04-25T19:05:45.285 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:05:46.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:46 smithi183 ceph-mon[32344]: pgmap v87: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T19:05:46.704 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:46 smithi086 ceph-mon[27783]: pgmap v87: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T19:05:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:46 smithi194 ceph-mon[32740]: pgmap v87: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T19:05:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 -- 172.21.15.86:0/802577999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6438108a30 msgr2=0x7f6438108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 --2- 172.21.15.86:0/802577999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6438108a30 0x7f6438108e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64280098f0 tx=0x7f6428009c00 comp rx=0 tx=0).stop 2024-04-25T19:05:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 -- 172.21.15.86:0/802577999 shutdown_connections 2024-04-25T19:05:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 --2- 172.21.15.86:0/802577999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643806b0f0 0x7f643810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 --2- 172.21.15.86:0/802577999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6438108a30 0x7f6438108e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 --2- 172.21.15.86:0/802577999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 -- 172.21.15.86:0/802577999 >> 172.21.15.86:0/802577999 conn(0x7f64380788f0 msgr2=0x7f6438078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.580+0000 7f643de00700 1 -- 172.21.15.86:0/802577999 shutdown_connections 2024-04-25T19:05:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.581+0000 7f643de00700 1 -- 172.21.15.86:0/802577999 wait complete. 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.582+0000 7f643de00700 1 Processor -- start 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.582+0000 7f643de00700 1 -- start start 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f643de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f643de00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f643806b0f0 0x7f643819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f643de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6438108a30 0x7f643819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f643de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6438112c00 con 0x7f6438108a30 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f643de00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6438112a80 con 0x7f643806a750 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f643de00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6438112d80 con 0x7f643806b0f0 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f6436a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f6436000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f643806b0f0 0x7f643819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f6436a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:34964/0 (socket says 172.21.15.86:34964) 2024-04-25T19:05:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f6436a00700 1 -- 172.21.15.86:0/2168592152 learned_addr learned my addr 172.21.15.86:0/2168592152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.583+0000 7f6437400700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6438108a30 0x7f643819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.584+0000 7f6436000700 1 -- 172.21.15.86:0/2168592152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 msgr2=0x7f643819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.584+0000 7f6436000700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.584+0000 7f6436000700 1 -- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6438108a30 msgr2=0x7f643819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.584+0000 7f6436000700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6438108a30 0x7f643819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.584+0000 7f6436000700 1 -- 172.21.15.86:0/2168592152 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381a03e0 con 0x7f643806b0f0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.584+0000 7f6436a00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.585+0000 7f6436000700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f643806b0f0 0x7f643819b910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9f0 tx=0x7f642400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.585+0000 7f643ca00700 1 -- 172.21.15.86:0/2168592152 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642400cb70 con 0x7f643806b0f0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.585+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64381a06d0 con 0x7f643806b0f0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.585+0000 7f643ca00700 1 -- 172.21.15.86:0/2168592152 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642400ee90 con 0x7f643806b0f0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.585+0000 7f643ca00700 1 -- 172.21.15.86:0/2168592152 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424003d40 con 0x7f643806b0f0 2024-04-25T19:05:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.585+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381a7f50 con 0x7f643806b0f0 2024-04-25T19:05:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.589+0000 7f643ca00700 1 -- 172.21.15.86:0/2168592152 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6424003ee0 con 0x7f643806b0f0 2024-04-25T19:05:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.590+0000 7f6448815b80 1 Processor -- start 2024-04-25T19:05:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.590+0000 7f6448815b80 1 -- start start 2024-04-25T19:05:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6448815b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6448815b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75bbdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6448815b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75bc08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6448815b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c75b05f90 con 0x557c75bb2d40 2024-04-25T19:05:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6448815b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c75aac830 con 0x557c75bb3660 2024-04-25T19:05:47.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6448815b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557c75b01f50 con 0x557c75bbe4d0 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6437400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6436a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75bbdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6437400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bb3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57530/0 (socket says 172.21.15.86:57530) 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6437400700 1 -- 172.21.15.86:0/2179801664 learned_addr learned my addr 172.21.15.86:0/2179801664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f6436000700 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75bc08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.591+0000 7f643ca00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f641c0a3db0 0x7f641c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6436a00700 1 -- 172.21.15.86:0/2179801664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 msgr2=0x557c75bc08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f643ca00700 1 -- 172.21.15.86:0/2168592152 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f64240d09b0 con 0x7f643806b0f0 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6437400700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f641c0a3db0 0x7f641c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6436a00700 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75bc08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6436a00700 1 -- 172.21.15.86:0/2179801664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 msgr2=0x557c75bb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6436a00700 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6436a00700 1 -- 172.21.15.86:0/2179801664 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c75aa2780 con 0x557c75bb3660 2024-04-25T19:05:47.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6436000700 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75bc08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:47.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.592+0000 7f6437400700 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.593+0000 7f6436a00700 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75bbdf90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6428009810 tx=0x7f64280095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4de073565e13cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.593+0000 7f6437400700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f641c0a3db0 0x7f641c0a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6430007b30 tx=0x7f6430004b40 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.593+0000 7f642cc00700 1 -- 172.21.15.86:0/2179801664 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642800f6e0 con 0x557c75bb3660 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.593+0000 7f642cc00700 1 -- 172.21.15.86:0/2179801664 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642800f880 con 0x557c75bb3660 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.593+0000 7f642cc00700 1 -- 172.21.15.86:0/2179801664 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642800bb30 con 0x557c75bb3660 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 -- 172.21.15.86:0/2179801664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 msgr2=0x557c75bbdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75bbdf90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6428009810 tx=0x7f64280095f0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 -- 172.21.15.86:0/2179801664 shutdown_connections 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75bc08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 --2- 172.21.15.86:0/2179801664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75bbdf90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 -- 172.21.15.86:0/2179801664 >> 172.21.15.86:0/2179801664 conn(0x557c75ae2050 msgr2=0x557c75baf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 -- 172.21.15.86:0/2179801664 shutdown_connections 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.594+0000 7f6448815b80 1 -- 172.21.15.86:0/2179801664 wait complete. 2024-04-25T19:05:47.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.595+0000 7f6448815b80 1 -- start start 2024-04-25T19:05:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.595+0000 7f6448815b80 1 Processor -- start 2024-04-25T19:05:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bbdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bbaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c75bc0ea0 con 0x557c75bb2d40 2024-04-25T19:05:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c75abf960 con 0x557c75bbe4d0 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557c75bc1020 con 0x557c75bb3660 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6448815b80 10 client.0 _ll_register_callbacks cb 0x557c75a9e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bbdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bbdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57542/0 (socket says 172.21.15.86:57542) 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 -- 172.21.15.86:0/2821599666 learned_addr learned my addr 172.21.15.86:0/2821599666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6437400700 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 -- 172.21.15.86:0/2821599666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 msgr2=0x557c75bbaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bbaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 -- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 msgr2=0x557c75bbb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.596+0000 7f6436a00700 1 -- 172.21.15.86:0/2821599666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c75a539d0 con 0x557c75bb2d40 2024-04-25T19:05:47.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6436a00700 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bbdc00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f642800bac0 tx=0x7f642800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8fdd0a1fb40c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6421600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:05:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6421600700 1 -- 172.21.15.86:0/2821599666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428030030 con 0x557c75bb2d40 2024-04-25T19:05:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6421600700 1 -- 172.21.15.86:0/2821599666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f642800bdc0 con 0x557c75bb2d40 2024-04-25T19:05:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642800f880 con 0x557c75bb2d40 2024-04-25T19:05:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6421600700 1 -- 172.21.15.86:0/2821599666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f642800b560 con 0x557c75bb2d40 2024-04-25T19:05:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6448815b80 10 client.15522 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.597+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c75a9cbf0 con 0x557c75bb2d40 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6421600700 1 -- 172.21.15.86:0/2821599666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642800f880 con 0x557c75bb2d40 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6421600700 1 -- 172.21.15.86:0/2821599666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f64280374d0 con 0x557c75bb2d40 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6448815b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6448815b80 10 client.15522 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6448815b80 20 client.15522 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6448815b80 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75bcda60 0x557c75bcfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6448815b80 4 client.15522 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.598+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c75b13780 con 0x557c75bcda60 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.599+0000 7f6436000700 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75bcda60 0x557c75bcfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.599+0000 7f6436000700 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75bcda60 0x557c75bcfe50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cd6ebf2111801e server_cookie=24a7b9deec993eb6 in_seq=0 out_seq=0 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.599+0000 7f6421600700 10 client.15522 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6421600700 1 -- 172.21.15.86:0/2821599666 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c75b13780 con 0x557c75bcda60 2024-04-25T19:05:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6421600700 10 client.15522 handle_command_reply: tid=0 2024-04-25T19:05:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 client.15522 shutdown 2024-04-25T19:05:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 msgr2=0x557c75bbdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bbdc00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f642800bac0 tx=0x7f642800baf0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 shutdown_connections 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75bcda60 0x557c75bcfe50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bbdc00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 --2- 172.21.15.86:0/2821599666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bbaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 >> 172.21.15.86:0/2821599666 conn(0x557c75ae2050 msgr2=0x557c75baee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6421600700 0 client.15522 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.600+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 shutdown_connections 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.601+0000 7f6448815b80 1 -- 172.21.15.86:0/2821599666 wait complete. 2024-04-25T19:05:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.601+0000 7f6448815b80 20 client.15522 trim_cache size 0 max 16384 2024-04-25T19:05:47.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 Processor -- start 2024-04-25T19:05:47.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 -- start start 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bd2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bd2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64381af700 con 0x557c75bb2d40 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b1240 con 0x557c75bbe4d0 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.622+0000 7f6448815b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f64381afdd0 con 0x557c75bb3660 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bd2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bd2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57548/0 (socket says 172.21.15.86:57548) 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436a00700 1 -- 172.21.15.86:0/3053012221 learned_addr learned my addr 172.21.15.86:0/3053012221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436000700 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bd2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436000700 1 -- 172.21.15.86:0/3053012221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 msgr2=0x557c75bbce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436000700 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436000700 1 -- 172.21.15.86:0/3053012221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 msgr2=0x557c75bd2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436000700 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bd2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436000700 1 -- 172.21.15.86:0/3053012221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c75bb3ce0 con 0x557c75bb3660 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.623+0000 7f6436a00700 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bd2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.624+0000 7f6436000700 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bd2fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64240ce050 tx=0x7f64240d2e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22e388de8506b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.624+0000 7f6423e00700 1 -- 172.21.15.86:0/3053012221 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240a0c50 con 0x557c75bb3660 2024-04-25T19:05:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.624+0000 7f6423e00700 1 -- 172.21.15.86:0/3053012221 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64240d7040 con 0x557c75bb3660 2024-04-25T19:05:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.624+0000 7f6423e00700 1 -- 172.21.15.86:0/3053012221 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240ce2d0 con 0x557c75bb3660 2024-04-25T19:05:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 -- 172.21.15.86:0/3053012221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 msgr2=0x557c75bd2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bd2fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64240ce050 tx=0x7f64240d2e00 comp rx=0 tx=0).stop 2024-04-25T19:05:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 -- 172.21.15.86:0/3053012221 shutdown_connections 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bbe4d0 0x557c75bbce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75bd2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 --2- 172.21.15.86:0/3053012221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bb3660 0x557c75bd2fe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 -- 172.21.15.86:0/3053012221 >> 172.21.15.86:0/3053012221 conn(0x557c75ae2050 msgr2=0x557c75bbc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 -- 172.21.15.86:0/3053012221 shutdown_connections 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.625+0000 7f6448815b80 1 -- 172.21.15.86:0/3053012221 wait complete. 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 -- start start 2024-04-25T19:05:47.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 Processor -- start 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75c2db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75c2e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75c27f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b0b70 con 0x557c75bb2d40 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b04a0 con 0x557c75bb3660 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f64381ad6a0 con 0x557c75bbe4d0 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.626+0000 7f6448815b80 10 client.0 _ll_register_callbacks cb 0x557c75a9e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75c2db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75c2db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57564/0 (socket says 172.21.15.86:57564) 2024-04-25T19:05:47.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 -- 172.21.15.86:0/3441171715 learned_addr learned my addr 172.21.15.86:0/3441171715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6437400700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75c27f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436000700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75c2e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 -- 172.21.15.86:0/3441171715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 msgr2=0x557c75c27f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75c27f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 -- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 msgr2=0x557c75c2e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75c2e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436a00700 1 -- 172.21.15.86:0/3441171715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c75c28690 con 0x557c75bb2d40 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.627+0000 7f6436000700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75c2e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6436a00700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75c2db20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6428011040 tx=0x7f6428037cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49308d058033e5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6421600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6421600700 1 -- 172.21.15.86:0/3441171715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428041030 con 0x557c75bb2d40 2024-04-25T19:05:47.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6421600700 1 -- 172.21.15.86:0/3441171715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f642800be20 con 0x557c75bb2d40 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64240d7040 con 0x557c75bb2d40 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6421600700 1 -- 172.21.15.86:0/3441171715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6428037e60 con 0x557c75bb2d40 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6448815b80 10 client.15528 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.628+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c75c35390 con 0x557c75bb2d40 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6421600700 1 -- 172.21.15.86:0/3441171715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64240d7040 con 0x557c75bb2d40 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6421600700 1 -- 172.21.15.86:0/3441171715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f64280036a0 con 0x557c75bb2d40 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6448815b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6448815b80 10 client.15528 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6448815b80 20 client.15528 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6448815b80 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75c359d0 0x557c75c37dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6448815b80 4 client.15528 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "57c7c7b5", "release": true, "await": true}] 2024-04-25T19:05:47.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.629+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "57c7c7b5", "release": true, "await": true}) -- 0x557c75b0d820 con 0x557c75c359d0 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.630+0000 7f6436000700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75c359d0 0x557c75c37dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.630+0000 7f6436000700 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75c359d0 0x557c75c37dc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c0be9f0b442794 server_cookie=8d859d22480a2e89 in_seq=0 out_seq=0 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.630+0000 7f6421600700 10 client.15528 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6421600700 1 -- 172.21.15.86:0/3441171715 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x557c75b0d820 con 0x557c75c359d0 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6421600700 10 client.15528 handle_command_reply: tid=0 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 client.15528 shutdown 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 msgr2=0x557c75c2db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75c2db20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6428011040 tx=0x7f6428037cf0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 shutdown_connections 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557c75c359d0 0x557c75c37dc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c75bbe4d0 0x557c75c27f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c75bb2d40 0x557c75c2db20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 --2- 172.21.15.86:0/3441171715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c75bb3660 0x557c75c2e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 >> 172.21.15.86:0/3441171715 conn(0x557c75ae2050 msgr2=0x557c75bbcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6421600700 0 client.15528 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:47.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 shutdown_connections 2024-04-25T19:05:47.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.633+0000 7f6448815b80 1 -- 172.21.15.86:0/3441171715 wait complete. 2024-04-25T19:05:47.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.634+0000 7f6448815b80 20 client.15528 trim_cache size 0 max 16384 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 8, 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "57c7c7b5": { 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 8, 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:05:47.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 90.0, 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:47.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:47.639 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:05:47.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f641c0a3db0 msgr2=0x7f641c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f641c0a3db0 0x7f641c0a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6430007b30 tx=0x7f6430004b40 comp rx=0 tx=0).stop 2024-04-25T19:05:47.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f643806b0f0 msgr2=0x7f643819b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:47.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f643806b0f0 0x7f643819b910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9f0 tx=0x7f642400ed00 comp rx=0 tx=0).stop 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 shutdown_connections 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6438108a30 0x7f643819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643806a750 0x7f643819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f641c0a3db0 0x7f641c0a6270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 --2- 172.21.15.86:0/2168592152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f643806b0f0 0x7f643819b910 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 >> 172.21.15.86:0/2168592152 conn(0x7f64380788f0 msgr2=0x7f643810df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 shutdown_connections 2024-04-25T19:05:47.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:47.635+0000 7f643de00700 1 -- 172.21.15.86:0/2168592152 wait complete. 2024-04-25T19:05:48.597 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '57c7c7b5', seconds elapsed: 45.5 2024-04-25T19:05:48.598 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-25T19:05:48.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:05:48.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:48 smithi086 ceph-mon[27783]: pgmap v88: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T19:05:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:48 smithi183 ceph-mon[32344]: pgmap v88: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T19:05:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:48 smithi194 ceph-mon[32740]: pgmap v88: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-25T19:05:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:05:50.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:50 smithi183 ceph-mon[32344]: pgmap v89: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-25T19:05:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:50 smithi194 ceph-mon[32740]: pgmap v89: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-25T19:05:50.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:50 smithi086 ceph-mon[27783]: pgmap v89: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.332+0000 7f0ada400700 1 -- 172.21.15.86:0/622267975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4105870 msgr2=0x7f0ad41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.332+0000 7f0ada400700 1 --2- 172.21.15.86:0/622267975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4105870 0x7f0ad41098c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00099c0 tx=0x7f0ac0009cd0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 -- 172.21.15.86:0/622267975 shutdown_connections 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 --2- 172.21.15.86:0/622267975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 0x7f0ad4111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 --2- 172.21.15.86:0/622267975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4104ec0 0x7f0ad41052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 --2- 172.21.15.86:0/622267975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4105870 0x7f0ad41098c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 -- 172.21.15.86:0/622267975 >> 172.21.15.86:0/622267975 conn(0x7f0ad4100750 msgr2=0x7f0ad4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 -- 172.21.15.86:0/622267975 shutdown_connections 2024-04-25T19:05:51.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.333+0000 7f0ada400700 1 -- 172.21.15.86:0/622267975 wait complete. 2024-04-25T19:05:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.335+0000 7f0ada400700 1 Processor -- start 2024-04-25T19:05:51.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.335+0000 7f0ada400700 1 -- start start 2024-04-25T19:05:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ada400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4104ec0 0x7f0ad419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ada400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 0x7f0ad419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ada400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 0x7f0ad419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ada400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad4114e80 con 0x7f0ad4105870 2024-04-25T19:05:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ada400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad4114d00 con 0x7f0ad4109fa0 2024-04-25T19:05:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ada400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad4115000 con 0x7f0ad4104ec0 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ad2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 0x7f0ad419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ad2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 0x7f0ad419b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54786/0 (socket says 172.21.15.86:54786) 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ad2a00700 1 -- 172.21.15.86:0/2117248292 learned_addr learned my addr 172.21.15.86:0/2117248292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.336+0000 7f0ad3e00700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 0x7f0ad419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad2a00700 1 -- 172.21.15.86:0/2117248292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4104ec0 msgr2=0x7f0ad419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad2a00700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4104ec0 0x7f0ad419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad2a00700 1 -- 172.21.15.86:0/2117248292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 msgr2=0x7f0ad419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad2a00700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 0x7f0ad419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad2a00700 1 -- 172.21.15.86:0/2117248292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad41a03b0 con 0x7f0ad4105870 2024-04-25T19:05:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad3e00700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 0x7f0ad419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ad2a00700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 0x7f0ad419b930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000b580 tx=0x7f0ac0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0009e60 con 0x7f0ad4105870 2024-04-25T19:05:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac0003c70 con 0x7f0ad4105870 2024-04-25T19:05:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad41a0640 con 0x7f0ad4105870 2024-04-25T19:05:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.337+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00049c0 con 0x7f0ad4105870 2024-04-25T19:05:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.338+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad41a7f20 con 0x7f0ad4105870 2024-04-25T19:05:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.339+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad41063a0 con 0x7f0ad4105870 2024-04-25T19:05:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.341+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ac00037c0 con 0x7f0ad4105870 2024-04-25T19:05:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.342+0000 7f0acbe00700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ab80a3db0 0x7f0ab80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.342+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f0ac00d11a0 con 0x7f0ad4105870 2024-04-25T19:05:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.343+0000 7f0ad3400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ab80a3db0 0x7f0ab80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.343+0000 7f0ad3400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ab80a3db0 0x7f0ab80a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4005950 tx=0x7f0ac40058e0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.347+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ac0013070 con 0x7f0ad4105870 2024-04-25T19:05:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.624+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ad419c700 con 0x7f0ad4105870 2024-04-25T19:05:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.626+0000 7f0acbe00700 1 -- 172.21.15.86:0/2117248292 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f0ac009d820 con 0x7f0ad4105870 2024-04-25T19:05:51.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:05:51.628 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:05:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.629+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ab80a3db0 msgr2=0x7f0ab80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.629+0000 7f0ada400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ab80a3db0 0x7f0ab80a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4005950 tx=0x7f0ac40058e0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.629+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 msgr2=0x7f0ad419b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.629+0000 7f0ada400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 0x7f0ad419b930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000b580 tx=0x7f0ac0004270 comp rx=0 tx=0).stop 2024-04-25T19:05:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.629+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 shutdown_connections 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.629+0000 7f0ada400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ad4109fa0 0x7f0ad419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.630+0000 7f0ada400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ab80a3db0 0x7f0ab80a6270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.630+0000 7f0ada400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ad4104ec0 0x7f0ad419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.630+0000 7f0ada400700 1 --2- 172.21.15.86:0/2117248292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ad4105870 0x7f0ad419b930 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.630+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 >> 172.21.15.86:0/2117248292 conn(0x7f0ad4100750 msgr2=0x7f0ad4102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.630+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 shutdown_connections 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:51.630+0000 7f0ada400700 1 -- 172.21.15.86:0/2117248292 wait complete. 2024-04-25T19:05:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 -- 172.21.15.86:0/1763087496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c101fd0 msgr2=0x7f7d8c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 --2- 172.21.15.86:0/1763087496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c101fd0 0x7f7d8c1023b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009900 tx=0x7f7d7c02a410 comp rx=0 tx=0).stop 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 -- 172.21.15.86:0/1763087496 shutdown_connections 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 --2- 172.21.15.86:0/1763087496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 --2- 172.21.15.86:0/1763087496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c102980 0x7f7d8c1069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 --2- 172.21.15.86:0/1763087496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c101fd0 0x7f7d8c1023b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 -- 172.21.15.86:0/1763087496 >> 172.21.15.86:0/1763087496 conn(0x7f7d8c0fd8d0 msgr2=0x7f7d8c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 -- 172.21.15.86:0/1763087496 shutdown_connections 2024-04-25T19:05:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.636+0000 7f7d91800640 1 -- 172.21.15.86:0/1763087496 wait complete. 2024-04-25T19:05:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.638+0000 7f7d91800640 1 Processor -- start 2024-04-25T19:05:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.638+0000 7f7d91800640 1 -- start start 2024-04-25T19:05:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.638+0000 7f7d91800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d8c101fd0 0x7f7d8c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.638+0000 7f7d91800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.638+0000 7f7d91800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c074cd0 0x7f7d8c1a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d91800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c111fe0 con 0x7f7d8c1070a0 2024-04-25T19:05:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d91800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c111e60 con 0x7f7d8c074cd0 2024-04-25T19:05:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d91800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c112160 con 0x7f7d8c101fd0 2024-04-25T19:05:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54810/0 (socket says 172.21.15.86:54810) 2024-04-25T19:05:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d90e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c074cd0 0x7f7d8c1a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 -- 172.21.15.86:0/2446058023 learned_addr learned my addr 172.21.15.86:0/2446058023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8be00640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d8c101fd0 0x7f7d8c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 -- 172.21.15.86:0/2446058023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d8c101fd0 msgr2=0x7f7d8c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d8c101fd0 0x7f7d8c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 -- 172.21.15.86:0/2446058023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c074cd0 msgr2=0x7f7d8c1a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c074cd0 0x7f7d8c1a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d8b400640 1 -- 172.21.15.86:0/2446058023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c1a6e70 con 0x7f7d8c1070a0 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.639+0000 7f7d90e00640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c074cd0 0x7f7d8c1a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.640+0000 7f7d8b400640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c074790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9a0 tx=0x7f7d7800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.640+0000 7f7d88c00640 1 -- 172.21.15.86:0/2446058023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800cd80 con 0x7f7d8c1070a0 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.640+0000 7f7d88c00640 1 -- 172.21.15.86:0/2446058023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d780035f0 con 0x7f7d8c1070a0 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.640+0000 7f7d88c00640 1 -- 172.21.15.86:0/2446058023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78010690 con 0x7f7d8c1070a0 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.640+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c1a7160 con 0x7f7d8c1070a0 2024-04-25T19:05:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.640+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c1a7700 con 0x7f7d8c1070a0 2024-04-25T19:05:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.643+0000 7f7d95be6740 1 Processor -- start 2024-04-25T19:05:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 -- start start 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d88c00640 1 -- 172.21.15.86:0/2446058023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d78003fd0 con 0x7f7d8c1070a0 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 0x5581d2d04d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581d2c6a900 con 0x5581d2d0fb90 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581d2c10ef0 con 0x5581d2d049b0 2024-04-25T19:05:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d95be6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581d2c668c0 con 0x5581d2d052d0 2024-04-25T19:05:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d90e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 0x5581d2d04d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d90e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 0x5581d2d04d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43488/0 (socket says 172.21.15.86:43488) 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d90e00640 1 -- 172.21.15.86:0/921804646 learned_addr learned my addr 172.21.15.86:0/921804646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.644+0000 7f7d8be00640 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d90e00640 1 -- 172.21.15.86:0/921804646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 msgr2=0x5581d2d0f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d90e00640 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d90e00640 1 -- 172.21.15.86:0/921804646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 msgr2=0x5581d2d11f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d90e00640 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d11f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d90e00640 1 -- 172.21.15.86:0/921804646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d297d210 con 0x5581d2d049b0 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d90e00640 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 0x5581d2d04d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d84010710 tx=0x7f7d84010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85bb33d473a50da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d81600640 1 -- 172.21.15.86:0/921804646 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400e8d0 con 0x5581d2d049b0 2024-04-25T19:05:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d81600640 1 -- 172.21.15.86:0/921804646 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d84012450 con 0x5581d2d049b0 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.645+0000 7f7d81600640 1 -- 172.21.15.86:0/921804646 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8401c630 con 0x5581d2d049b0 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d88c00640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7d700a3dc0 0x7f7d700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d88c00640 1 -- 172.21.15.86:0/2446058023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7d780d0a10 con 0x7f7d8c1070a0 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d8be00640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7d700a3dc0 0x7f7d700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 -- 172.21.15.86:0/921804646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 msgr2=0x5581d2d04d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 0x5581d2d04d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d84010710 tx=0x7f7d84010bd0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 -- 172.21.15.86:0/921804646 shutdown_connections 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 --2- 172.21.15.86:0/921804646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d049b0 0x5581d2d04d90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.646+0000 7f7d95be6740 1 -- 172.21.15.86:0/921804646 >> 172.21.15.86:0/921804646 conn(0x5581d2d00ec0 msgr2=0x5581d2d012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.647+0000 7f7d95be6740 1 -- 172.21.15.86:0/921804646 shutdown_connections 2024-04-25T19:05:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.647+0000 7f7d95be6740 1 -- 172.21.15.86:0/921804646 wait complete. 2024-04-25T19:05:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.647+0000 7f7d8be00640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7d700a3dc0 0x7f7d700a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c031440 tx=0x7f7d7c005b90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.647+0000 7f7d95be6740 1 -- start start 2024-04-25T19:05:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.647+0000 7f7d95be6740 1 Processor -- start 2024-04-25T19:05:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d049b0 0x5581d2d0cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581d2d127c0 con 0x5581d2d049b0 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581d2c24310 con 0x5581d2d0fb90 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5581d2d12940 con 0x5581d2d052d0 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d95be6740 10 client.0 _ll_register_callbacks cb 0x5581d2bf2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d049b0 0x5581d2d0cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d90e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34966/0 (socket says 172.21.15.86:34966) 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d8b400640 1 -- 172.21.15.86:0/3632713507 learned_addr learned my addr 172.21.15.86:0/3632713507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d90e00640 1 -- 172.21.15.86:0/3632713507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 msgr2=0x5581d2d0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d90e00640 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d90e00640 1 -- 172.21.15.86:0/3632713507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d049b0 msgr2=0x5581d2d0cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.648+0000 7f7d90e00640 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d049b0 0x5581d2d0cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d90e00640 1 -- 172.21.15.86:0/3632713507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d2a49620 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d8b400640 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d90e00640 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d5d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e6c0 tx=0x7f7d84012c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f911a80c081bc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d76000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d76000640 1 -- 172.21.15.86:0/3632713507 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8402a060 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d84012450 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d76000640 1 -- 172.21.15.86:0/3632713507 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7d84004a40 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.649+0000 7f7d76000640 1 -- 172.21.15.86:0/3632713507 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d84004c40 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 10 client.25649 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581d2d1ea50 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d76000640 1 -- 172.21.15.86:0/3632713507 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d84012450 con 0x5581d2d0fb90 2024-04-25T19:05:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d76000640 1 -- 172.21.15.86:0/3632713507 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7d8400f040 con 0x5581d2d0fb90 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 10 client.25649 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 20 client.25649 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d1fe60 0x5581d2d22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 4 client.25649 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.650+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d2c7e8b0 con 0x5581d2d1fe60 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.651+0000 7f7d8be00640 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d1fe60 0x5581d2d22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.651+0000 7f7d8be00640 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d1fe60 0x5581d2d22250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e18781e78a70b0 server_cookie=c9530c7fb16c05f in_seq=0 out_seq=0 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.651+0000 7f7d76000640 10 client.25649 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.652+0000 7f7d76000640 1 -- 172.21.15.86:0/3632713507 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581d2c7e8b0 con 0x5581d2d1fe60 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.652+0000 7f7d76000640 10 client.25649 handle_command_reply: tid=0 2024-04-25T19:05:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.652+0000 7f7d95be6740 1 client.25649 shutdown 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 msgr2=0x5581d2d0d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d5d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e6c0 tx=0x7f7d84012c80 comp rx=0 tx=0).stop 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 shutdown_connections 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d1fe60 0x5581d2d22250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d5d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d76000640 0 client.25649 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d052d0 0x5581d2d0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3632713507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d049b0 0x5581d2d0cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 >> 172.21.15.86:0/3632713507 conn(0x5581d2d00ec0 msgr2=0x5581d2d06bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 shutdown_connections 2024-04-25T19:05:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 1 -- 172.21.15.86:0/3632713507 wait complete. 2024-04-25T19:05:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.653+0000 7f7d95be6740 20 client.25649 trim_cache size 0 max 16384 2024-04-25T19:05:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 Processor -- start 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 -- start start 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 0x5581d2d0cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1afdc0 con 0x5581d2d0fb90 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b1900 con 0x5581d2d052d0 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.670+0000 7f7d95be6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b0490 con 0x5581d2d049b0 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 0x5581d2d0cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 0x5581d2d0cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43518/0 (socket says 172.21.15.86:43518) 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 -- 172.21.15.86:0/3933589235 learned_addr learned my addr 172.21.15.86:0/3933589235 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d90e00640 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8be00640 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 -- 172.21.15.86:0/3933589235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 msgr2=0x5581d2d07370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d07370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 -- 172.21.15.86:0/3933589235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 msgr2=0x5581d2d0d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 -- 172.21.15.86:0/3933589235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d2d12810 con 0x5581d2d052d0 2024-04-25T19:05:51.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.671+0000 7f7d8b400640 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 0x5581d2d0cb10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ce1c0 tx=0x7f7d780bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceabd415ab35b332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.672+0000 7f7d80c00640 1 -- 172.21.15.86:0/3933589235 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780d9040 con 0x5581d2d052d0 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.672+0000 7f7d80c00640 1 -- 172.21.15.86:0/3933589235 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d780da040 con 0x5581d2d052d0 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.672+0000 7f7d80c00640 1 -- 172.21.15.86:0/3933589235 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7809c960 con 0x5581d2d052d0 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.672+0000 7f7d95be6740 1 -- 172.21.15.86:0/3933589235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 msgr2=0x5581d2d0cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.672+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 0x5581d2d0cb10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ce1c0 tx=0x7f7d780bd040 comp rx=0 tx=0).stop 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 -- 172.21.15.86:0/3933589235 shutdown_connections 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d0fb90 0x5581d2d0d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d052d0 0x5581d2d0cb10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 --2- 172.21.15.86:0/3933589235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 -- 172.21.15.86:0/3933589235 >> 172.21.15.86:0/3933589235 conn(0x5581d2d00ec0 msgr2=0x5581d2d00020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 -- 172.21.15.86:0/3933589235 shutdown_connections 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 -- 172.21.15.86:0/3933589235 wait complete. 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 -- start start 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.673+0000 7f7d95be6740 1 Processor -- start 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d052d0 0x5581d2d7a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d85050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b1230 con 0x5581d2d052d0 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b0b60 con 0x5581d2d0fb90 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1adda0 con 0x5581d2d049b0 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d95be6740 10 client.0 _ll_register_callbacks cb 0x7f7d7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d79fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34982/0 (socket says 172.21.15.86:34982) 2024-04-25T19:05:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 -- 172.21.15.86:0/2208200121 learned_addr learned my addr 172.21.15.86:0/2208200121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8b400640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d052d0 0x5581d2d7a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d90e00640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d85050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 -- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 msgr2=0x5581d2d85050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d85050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 -- 172.21.15.86:0/2208200121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d052d0 msgr2=0x5581d2d7a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d052d0 0x5581d2d7a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.674+0000 7f7d8be00640 1 -- 172.21.15.86:0/2208200121 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d2d85590 con 0x5581d2d049b0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d8be00640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d79fd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c005c40 tx=0x7f7d7c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f064fb24a9b80ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d76000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d76000640 1 -- 172.21.15.86:0/2208200121 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c035040 con 0x5581d2d049b0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d780da040 con 0x5581d2d049b0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d76000640 1 -- 172.21.15.86:0/2208200121 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7d7c005240 con 0x5581d2d049b0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d76000640 1 -- 172.21.15.86:0/2208200121 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c004490 con 0x5581d2d049b0 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d95be6740 10 client.25645 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.675+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581d2d867a0 con 0x5581d2d049b0 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d76000640 1 -- 172.21.15.86:0/2208200121 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d780da040 con 0x5581d2d049b0 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d76000640 1 -- 172.21.15.86:0/2208200121 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7d7c004910 con 0x5581d2d049b0 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d95be6740 10 client.25645 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d95be6740 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d95be6740 20 client.25645 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d95be6740 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d86ea0 0x5581d2d892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d95be6740 4 client.25645 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581d2c78320 con 0x5581d2d86ea0 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.676+0000 7f7d8b400640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d86ea0 0x5581d2d892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.677+0000 7f7d76000640 10 client.25645 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.677+0000 7f7d8b400640 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d86ea0 0x5581d2d892c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54bbed6528ceabb5 server_cookie=20de389f37365ff9 in_seq=0 out_seq=0 2024-04-25T19:05:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d76000640 1 -- 172.21.15.86:0/2208200121 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581d2c78320 con 0x5581d2d86ea0 2024-04-25T19:05:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d76000640 10 client.25645 handle_command_reply: tid=0 2024-04-25T19:05:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 client.25645 shutdown 2024-04-25T19:05:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 msgr2=0x5581d2d79fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d79fd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c005c40 tx=0x7f7d7c009260 comp rx=0 tx=0).stop 2024-04-25T19:05:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 shutdown_connections 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5581d2d86ea0 0x5581d2d892c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581d2d0fb90 0x5581d2d85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d2d052d0 0x5581d2d7a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 --2- 172.21.15.86:0/2208200121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5581d2d049b0 0x5581d2d79fd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 >> 172.21.15.86:0/2208200121 conn(0x5581d2d00ec0 msgr2=0x5581d2d0cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d76000640 0 client.25645 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 shutdown_connections 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.687+0000 7f7d95be6740 1 -- 172.21.15.86:0/2208200121 wait complete. 2024-04-25T19:05:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.688+0000 7f7d95be6740 20 client.25645 trim_cache size 0 max 16384 2024-04-25T19:05:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:05:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:05:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:05:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:05:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.688+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7d700a3dc0 msgr2=0x7f7d700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.688+0000 7f7d91800640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7d700a3dc0 0x7f7d700a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c031440 tx=0x7f7d7c005b90 comp rx=0 tx=0).stop 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.688+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 msgr2=0x7f7d8c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.688+0000 7f7d91800640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c074790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9a0 tx=0x7f7d7800ee60 comp rx=0 tx=0).stop 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 shutdown_connections 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7d700a3dc0 0x7f7d700a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d8c074cd0 0x7f7d8c1a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d8c1070a0 0x7f7d8c074790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 --2- 172.21.15.86:0/2446058023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d8c101fd0 0x7f7d8c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 >> 172.21.15.86:0/2446058023 conn(0x7f7d8c0fd8d0 msgr2=0x7f7d8c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 shutdown_connections 2024-04-25T19:05:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:05:51.689+0000 7f7d91800640 1 -- 172.21.15.86:0/2446058023 wait complete. 2024-04-25T19:05:52.493 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-25T19:05:52.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-25T19:05:52.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:52 smithi183 ceph-mon[32344]: pgmap v90: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-25T19:05:52.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:52 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2117248292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:52.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:52 smithi194 ceph-mon[32740]: pgmap v90: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-25T19:05:52.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:52 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2117248292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:52 smithi086 ceph-mon[27783]: pgmap v90: 97 pgs: 97 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-25T19:05:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:52 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2117248292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:05:52.935 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:05:54.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:54 smithi086 ceph-mon[27783]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T19:05:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:54 smithi183 ceph-mon[32344]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T19:05:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:54 smithi194 ceph-mon[32740]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 -- 172.21.15.86:0/4118837638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c075590 msgr2=0x7f826c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 --2- 172.21.15.86:0/4118837638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c075590 0x7f826c10ef60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254009ca0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 -- 172.21.15.86:0/4118837638 shutdown_connections 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 --2- 172.21.15.86:0/4118837638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 0x7f826c111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 --2- 172.21.15.86:0/4118837638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c074c70 0x7f826c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 --2- 172.21.15.86:0/4118837638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c075590 0x7f826c10ef60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 -- 172.21.15.86:0/4118837638 >> 172.21.15.86:0/4118837638 conn(0x7f826c079b10 msgr2=0x7f826c079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 -- 172.21.15.86:0/4118837638 shutdown_connections 2024-04-25T19:05:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.142+0000 7f8271000700 1 -- 172.21.15.86:0/4118837638 wait complete. 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.143+0000 7f8271000700 1 Processor -- start 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.143+0000 7f8271000700 1 -- start start 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8271000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 0x7f826c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8271000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c075590 0x7f826c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8271000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 0x7f826c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8271000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f826c114e00 con 0x7f826c074c70 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8271000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f826c114c80 con 0x7f826c075590 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8271000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f826c114f80 con 0x7f826c10f4a0 2024-04-25T19:05:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 0x7f826c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 0x7f826c19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54874/0 (socket says 172.21.15.86:54874) 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8269e00700 1 -- 172.21.15.86:0/2043427600 learned_addr learned my addr 172.21.15.86:0/2043427600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f8261400700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c075590 0x7f826c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.144+0000 7f826a800700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 0x7f826c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8269e00700 1 -- 172.21.15.86:0/2043427600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 msgr2=0x7f826c19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8269e00700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 0x7f826c19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8269e00700 1 -- 172.21.15.86:0/2043427600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c075590 msgr2=0x7f826c19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8269e00700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c075590 0x7f826c19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8269e00700 1 -- 172.21.15.86:0/2043427600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c1a0220 con 0x7f826c074c70 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f826a800700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 0x7f826c19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8261400700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c075590 0x7f826c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8269e00700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 0x7f826c19b2b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f825c00e9a0 tx=0x7f825c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8263400700 1 -- 172.21.15.86:0/2043427600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c00cbd0 con 0x7f826c074c70 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8263400700 1 -- 172.21.15.86:0/2043427600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c00cd70 con 0x7f826c074c70 2024-04-25T19:05:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c1a0510 con 0x7f826c074c70 2024-04-25T19:05:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.145+0000 7f8263400700 1 -- 172.21.15.86:0/2043427600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c018680 con 0x7f826c074c70 2024-04-25T19:05:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.146+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1a7d90 con 0x7f826c074c70 2024-04-25T19:05:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.149+0000 7f8263400700 1 -- 172.21.15.86:0/2043427600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f825c010430 con 0x7f826c074c70 2024-04-25T19:05:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.150+0000 7f8263400700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82500a3f50 0x7f82500a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.150+0000 7f827b8fbb80 1 Processor -- start 2024-04-25T19:05:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.150+0000 7f8263400700 1 -- 172.21.15.86:0/2043427600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f825c0d0ab0 con 0x7f826c074c70 2024-04-25T19:05:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.150+0000 7f827b8fbb80 1 -- start start 2024-04-25T19:05:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f8261400700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82500a3f50 0x7f82500a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c9994d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c999fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99a2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f7c98e8020 con 0x55f7c99a0180 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c988e8c0 con 0x55f7c99949b0 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f7c98e3fe0 con 0x55f7c99952d0 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f8269e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c9994d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f8269e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c9994d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43556/0 (socket says 172.21.15.86:43556) 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f8269e00700 1 -- 172.21.15.86:0/275942318 learned_addr learned my addr 172.21.15.86:0/275942318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f826a800700 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.151+0000 7f8261400700 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c999fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8261400700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82500a3f50 0x7f82500a6410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82540098e0 tx=0x7f8254005960 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8269e00700 1 -- 172.21.15.86:0/275942318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 msgr2=0x55f7c999fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8269e00700 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c999fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8269e00700 1 -- 172.21.15.86:0/275942318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 msgr2=0x55f7c99a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8269e00700 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8269e00700 1 -- 172.21.15.86:0/275942318 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c9884810 con 0x55f7c99949b0 2024-04-25T19:05:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f8269e00700 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c9994d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f825c099370 tx=0x7f825c098d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd40220e85fb1999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f825b400700 1 -- 172.21.15.86:0/275942318 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c09c9a0 con 0x55f7c99949b0 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f825b400700 1 -- 172.21.15.86:0/275942318 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c0a0d50 con 0x55f7c99949b0 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.152+0000 7f825b400700 1 -- 172.21.15.86:0/275942318 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0870f0 con 0x55f7c99949b0 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 -- 172.21.15.86:0/275942318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 msgr2=0x55f7c9994d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c9994d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f825c099370 tx=0x7f825c098d70 comp rx=0 tx=0).stop 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 -- 172.21.15.86:0/275942318 shutdown_connections 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c9994d90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/275942318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c999fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 -- 172.21.15.86:0/275942318 >> 172.21.15.86:0/275942318 conn(0x55f7c98c3730 msgr2=0x55f7c9990e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.153+0000 7f827b8fbb80 1 -- 172.21.15.86:0/275942318 shutdown_connections 2024-04-25T19:05:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.154+0000 7f827b8fbb80 1 -- 172.21.15.86:0/275942318 wait complete. 2024-04-25T19:05:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.154+0000 7f827b8fbb80 1 -- start start 2024-04-25T19:05:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.154+0000 7f827b8fbb80 1 Processor -- start 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c999f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99952d0 0x55f7c999cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c999d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f7c99a2c40 con 0x55f7c99952d0 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c98a19f0 con 0x55f7c99949b0 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f7c99a2dc0 con 0x55f7c99a0180 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f827b8fbb80 10 client.0 _ll_register_callbacks cb 0x55f7c98808b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f8261400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99952d0 0x55f7c999cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f8269e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c999f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f8261400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99952d0 0x55f7c999cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54896/0 (socket says 172.21.15.86:54896) 2024-04-25T19:05:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f8261400700 1 -- 172.21.15.86:0/3314815163 learned_addr learned my addr 172.21.15.86:0/3314815163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.155+0000 7f826a800700 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c999d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f8269e00700 1 -- 172.21.15.86:0/3314815163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 msgr2=0x55f7c999d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f8269e00700 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c999d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f8269e00700 1 -- 172.21.15.86:0/3314815163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99952d0 msgr2=0x55f7c999cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f8269e00700 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99952d0 0x55f7c999cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f8269e00700 1 -- 172.21.15.86:0/3314815163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c9835a60 con 0x55f7c99949b0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f8269e00700 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c999f8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f825c099370 tx=0x7f825c010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0b3ef03276b76b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f824fe00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f824fe00700 1 -- 172.21.15.86:0/3314815163 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0b5030 con 0x55f7c99949b0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f824fe00700 1 -- 172.21.15.86:0/3314815163 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f825c087300 con 0x55f7c99949b0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c0a0d50 con 0x55f7c99949b0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f824fe00700 1 -- 172.21.15.86:0/3314815163 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f825c080050 con 0x55f7c99949b0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f827b8fbb80 10 client.25667 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.156+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7c987ec80 con 0x55f7c99949b0 2024-04-25T19:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f824fe00700 1 -- 172.21.15.86:0/3314815163 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c0a0d50 con 0x55f7c99949b0 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f824fe00700 1 -- 172.21.15.86:0/3314815163 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f825c0bb030 con 0x55f7c99949b0 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f827b8fbb80 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f827b8fbb80 10 client.25667 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f827b8fbb80 20 client.25667 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c99af7f0 0x55f7c99b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f827b8fbb80 4 client.25667 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.157+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c98f5810 con 0x55f7c99af7f0 2024-04-25T19:05:55.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.158+0000 7f8261400700 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c99af7f0 0x55f7c99b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.158+0000 7f8261400700 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c99af7f0 0x55f7c99b1be0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b35c231dfea1604 server_cookie=bd9f5429d822a1d6 in_seq=0 out_seq=0 2024-04-25T19:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.158+0000 7f824fe00700 10 client.25667 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f824fe00700 1 -- 172.21.15.86:0/3314815163 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7c98f5810 con 0x55f7c99af7f0 2024-04-25T19:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f824fe00700 10 client.25667 handle_command_reply: tid=0 2024-04-25T19:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 client.25667 shutdown 2024-04-25T19:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 msgr2=0x55f7c999f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c999f8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f825c099370 tx=0x7f825c010cb0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 shutdown_connections 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c99af7f0 0x55f7c99b1be0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c999d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c999f8f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/3314815163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99952d0 0x55f7c999cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f824fe00700 0 client.25667 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.159+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 >> 172.21.15.86:0/3314815163 conn(0x55f7c98c3730 msgr2=0x55f7c9990a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.160+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 shutdown_connections 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.160+0000 7f827b8fbb80 1 -- 172.21.15.86:0/3314815163 wait complete. 2024-04-25T19:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.160+0000 7f827b8fbb80 20 client.25667 trim_cache size 0 max 16384 2024-04-25T19:05:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 Processor -- start 2024-04-25T19:05:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 -- start start 2024-04-25T19:05:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 0x55f7c9998140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99952d0 0x55f7c999d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c9999f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1af5c0 con 0x55f7c99949b0 2024-04-25T19:05:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1b1100 con 0x55f7c99952d0 2024-04-25T19:05:55.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.181+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1afc90 con 0x55f7c99a0180 2024-04-25T19:05:55.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 0x55f7c9998140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 0x55f7c9998140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54904/0 (socket says 172.21.15.86:54904) 2024-04-25T19:05:55.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 -- 172.21.15.86:0/500740149 learned_addr learned my addr 172.21.15.86:0/500740149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:55.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8261400700 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99952d0 0x55f7c999d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f826a800700 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c9999f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 -- 172.21.15.86:0/500740149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 msgr2=0x55f7c9999f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c9999f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 -- 172.21.15.86:0/500740149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99952d0 msgr2=0x55f7c999d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99952d0 0x55f7c999d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8269e00700 1 -- 172.21.15.86:0/500740149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c99956b0 con 0x55f7c99949b0 2024-04-25T19:05:55.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.182+0000 7f8261400700 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99952d0 0x55f7c999d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.183+0000 7f8269e00700 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 0x55f7c9998140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c5b0 tx=0x7f825c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b82dfb44c4f1a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.183+0000 7f825aa00700 1 -- 172.21.15.86:0/500740149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0b5070 con 0x55f7c99949b0 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.183+0000 7f825aa00700 1 -- 172.21.15.86:0/500740149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c010720 con 0x55f7c99949b0 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.183+0000 7f825aa00700 1 -- 172.21.15.86:0/500740149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0bbb60 con 0x55f7c99949b0 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.183+0000 7f827b8fbb80 1 -- 172.21.15.86:0/500740149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 msgr2=0x55f7c9998140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.183+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 0x55f7c9998140 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c5b0 tx=0x7f825c00cf90 comp rx=0 tx=0).stop 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 -- 172.21.15.86:0/500740149 shutdown_connections 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99a0180 0x55f7c9999f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99949b0 0x55f7c9998140 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/500740149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99952d0 0x55f7c999d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 -- 172.21.15.86:0/500740149 >> 172.21.15.86:0/500740149 conn(0x55f7c98c3730 msgr2=0x55f7c99972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 -- 172.21.15.86:0/500740149 shutdown_connections 2024-04-25T19:05:55.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 -- 172.21.15.86:0/500740149 wait complete. 2024-04-25T19:05:55.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.184+0000 7f827b8fbb80 1 -- start start 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 Processor -- start 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c99b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c99b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1b0a30 con 0x55f7c99a0180 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1b0360 con 0x55f7c99949b0 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1ad560 con 0x55f7c99952d0 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.185+0000 7f827b8fbb80 10 client.0 _ll_register_callbacks cb 0x55f7c98808b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c99b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f826a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c99b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35046/0 (socket says 172.21.15.86:35046) 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 -- 172.21.15.86:0/2239094096 learned_addr learned my addr 172.21.15.86:0/2239094096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8269e00700 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c99b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 -- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 msgr2=0x55f7c99b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c99b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 -- 172.21.15.86:0/2239094096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 msgr2=0x55f7c99b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 -- 172.21.15.86:0/2239094096 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c9a15a10 con 0x55f7c99952d0 2024-04-25T19:05:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f8261400700 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c99b3bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82540059d0 tx=0x7f8254009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2469470e6871e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:55.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.186+0000 7f824fe00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:05:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f824fe00700 1 -- 172.21.15.86:0/2239094096 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254003ce0 con 0x55f7c99952d0 2024-04-25T19:05:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c010720 con 0x55f7c99952d0 2024-04-25T19:05:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f824fe00700 1 -- 172.21.15.86:0/2239094096 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f82540360b0 con 0x55f7c99952d0 2024-04-25T19:05:55.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f824fe00700 1 -- 172.21.15.86:0/2239094096 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8254046050 con 0x55f7c99952d0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f827b8fbb80 10 client.25663 fetch_fsmap learned FSMap version 29 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7c9a16d60 con 0x55f7c99952d0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f824fe00700 1 -- 172.21.15.86:0/2239094096 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c010720 con 0x55f7c99952d0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f824fe00700 1 -- 172.21.15.86:0/2239094096 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f825400f770 con 0x55f7c99952d0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f827b8fbb80 10 client.25663 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f827b8fbb80 10 client.25663 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.187+0000 7f827b8fbb80 20 client.25663 populate_metadata read hostname 'smithi086' 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.188+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c9a174c0 0x55f7c9a198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.188+0000 7f827b8fbb80 4 client.25663 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.188+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55f7c98ef8b0 con 0x55f7c9a174c0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.188+0000 7f8269e00700 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c9a174c0 0x55f7c9a198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.188+0000 7f8269e00700 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c9a174c0 0x55f7c9a198e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4666c1bcef0fe74 server_cookie=a9e9ece8d1dc302f in_seq=0 out_seq=0 2024-04-25T19:05:55.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.188+0000 7f824fe00700 10 client.25663 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:55.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f824fe00700 1 -- 172.21.15.86:0/2239094096 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f7c98ef8b0 con 0x55f7c9a174c0 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f824fe00700 10 client.25663 handle_command_reply: tid=0 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 client.25663 shutdown 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 msgr2=0x55f7c99b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c99b3bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82540059d0 tx=0x7f8254009310 comp rx=0 tx=0).stop 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 shutdown_connections 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7c9a174c0 0x55f7c9a198e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7c99a0180 0x55f7c99b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c99949b0 0x55f7c99b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 --2- 172.21.15.86:0/2239094096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7c99952d0 0x55f7c99b3bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 >> 172.21.15.86:0/2239094096 conn(0x55f7c98c3730 msgr2=0x55f7c999e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:55.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.204+0000 7f824fe00700 0 client.25663 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:05:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.205+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 shutdown_connections 2024-04-25T19:05:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.205+0000 7f827b8fbb80 1 -- 172.21.15.86:0/2239094096 wait complete. 2024-04-25T19:05:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.205+0000 7f827b8fbb80 20 client.25663 trim_cache size 0 max 16384 2024-04-25T19:05:55.208 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 10, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "ff9b5964": { 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 10, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 90.0, 2024-04-25T19:05:55.209 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:05:55.210 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82500a3f50 msgr2=0x7f82500a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82500a3f50 0x7f82500a6410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82540098e0 tx=0x7f8254005960 comp rx=0 tx=0).stop 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 msgr2=0x7f826c19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 0x7f826c19b2b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f825c00e9a0 tx=0x7f825c00ecb0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 shutdown_connections 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f82500a3f50 0x7f82500a6410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f826c10f4a0 0x7f826c19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f826c074c70 0x7f826c19b2b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 --2- 172.21.15.86:0/2043427600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f826c075590 0x7f826c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 >> 172.21.15.86:0/2043427600 conn(0x7f826c079b10 msgr2=0x7f826c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 shutdown_connections 2024-04-25T19:05:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:05:55.206+0000 7f8271000700 1 -- 172.21.15.86:0/2043427600 wait complete. 2024-04-25T19:05:56.284 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ff9b5964', quiesce took 7.7 seconds. Will release after: 30.0 2024-04-25T19:05:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:56 smithi086 ceph-mon[27783]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T19:05:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:56 smithi183 ceph-mon[32344]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T19:05:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:56 smithi194 ceph-mon[32740]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T19:05:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:05:58 smithi086 ceph-mon[27783]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T19:05:58.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:05:58 smithi183 ceph-mon[32344]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T19:05:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:05:58 smithi194 ceph-mon[32740]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T19:06:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:00 smithi183 ceph-mon[32344]: pgmap v94: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T19:06:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:00 smithi194 ceph-mon[32740]: pgmap v94: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T19:06:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:00 smithi086 ceph-mon[27783]: pgmap v94: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-25T19:06:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:02 smithi183 ceph-mon[32344]: pgmap v95: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-25T19:06:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:02 smithi194 ceph-mon[32740]: pgmap v95: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-25T19:06:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:02 smithi086 ceph-mon[27783]: pgmap v95: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-25T19:06:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:04 smithi183 ceph-mon[32344]: pgmap v96: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 KiB/s wr, 33 op/s 2024-04-25T19:06:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:04 smithi194 ceph-mon[32740]: pgmap v96: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 KiB/s wr, 33 op/s 2024-04-25T19:06:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:04 smithi086 ceph-mon[27783]: pgmap v96: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 39 KiB/s wr, 33 op/s 2024-04-25T19:06:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.829+0000 7f614b400640 1 -- 172.21.15.86:0/291229481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 msgr2=0x7f614406a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.829+0000 7f614b400640 1 --2- 172.21.15.86:0/291229481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614406a780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6138009860 tx=0x7f613802a340 comp rx=0 tx=0).stop 2024-04-25T19:06:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 -- 172.21.15.86:0/291229481 shutdown_connections 2024-04-25T19:06:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 --2- 172.21.15.86:0/291229481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f614406b3a0 0x7f6144104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 --2- 172.21.15.86:0/291229481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614406acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 --2- 172.21.15.86:0/291229481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614406a780 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 -- 172.21.15.86:0/291229481 >> 172.21.15.86:0/291229481 conn(0x7f6144076b80 msgr2=0x7f6144076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 -- 172.21.15.86:0/291229481 shutdown_connections 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.830+0000 7f614b400640 1 -- 172.21.15.86:0/291229481 wait complete. 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.831+0000 7f614b400640 1 Processor -- start 2024-04-25T19:06:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.831+0000 7f614b400640 1 -- start start 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f614b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f614b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f614b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61441a3610 0x7f61441a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f614b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6144118880 con 0x7f61441073b0 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f614b400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6144118700 con 0x7f61441a3610 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f614b400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6144118a00 con 0x7f6144106f40 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614419f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42152/0 (socket says 172.21.15.86:42152) 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 -- 172.21.15.86:0/2722244947 learned_addr learned my addr 172.21.15.86:0/2722244947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6149400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61441a3610 0x7f61441a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6148a00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 -- 172.21.15.86:0/2722244947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 msgr2=0x7f614419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 -- 172.21.15.86:0/2722244947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61441a3610 msgr2=0x7f61441a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61441a3610 0x7f61441a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6143e00640 1 -- 172.21.15.86:0/2722244947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441a4100 con 0x7f61441073b0 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.832+0000 7f6148a00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.833+0000 7f6143e00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614419f2e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f61300058c0 tx=0x7f613000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.833+0000 7f6141600640 1 -- 172.21.15.86:0/2722244947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000cd30 con 0x7f61441073b0 2024-04-25T19:06:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.833+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441a43f0 con 0x7f61441073b0 2024-04-25T19:06:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.833+0000 7f6141600640 1 -- 172.21.15.86:0/2722244947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61300035f0 con 0x7f61441073b0 2024-04-25T19:06:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.833+0000 7f6141600640 1 -- 172.21.15.86:0/2722244947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300044e0 con 0x7f61441073b0 2024-04-25T19:06:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.833+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441abc30 con 0x7f61441073b0 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.836+0000 7f614e376740 1 Processor -- start 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.836+0000 7f6141600640 1 -- 172.21.15.86:0/2722244947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6130003a50 con 0x7f61441073b0 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 -- start start 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 0x562334a1ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a29a00 0x562334a2bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562334984900 con 0x562334a1e820 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56233492aef0 con 0x562334a29a00 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f614e376740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5623349808c0 con 0x562334a1f140 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f6149400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 0x562334a1ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f6149400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 0x562334a1ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42162/0 (socket says 172.21.15.86:42162) 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f6149400640 1 -- 172.21.15.86:0/2947499254 learned_addr learned my addr 172.21.15.86:0/2947499254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f6148a00640 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.837+0000 7f6143e00640 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a29a00 0x562334a2bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6149400640 1 -- 172.21.15.86:0/2947499254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 msgr2=0x562334a294c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6149400640 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a294c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6149400640 1 -- 172.21.15.86:0/2947499254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a29a00 msgr2=0x562334a2bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6149400640 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a29a00 0x562334a2bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6149400640 1 -- 172.21.15.86:0/2947499254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562334697210 con 0x562334a1e820 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6149400640 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 0x562334a1ec00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f613c009fd0 tx=0x7f613c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3373db8f290fc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6135c00640 1 -- 172.21.15.86:0/2947499254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c004d20 con 0x562334a1e820 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6135c00640 1 -- 172.21.15.86:0/2947499254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f613c004050 con 0x562334a1e820 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6135c00640 1 -- 172.21.15.86:0/2947499254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c005280 con 0x562334a1e820 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.838+0000 7f6141600640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f612c0a3e40 0x7f612c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f6141600640 1 -- 172.21.15.86:0/2722244947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f61300d0af0 con 0x7f61441073b0 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f6148a00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f612c0a3e40 0x7f612c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 -- 172.21.15.86:0/2947499254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 msgr2=0x562334a1ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 0x562334a1ec00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f613c009fd0 tx=0x7f613c0118d0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 -- 172.21.15.86:0/2947499254 shutdown_connections 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a29a00 0x562334a2bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 --2- 172.21.15.86:0/2947499254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1e820 0x562334a1ec00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.839+0000 7f614e376740 1 -- 172.21.15.86:0/2947499254 >> 172.21.15.86:0/2947499254 conn(0x562334a1ad30 msgr2=0x562334a1b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.840+0000 7f614e376740 1 -- 172.21.15.86:0/2947499254 shutdown_connections 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.840+0000 7f614e376740 1 -- 172.21.15.86:0/2947499254 wait complete. 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.840+0000 7f6148a00640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f612c0a3e40 0x7f612c0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6138000c00 tx=0x7f6138005a90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.840+0000 7f614e376740 1 -- start start 2024-04-25T19:06:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.840+0000 7f614e376740 1 Processor -- start 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a26980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a26ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a27400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562334a2c630 con 0x562334a1f140 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56233493e310 con 0x562334a1e820 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562334a2c7b0 con 0x562334a29a00 2024-04-25T19:06:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f614e376740 10 client.0 _ll_register_callbacks cb 0x56233490ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f6143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a26ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f6148a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a26980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f6149400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a27400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f6148a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a26980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46414/0 (socket says 172.21.15.86:46414) 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.841+0000 7f6148a00640 1 -- 172.21.15.86:0/1953814071 learned_addr learned my addr 172.21.15.86:0/1953814071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6148a00640 1 -- 172.21.15.86:0/1953814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 msgr2=0x562334a27400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6148a00640 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a27400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6148a00640 1 -- 172.21.15.86:0/1953814071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 msgr2=0x562334a26ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6148a00640 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a26ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6148a00640 1 -- 172.21.15.86:0/1953814071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562334763620 con 0x562334a1e820 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6149400640 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a27400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f6148a00640 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a26980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6138034000 tx=0x7f613802a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70adfce49ab5c360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f612a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:06:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f612a000640 1 -- 172.21.15.86:0/1953814071 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138037040 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f613c004050 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.842+0000 7f612a000640 1 -- 172.21.15.86:0/1953814071 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f6138030860 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.843+0000 7f612a000640 1 -- 172.21.15.86:0/1953814071 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6138057050 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.843+0000 7f614e376740 10 client.25685 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.843+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562334a388c0 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.843+0000 7f612a000640 1 -- 172.21.15.86:0/1953814071 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f613c004050 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.843+0000 7f612a000640 1 -- 172.21.15.86:0/1953814071 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6138052030 con 0x562334a1e820 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f614e376740 10 client.25685 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f614e376740 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f614e376740 20 client.25685 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f614e376740 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334a39dd0 0x562334a3c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f614e376740 4 client.25685 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623349988b0 con 0x562334a39dd0 2024-04-25T19:06:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.844+0000 7f6143e00640 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334a39dd0 0x562334a3c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.845+0000 7f612a000640 10 client.25685 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:04.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.845+0000 7f6143e00640 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334a39dd0 0x562334a3c200 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01eb4077f025074 server_cookie=bfeb7e087b939a02 in_seq=0 out_seq=0 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.845+0000 7f612a000640 1 -- 172.21.15.86:0/1953814071 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623349988b0 con 0x562334a39dd0 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.845+0000 7f612a000640 10 client.25685 handle_command_reply: tid=0 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.845+0000 7f614e376740 1 client.25685 shutdown 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 msgr2=0x562334a26980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a26980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6138034000 tx=0x7f613802a340 comp rx=0 tx=0).stop 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 shutdown_connections 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334a39dd0 0x562334a3c200 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a27400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a26ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 --2- 172.21.15.86:0/1953814071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a26980 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f612a000640 0 client.25685 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 >> 172.21.15.86:0/1953814071 conn(0x562334a1ad30 msgr2=0x562334a20a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 shutdown_connections 2024-04-25T19:06:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 1 -- 172.21.15.86:0/1953814071 wait complete. 2024-04-25T19:06:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.846+0000 7f614e376740 20 client.25685 trim_cache size 0 max 16384 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 Processor -- start 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 -- start start 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a41b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61441b42b0 con 0x562334a1f140 2024-04-25T19:06:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61441b5df0 con 0x562334a1e820 2024-04-25T19:06:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.862+0000 7f614e376740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f61441b4980 con 0x562334a29a00 2024-04-25T19:06:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a15f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42168/0 (socket says 172.21.15.86:42168) 2024-04-25T19:06:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 -- 172.21.15.86:0/2776835289 learned_addr learned my addr 172.21.15.86:0/2776835289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6148a00640 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6149400640 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a41b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 -- 172.21.15.86:0/2776835289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 msgr2=0x562334a41b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a41b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 -- 172.21.15.86:0/2776835289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 msgr2=0x562334a354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 -- 172.21.15.86:0/2776835289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562334a25970 con 0x562334a1f140 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.863+0000 7f6143e00640 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a15f60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f61300993b0 tx=0x7f6130087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df552a4923454b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f6135200640 1 -- 172.21.15.86:0/2776835289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613009c340 con 0x562334a1f140 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f6135200640 1 -- 172.21.15.86:0/2776835289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6130009530 con 0x562334a1f140 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f6135200640 1 -- 172.21.15.86:0/2776835289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613009aa80 con 0x562334a1f140 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 -- 172.21.15.86:0/2776835289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 msgr2=0x562334a15f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a15f60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f61300993b0 tx=0x7f6130087040 comp rx=0 tx=0).stop 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 -- 172.21.15.86:0/2776835289 shutdown_connections 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a29a00 0x562334a41b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a1f140 0x562334a15f60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 --2- 172.21.15.86:0/2776835289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.864+0000 7f614e376740 1 -- 172.21.15.86:0/2776835289 >> 172.21.15.86:0/2776835289 conn(0x562334a1ad30 msgr2=0x562334a19e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.865+0000 7f614e376740 1 -- 172.21.15.86:0/2776835289 shutdown_connections 2024-04-25T19:06:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.865+0000 7f614e376740 1 -- 172.21.15.86:0/2776835289 wait complete. 2024-04-25T19:06:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.865+0000 7f614e376740 1 -- start start 2024-04-25T19:06:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.865+0000 7f614e376740 1 Processor -- start 2024-04-25T19:06:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a31710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a31c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a29a00 0x562334a2fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61441b5720 con 0x562334a29a00 2024-04-25T19:06:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61441b5050 con 0x562334a1e820 2024-04-25T19:06:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f61441b2290 con 0x562334a1f140 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f614e376740 10 client.0 _ll_register_callbacks cb 0x7f61380096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a31710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6143e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a31c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a31710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46420/0 (socket says 172.21.15.86:46420) 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 -- 172.21.15.86:0/2014653098 learned_addr learned my addr 172.21.15.86:0/2014653098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6149400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a29a00 0x562334a2fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 -- 172.21.15.86:0/2014653098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 msgr2=0x562334a31c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a31c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 -- 172.21.15.86:0/2014653098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a29a00 msgr2=0x562334a2fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a29a00 0x562334a2fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.866+0000 7f6148a00640 1 -- 172.21.15.86:0/2014653098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562334a30340 con 0x562334a1e820 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f6148a00640 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a31710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6138004db0 tx=0x7f6138003670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c79348c20f557b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f612a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:06:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f612a000640 1 -- 172.21.15.86:0/2014653098 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138037040 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562334992320 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f612a000640 1 -- 172.21.15.86:0/2014653098 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f61380046c0 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f612a000640 1 -- 172.21.15.86:0/2014653098 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61380048c0 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f614e376740 10 client.25691 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562334aa0df0 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.867+0000 7f612a000640 1 -- 172.21.15.86:0/2014653098 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562334992320 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f612a000640 1 -- 172.21.15.86:0/2014653098 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6138052040 con 0x562334a1e820 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f614e376740 10 client.25691 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f614e376740 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f614e376740 20 client.25691 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f614e376740 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334aa1570 0x562334aa3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f614e376740 4 client.25691 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56233498c190 con 0x562334aa1570 2024-04-25T19:06:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.868+0000 7f6143e00640 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334aa1570 0x562334aa3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.869+0000 7f6143e00640 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334aa1570 0x562334aa3990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d21498db46b2cf server_cookie=341c2fa97c8d0e7 in_seq=0 out_seq=0 2024-04-25T19:06:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.869+0000 7f612a000640 10 client.25691 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f612a000640 1 -- 172.21.15.86:0/2014653098 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56233498c190 con 0x562334aa1570 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f612a000640 10 client.25691 handle_command_reply: tid=0 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 client.25691 shutdown 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 msgr2=0x562334a31710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a31710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6138004db0 tx=0x7f6138003670 comp rx=0 tx=0).stop 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 shutdown_connections 2024-04-25T19:06:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562334aa1570 0x562334aa3990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562334a29a00 0x562334a2fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562334a1f140 0x562334a31c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 --2- 172.21.15.86:0/2014653098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562334a1e820 0x562334a31710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 >> 172.21.15.86:0/2014653098 conn(0x562334a1ad30 msgr2=0x562334a201d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f612a000640 0 client.25691 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 shutdown_connections 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 1 -- 172.21.15.86:0/2014653098 wait complete. 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.881+0000 7f614e376740 20 client.25691 trim_cache size 0 max 16384 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:06:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:06:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:06:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f612c0a3e40 msgr2=0x7f612c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f612c0a3e40 0x7f612c0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6138000c00 tx=0x7f6138005a90 comp rx=0 tx=0).stop 2024-04-25T19:06:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 msgr2=0x7f614419f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614419f2e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f61300058c0 tx=0x7f613000ee50 comp rx=0 tx=0).stop 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 shutdown_connections 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f612c0a3e40 0x7f612c0a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61441a3610 0x7f61441a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61441073b0 0x7f614419f2e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 --2- 172.21.15.86:0/2722244947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6144106f40 0x7f614419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 >> 172.21.15.86:0/2722244947 conn(0x7f6144076b80 msgr2=0x7f6144101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 shutdown_connections 2024-04-25T19:06:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:04.882+0000 7f614b400640 1 -- 172.21.15.86:0/2722244947 wait complete. 2024-04-25T19:06:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:06 smithi183 ceph-mon[32344]: pgmap v97: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-25T19:06:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:06 smithi194 ceph-mon[32740]: pgmap v97: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-25T19:06:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:06 smithi086 ceph-mon[27783]: pgmap v97: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-25T19:06:08.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:08 smithi183 ceph-mon[32344]: pgmap v98: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-25T19:06:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:08 smithi194 ceph-mon[32740]: pgmap v98: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-25T19:06:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:08 smithi086 ceph-mon[27783]: pgmap v98: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-25T19:06:09.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:09 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:06:09.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:09 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:06:09.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:09 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:06:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:10 smithi183 ceph-mon[32344]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 31 op/s 2024-04-25T19:06:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:10 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:06:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:10 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:06:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:10 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:06:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:10 smithi194 ceph-mon[32740]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 31 op/s 2024-04-25T19:06:10.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:10 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:06:10.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:10 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:06:10.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:10 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:06:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:10 smithi086 ceph-mon[27783]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 31 op/s 2024-04-25T19:06:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:10 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:06:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:10 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:06:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:10 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:06:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:12 smithi183 ceph-mon[32344]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-25T19:06:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:12 smithi194 ceph-mon[32740]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-25T19:06:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:12 smithi086 ceph-mon[27783]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-25T19:06:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:14 smithi194 ceph-mon[32740]: pgmap v101: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-04-25T19:06:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:14 smithi086 ceph-mon[27783]: pgmap v101: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-04-25T19:06:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:14 smithi183 ceph-mon[32344]: pgmap v101: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-04-25T19:06:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:16 smithi194 ceph-mon[32740]: pgmap v102: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:06:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:16 smithi086 ceph-mon[27783]: pgmap v102: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:06:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:16 smithi183 ceph-mon[32344]: pgmap v102: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 -- 172.21.15.86:0/2375426673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 msgr2=0x7f90f010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 --2- 172.21.15.86:0/2375426673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 0x7f90f010bd10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009860 tx=0x7f90dc02a3d0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 -- 172.21.15.86:0/2375426673 shutdown_connections 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 --2- 172.21.15.86:0/2375426673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f010c330 0x7f90f010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 --2- 172.21.15.86:0/2375426673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 0x7f90f010bd10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 --2- 172.21.15.86:0/2375426673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f0102f10 0x7f90f01032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 -- 172.21.15.86:0/2375426673 >> 172.21.15.86:0/2375426673 conn(0x7f90f00fc770 msgr2=0x7f90f00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 -- 172.21.15.86:0/2375426673 shutdown_connections 2024-04-25T19:06:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.016+0000 7f90f5200640 1 -- 172.21.15.86:0/2375426673 wait complete. 2024-04-25T19:06:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.017+0000 7f90f5200640 1 Processor -- start 2024-04-25T19:06:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.017+0000 7f90f5200640 1 -- start start 2024-04-25T19:06:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90f5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 0x7f90f01a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90f5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 0x7f90f01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90f5200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f010c330 0x7f90f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90f5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0112010 con 0x7f90f0103830 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90f5200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0111e90 con 0x7f90f0102f10 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90f5200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0112190 con 0x7f90f010c330 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 0x7f90f01a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 0x7f90f01a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:32946/0 (socket says 172.21.15.86:32946) 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 -- 172.21.15.86:0/148624988 learned_addr learned my addr 172.21.15.86:0/148624988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90eea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 0x7f90f01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90efe00640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f010c330 0x7f90f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 -- 172.21.15.86:0/148624988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f010c330 msgr2=0x7f90f019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f010c330 0x7f90f019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 -- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 msgr2=0x7f90f01a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 0x7f90f01a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.018+0000 7f90ef400640 1 -- 172.21.15.86:0/148624988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f019aef0 con 0x7f90f0102f10 2024-04-25T19:06:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.019+0000 7f90ef400640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 0x7f90f01a0570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90e000e9f0 tx=0x7f90e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.019+0000 7f90e7e00640 1 -- 172.21.15.86:0/148624988 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000ccc0 con 0x7f90f0102f10 2024-04-25T19:06:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.019+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f019b1e0 con 0x7f90f0102f10 2024-04-25T19:06:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.019+0000 7f90e7e00640 1 -- 172.21.15.86:0/148624988 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e00054d0 con 0x7f90f0102f10 2024-04-25T19:06:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.019+0000 7f90e7e00640 1 -- 172.21.15.86:0/148624988 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0010430 con 0x7f90f0102f10 2024-04-25T19:06:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.019+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a74f0 con 0x7f90f0102f10 2024-04-25T19:06:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.023+0000 7f90f95da740 1 Processor -- start 2024-04-25T19:06:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 -- start start 2024-04-25T19:06:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90e7e00640 1 -- 172.21.15.86:0/148624988 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90e0005020 con 0x7f90f0102f10 2024-04-25T19:06:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d76d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d81650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d83f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560515cdc900 con 0x560515d769b0 2024-04-25T19:06:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560515c82ef0 con 0x560515d772d0 2024-04-25T19:06:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90f95da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560515cd88c0 con 0x560515d81b90 2024-04-25T19:06:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d76d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d76d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51072/0 (socket says 172.21.15.86:51072) 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90efe00640 1 -- 172.21.15.86:0/644184676 learned_addr learned my addr 172.21.15.86:0/644184676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90ef400640 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d81650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.024+0000 7f90eea00640 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d83f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90efe00640 1 -- 172.21.15.86:0/644184676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 msgr2=0x560515d83f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90efe00640 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d83f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90efe00640 1 -- 172.21.15.86:0/644184676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 msgr2=0x560515d81650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90efe00640 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d81650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90efe00640 1 -- 172.21.15.86:0/644184676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605159ef210 con 0x560515d769b0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90eea00640 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d83f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90efe00640 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d76d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f60 tx=0x7f90e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f931356a369e0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90e4c00640 1 -- 172.21.15.86:0/644184676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e801e070 con 0x560515d769b0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.025+0000 7f90e4c00640 1 -- 172.21.15.86:0/644184676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e8003c30 con 0x560515d769b0 2024-04-25T19:06:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90e4c00640 1 -- 172.21.15.86:0/644184676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800f840 con 0x560515d769b0 2024-04-25T19:06:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90e7e00640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f90d40a3d10 0x7f90d40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90e7e00640 1 -- 172.21.15.86:0/148624988 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f90e00d0b30 con 0x7f90f0102f10 2024-04-25T19:06:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90ef400640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f90d40a3d10 0x7f90d40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 -- 172.21.15.86:0/644184676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 msgr2=0x560515d76d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d76d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f60 tx=0x7f90e8009f90 comp rx=0 tx=0).stop 2024-04-25T19:06:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 -- 172.21.15.86:0/644184676 shutdown_connections 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d83f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 --2- 172.21.15.86:0/644184676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d76d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.026+0000 7f90f95da740 1 -- 172.21.15.86:0/644184676 >> 172.21.15.86:0/644184676 conn(0x560515d72ec0 msgr2=0x560515d732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.027+0000 7f90f95da740 1 -- 172.21.15.86:0/644184676 shutdown_connections 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.027+0000 7f90ef400640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f90d40a3d10 0x7f90d40a61d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f90e00ce2e0 tx=0x7f90e00d4680 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.027+0000 7f90f95da740 1 -- 172.21.15.86:0/644184676 wait complete. 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.027+0000 7f90f95da740 1 -- start start 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.027+0000 7f90f95da740 1 Processor -- start 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d7ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d7f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d7f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560515d847c0 con 0x560515d769b0 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560515c96310 con 0x560515d772d0 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560515d84940 con 0x560515d81b90 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90f95da740 10 client.0 _ll_register_callbacks cb 0x560515c64cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d7ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d7ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51080/0 (socket says 172.21.15.86:51080) 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90efe00640 1 -- 172.21.15.86:0/323996107 learned_addr learned my addr 172.21.15.86:0/323996107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90eea00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d7f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90efe00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d7f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90efe00640 1 -- 172.21.15.86:0/323996107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 msgr2=0x560515d7f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.028+0000 7f90efe00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d7f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90efe00640 1 -- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 msgr2=0x560515d7f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90efe00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d7f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90efe00640 1 -- 172.21.15.86:0/323996107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560515abb620 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90efe00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d7ead0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f90e8003af0 tx=0x7f90e800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5ba085aba31194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90d9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90d9600640 1 -- 172.21.15.86:0/323996107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e802d060 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e8003c30 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90d9600640 1 -- 172.21.15.86:0/323996107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f90e8004460 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90d9600640 1 -- 172.21.15.86:0/323996107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f90e801e070 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90f95da740 10 client.15600 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.029+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560515d909d0 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90d9600640 1 -- 172.21.15.86:0/323996107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e8003c30 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90d9600640 1 -- 172.21.15.86:0/323996107 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f90e8019430 con 0x560515d769b0 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90f95da740 10 client.15600 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90f95da740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90f95da740 20 client.15600 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90f95da740 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515d91de0 0x560515d941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90f95da740 4 client.15600 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.030+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560515cf08b0 con 0x560515d91de0 2024-04-25T19:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.031+0000 7f90eea00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515d91de0 0x560515d941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.031+0000 7f90eea00640 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515d91de0 0x560515d941d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e61f381e7ef9fe server_cookie=b5a1e9645b3eac0d in_seq=0 out_seq=0 2024-04-25T19:06:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.031+0000 7f90d9600640 10 client.15600 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90d9600640 1 -- 172.21.15.86:0/323996107 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560515cf08b0 con 0x560515d91de0 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90d9600640 10 client.15600 handle_command_reply: tid=0 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 client.15600 shutdown 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 msgr2=0x560515d7ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d7ead0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f90e8003af0 tx=0x7f90e800f520 comp rx=0 tx=0).stop 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 shutdown_connections 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515d91de0 0x560515d941d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d81b90 0x560515d7f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d772d0 0x560515d7f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90d9600640 0 client.15600 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 --2- 172.21.15.86:0/323996107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d7ead0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.032+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 >> 172.21.15.86:0/323996107 conn(0x560515d72ec0 msgr2=0x560515d78c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.033+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 shutdown_connections 2024-04-25T19:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.033+0000 7f90f95da740 1 -- 172.21.15.86:0/323996107 wait complete. 2024-04-25T19:06:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.033+0000 7f90f95da740 20 client.15600 trim_cache size 0 max 16384 2024-04-25T19:06:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.061+0000 7f90f95da740 1 Processor -- start 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 -- start start 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d8d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d772d0 0x560515d78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515d998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01afe40 con 0x560515d769b0 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b1980 con 0x560515d81b90 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90f95da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b0510 con 0x560515d772d0 2024-04-25T19:06:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d8d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d8d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51088/0 (socket says 172.21.15.86:51088) 2024-04-25T19:06:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90efe00640 1 -- 172.21.15.86:0/991617693 learned_addr learned my addr 172.21.15.86:0/991617693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.062+0000 7f90eea00640 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d772d0 0x560515d78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90efe00640 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515d998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90eea00640 1 -- 172.21.15.86:0/991617693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 msgr2=0x560515d998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90eea00640 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515d998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90eea00640 1 -- 172.21.15.86:0/991617693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 msgr2=0x560515d8d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90eea00640 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d8d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90eea00640 1 -- 172.21.15.86:0/991617693 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560515d73840 con 0x560515d772d0 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90efe00640 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d8d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.063+0000 7f90eea00640 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d772d0 0x560515d78a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009830 tx=0x7f90dc005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddc5fac66a94da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.064+0000 7f90dbe00640 1 -- 172.21.15.86:0/991617693 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc04e070 con 0x560515d772d0 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.064+0000 7f90dbe00640 1 -- 172.21.15.86:0/991617693 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90dc030e30 con 0x560515d772d0 2024-04-25T19:06:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.064+0000 7f90dbe00640 1 -- 172.21.15.86:0/991617693 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc049820 con 0x560515d772d0 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 -- 172.21.15.86:0/991617693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d772d0 msgr2=0x560515d78a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d772d0 0x560515d78a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009830 tx=0x7f90dc005c90 comp rx=0 tx=0).stop 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 -- 172.21.15.86:0/991617693 shutdown_connections 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515d998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d772d0 0x560515d78a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 --2- 172.21.15.86:0/991617693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d769b0 0x560515d8d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 -- 172.21.15.86:0/991617693 >> 172.21.15.86:0/991617693 conn(0x560515d72ec0 msgr2=0x560515d78360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 -- 172.21.15.86:0/991617693 shutdown_connections 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 -- 172.21.15.86:0/991617693 wait complete. 2024-04-25T19:06:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.065+0000 7f90f95da740 1 -- start start 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 Processor -- start 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d769b0 0x560515d86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 0x560515d872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515df6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b12b0 con 0x560515d772d0 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b0be0 con 0x560515d81b90 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ade20 con 0x560515d769b0 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90f95da740 10 client.0 _ll_register_callbacks cb 0x7f90e80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90eea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 0x560515d872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.066+0000 7f90efe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d769b0 0x560515d86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90eea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 0x560515d872f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51098/0 (socket says 172.21.15.86:51098) 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90eea00640 1 -- 172.21.15.86:0/401792294 learned_addr learned my addr 172.21.15.86:0/401792294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515df6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 -- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 msgr2=0x560515df6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515df6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 -- 172.21.15.86:0/401792294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 msgr2=0x560515d872f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 0x560515d872f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 -- 172.21.15.86:0/401792294 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560515df6770 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90eea00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 0x560515d872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90efe00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d769b0 0x560515d86db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009560 tx=0x7f90e8005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbd64da415b0befb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90d9600640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.067+0000 7f90d9600640 1 -- 172.21.15.86:0/401792294 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e802d070 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90dc030e30 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90d9600640 1 -- 172.21.15.86:0/401792294 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f90e8004470 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90d9600640 1 -- 172.21.15.86:0/401792294 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f90e8004670 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 10 client.25699 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560515df7a80 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90d9600640 1 -- 172.21.15.86:0/401792294 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90dc030e30 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90d9600640 1 -- 172.21.15.86:0/401792294 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f90e8046480 con 0x560515d769b0 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 10 client.25699 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 20 client.25699 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515df80c0 0x560515dfa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 4 client.25699 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.068+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560515cea320 con 0x560515df80c0 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.069+0000 7f90eea00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515df80c0 0x560515dfa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.069+0000 7f90eea00640 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515df80c0 0x560515dfa4e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3bdf7300a722f7 server_cookie=d202a15e8d5968b9 in_seq=0 out_seq=0 2024-04-25T19:06:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.069+0000 7f90d9600640 10 client.25699 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90d9600640 1 -- 172.21.15.86:0/401792294 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560515cea320 con 0x560515df80c0 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90d9600640 10 client.25699 handle_command_reply: tid=0 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 client.25699 shutdown 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d769b0 msgr2=0x560515d86db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d769b0 0x560515d86db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009560 tx=0x7f90e8005bd0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 shutdown_connections 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560515df80c0 0x560515dfa4e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560515d81b90 0x560515df6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560515d772d0 0x560515d872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 --2- 172.21.15.86:0/401792294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560515d769b0 0x560515d86db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90d9600640 0 client.25699 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 >> 172.21.15.86:0/401792294 conn(0x560515d72ec0 msgr2=0x560515d7f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 shutdown_connections 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 1 -- 172.21.15.86:0/401792294 wait complete. 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.079+0000 7f90f95da740 20 client.25699 trim_cache size 0 max 16384 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:06:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f90d40a3d10 msgr2=0x7f90d40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f90d40a3d10 0x7f90d40a61d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f90e00ce2e0 tx=0x7f90e00d4680 comp rx=0 tx=0).stop 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 msgr2=0x7f90f01a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 0x7f90f01a0570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90e000e9f0 tx=0x7f90e000edf0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 shutdown_connections 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f90d40a3d10 0x7f90d40a61d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f010c330 0x7f90f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0103830 0x7f90f01a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 --2- 172.21.15.86:0/148624988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f0102f10 0x7f90f01a0570 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 >> 172.21.15.86:0/148624988 conn(0x7f90f00fc770 msgr2=0x7f90f0101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 shutdown_connections 2024-04-25T19:06:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:18.080+0000 7f90f5200640 1 -- 172.21.15.86:0/148624988 wait complete. 2024-04-25T19:06:18.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:18 smithi086 ceph-mon[27783]: pgmap v103: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:06:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:18 smithi194 ceph-mon[32740]: pgmap v103: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:06:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:18 smithi183 ceph-mon[32344]: pgmap v103: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:06:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:20 smithi194 ceph-mon[32740]: pgmap v104: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2024-04-25T19:06:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:20 smithi086 ceph-mon[27783]: pgmap v104: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2024-04-25T19:06:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:20 smithi183 ceph-mon[32344]: pgmap v104: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2024-04-25T19:06:22.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:22 smithi194 ceph-mon[32740]: pgmap v105: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:06:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:22 smithi086 ceph-mon[27783]: pgmap v105: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:06:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:22 smithi183 ceph-mon[32344]: pgmap v105: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:06:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:24 smithi194 ceph-mon[32740]: pgmap v106: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T19:06:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:24 smithi086 ceph-mon[27783]: pgmap v106: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T19:06:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:24 smithi183 ceph-mon[32344]: pgmap v106: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T19:06:26.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:06:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:26 smithi194 ceph-mon[32740]: pgmap v107: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:26.743 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:06:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:26 smithi086 ceph-mon[27783]: pgmap v107: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:26 smithi183 ceph-mon[32344]: pgmap v107: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:28.643 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:28 smithi086 ceph-mon[27783]: pgmap v108: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:28 smithi194 ceph-mon[32740]: pgmap v108: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:28 smithi183 ceph-mon[32344]: pgmap v108: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 -- 172.21.15.86:0/3989843016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806a750 msgr2=0x7f78b806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3989843016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806a750 0x7f78b806abb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f78a40098f0 tx=0x7f78a4009c00 comp rx=0 tx=0).stop 2024-04-25T19:06:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 -- 172.21.15.86:0/3989843016 shutdown_connections 2024-04-25T19:06:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3989843016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806b0f0 0x7f78b810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3989843016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78b8108a30 0x7f78b8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3989843016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806a750 0x7f78b806abb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 -- 172.21.15.86:0/3989843016 >> 172.21.15.86:0/3989843016 conn(0x7f78b80788f0 msgr2=0x7f78b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 -- 172.21.15.86:0/3989843016 shutdown_connections 2024-04-25T19:06:28.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.974+0000 7f78bfe00700 1 -- 172.21.15.86:0/3989843016 wait complete. 2024-04-25T19:06:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.975+0000 7f78bfe00700 1 Processor -- start 2024-04-25T19:06:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.975+0000 7f78bfe00700 1 -- start start 2024-04-25T19:06:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 0x7f78b8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806b0f0 0x7f78b8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78b8108a30 0x7f78b819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bfe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8112c40 con 0x7f78b8108a30 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bfe00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8112ac0 con 0x7f78b806a750 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bfe00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8112dc0 con 0x7f78b806b0f0 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bd400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 0x7f78b8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806b0f0 0x7f78b8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bd400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 0x7f78b8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58852/0 (socket says 172.21.15.86:58852) 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bd400700 1 -- 172.21.15.86:0/3414202272 learned_addr learned my addr 172.21.15.86:0/3414202272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bde00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78b8108a30 0x7f78b819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bca00700 1 -- 172.21.15.86:0/3414202272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 msgr2=0x7f78b8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bca00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 0x7f78b8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bca00700 1 -- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78b8108a30 msgr2=0x7f78b819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bca00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78b8108a30 0x7f78b819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.976+0000 7f78bca00700 1 -- 172.21.15.86:0/3414202272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b819e1f0 con 0x7f78b806b0f0 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78bd400700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 0x7f78b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78bca00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806b0f0 0x7f78b8199770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c010 tx=0x7f78a400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a40050c0 con 0x7f78b806b0f0 2024-04-25T19:06:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b819e480 con 0x7f78b806b0f0 2024-04-25T19:06:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81a5d60 con 0x7f78b806b0f0 2024-04-25T19:06:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a400b560 con 0x7f78b806b0f0 2024-04-25T19:06:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.977+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4003a00 con 0x7f78b806b0f0 2024-04-25T19:06:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.978+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7898005360 con 0x7f78b806b0f0 2024-04-25T19:06:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.981+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78a4003ba0 con 0x7f78b806b0f0 2024-04-25T19:06:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.982+0000 7f78b6000700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f78a00a3e30 0x7f78a00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.982+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f78a40d1780 con 0x7f78b806b0f0 2024-04-25T19:06:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.982+0000 7f78bd400700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f78a00a3e30 0x7f78a00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.983+0000 7f78bd400700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f78a00a3e30 0x7f78a00a62f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009750 tx=0x7f78ac009310 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:28.985+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78a4013030 con 0x7f78b806b0f0 2024-04-25T19:06:29.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.257+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7898005650 con 0x7f78b806b0f0 2024-04-25T19:06:29.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.258+0000 7f78b6000700 1 -- 172.21.15.86:0/3414202272 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f78a4021380 con 0x7f78b806b0f0 2024-04-25T19:06:29.260 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:06:29.260 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:06:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.261+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f78a00a3e30 msgr2=0x7f78a00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f78a00a3e30 0x7f78a00a62f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009750 tx=0x7f78ac009310 comp rx=0 tx=0).stop 2024-04-25T19:06:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806b0f0 msgr2=0x7f78b8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:29.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806b0f0 0x7f78b8199770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c010 tx=0x7f78a400fa00 comp rx=0 tx=0).stop 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 shutdown_connections 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78b8108a30 0x7f78b819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b806a750 0x7f78b8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f78a00a3e30 0x7f78a00a62f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 --2- 172.21.15.86:0/3414202272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78b806b0f0 0x7f78b8199770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 >> 172.21.15.86:0/3414202272 conn(0x7f78b80788f0 msgr2=0x7f78b80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 shutdown_connections 2024-04-25T19:06:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:29.262+0000 7f78bfe00700 1 -- 172.21.15.86:0/3414202272 wait complete. 2024-04-25T19:06:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:06:29.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:29 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3414202272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:06:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:29 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3414202272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:06:29.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:29 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3414202272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:06:30.164 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id ff9b5964 --release --await' 2024-04-25T19:06:30.165 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id ff9b5964 --release --await 2024-04-25T19:06:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:30 smithi086 ceph-mon[27783]: pgmap v109: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T19:06:30.592 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:06:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:30 smithi194 ceph-mon[32740]: pgmap v109: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T19:06:30.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:30 smithi183 ceph-mon[32344]: pgmap v109: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T19:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 -- 172.21.15.86:0/17572519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103200 msgr2=0x7f7c241035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/17572519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103200 0x7f7c241035e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009860 tx=0x7f7c1402a3d0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 -- 172.21.15.86:0/17572519 shutdown_connections 2024-04-25T19:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/17572519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c2410c5d0 0x7f7c2410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/17572519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c24103b20 0x7f7c2410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/17572519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103200 0x7f7c241035e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 -- 172.21.15.86:0/17572519 >> 172.21.15.86:0/17572519 conn(0x7f7c240771d0 msgr2=0x7f7c240775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 -- 172.21.15.86:0/17572519 shutdown_connections 2024-04-25T19:06:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.213+0000 7f7c2ac00640 1 -- 172.21.15.86:0/17572519 wait complete. 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 Processor -- start 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 -- start start 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c24103200 0x7f7c24100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 0x7f7c24101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2410c5d0 0x7f7c241018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24112250 con 0x7f7c24103b20 2024-04-25T19:06:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241120d0 con 0x7f7c2410c5d0 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.214+0000 7f7c2ac00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241123d0 con 0x7f7c24103200 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 0x7f7c24101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 0x7f7c24101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40592/0 (socket says 172.21.15.86:40592) 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 -- 172.21.15.86:0/2319981233 learned_addr learned my addr 172.21.15.86:0/2319981233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23e00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c24103200 0x7f7c24100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c28c00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2410c5d0 0x7f7c241018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 -- 172.21.15.86:0/2319981233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c24103200 msgr2=0x7f7c24100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c24103200 0x7f7c24100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 -- 172.21.15.86:0/2319981233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2410c5d0 msgr2=0x7f7c241018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2410c5d0 0x7f7c241018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 -- 172.21.15.86:0/2319981233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c241a6b70 con 0x7f7c24103b20 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23e00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c24103200 0x7f7c24100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c28c00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2410c5d0 0x7f7c241018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.215+0000 7f7c23400640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 0x7f7c24101390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000e9f0 tx=0x7f7c1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.216+0000 7f7c20c00640 1 -- 172.21.15.86:0/2319981233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000cd80 con 0x7f7c24103b20 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.216+0000 7f7c20c00640 1 -- 172.21.15.86:0/2319981233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c100108e0 con 0x7f7c24103b20 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.216+0000 7f7c20c00640 1 -- 172.21.15.86:0/2319981233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10018980 con 0x7f7c24103b20 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.216+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c241a6e60 con 0x7f7c24103b20 2024-04-25T19:06:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.216+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241a7370 con 0x7f7c24103b20 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.219+0000 7f7c20c00640 1 -- 172.21.15.86:0/2319981233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c10010430 con 0x7f7c24103b20 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.219+0000 7f7c2da27740 1 Processor -- start 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.219+0000 7f7c2da27740 1 -- start start 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf54820 0x562b8bf54c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 0x562b8bf5f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf61df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c2da27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b8beba900 con 0x562b8bf54820 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c2da27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b8be60ef0 con 0x562b8bf5fa00 2024-04-25T19:06:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c2da27740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b8beb68c0 con 0x562b8bf55140 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c28c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf54820 0x562b8bf54c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 0x562b8bf5f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 0x562b8bf5f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48354/0 (socket says 172.21.15.86:48354) 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 -- 172.21.15.86:0/1510485832 learned_addr learned my addr 172.21.15.86:0/1510485832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23400640 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf61df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 -- 172.21.15.86:0/1510485832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 msgr2=0x562b8bf61df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf61df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 -- 172.21.15.86:0/1510485832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf54820 msgr2=0x562b8bf54c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf54820 0x562b8bf54c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.220+0000 7f7c23e00640 1 -- 172.21.15.86:0/1510485832 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8bbcd210 con 0x562b8bf55140 2024-04-25T19:06:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c23e00640 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 0x562b8bf5f4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009860 tx=0x7f7c14030640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582b6cfd35549ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c20c00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7c080a3e70 0x7f7c080a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c20c00640 1 -- 172.21.15.86:0/2319981233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7c100d0c70 con 0x7f7c24103b20 2024-04-25T19:06:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c28c00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7c080a3e70 0x7f7c080a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c19200640 1 -- 172.21.15.86:0/1510485832 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14005c90 con 0x562b8bf55140 2024-04-25T19:06:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c19200640 1 -- 172.21.15.86:0/1510485832 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c14040070 con 0x562b8bf55140 2024-04-25T19:06:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.221+0000 7f7c19200640 1 -- 172.21.15.86:0/1510485832 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14004ac0 con 0x562b8bf55140 2024-04-25T19:06:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 -- 172.21.15.86:0/1510485832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 msgr2=0x562b8bf5f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c28c00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7c080a3e70 0x7f7c080a6330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c2406b860 tx=0x7f7c1c011040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 0x562b8bf5f4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009860 tx=0x7f7c14030640 comp rx=0 tx=0).stop 2024-04-25T19:06:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 -- 172.21.15.86:0/1510485832 shutdown_connections 2024-04-25T19:06:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf61df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf55140 0x562b8bf5f4c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 --2- 172.21.15.86:0/1510485832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf54820 0x562b8bf54c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 -- 172.21.15.86:0/1510485832 >> 172.21.15.86:0/1510485832 conn(0x562b8bf50d30 msgr2=0x562b8bf51110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 -- 172.21.15.86:0/1510485832 shutdown_connections 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 -- 172.21.15.86:0/1510485832 wait complete. 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.222+0000 7f7c2da27740 1 -- start start 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 Processor -- start 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf5c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf5ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b8bf625d0 con 0x562b8bf55140 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b8be74310 con 0x562b8bf5fa00 2024-04-25T19:06:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b8bf62750 con 0x562b8bf54820 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c2da27740 10 client.0 _ll_register_callbacks cb 0x562b8be42cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf5ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf5ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40614/0 (socket says 172.21.15.86:40614) 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.223+0000 7f7c23400640 1 -- 172.21.15.86:0/388519143 learned_addr learned my addr 172.21.15.86:0/388519143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23e00640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf5c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c28c00640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23400640 1 -- 172.21.15.86:0/388519143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 msgr2=0x562b8bf5c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23400640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf5c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23400640 1 -- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 msgr2=0x562b8bf5d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23400640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23400640 1 -- 172.21.15.86:0/388519143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8bc99620 con 0x562b8bf55140 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23e00640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf5c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c23400640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf5ce00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7c10000c00 tx=0x7f7c1009c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bc632a1136f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c0d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c0d600640 1 -- 172.21.15.86:0/388519143 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1009d260 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c0d600640 1 -- 172.21.15.86:0/388519143 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7c100991d0 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c14040070 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c0d600640 1 -- 172.21.15.86:0/388519143 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c1009cb60 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c2da27740 10 client.15624 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.224+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b8bf6e7f0 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c0d600640 1 -- 172.21.15.86:0/388519143 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c14040070 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c0d600640 1 -- 172.21.15.86:0/388519143 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7c10018b20 con 0x562b8bf55140 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c2da27740 10 client.15624 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c2da27740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c2da27740 20 client.15624 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c2da27740 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bf6fcb0 0x562b8bf720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c2da27740 4 client.15624 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b8bece8b0 con 0x562b8bf6fcb0 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.225+0000 7f7c23e00640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bf6fcb0 0x562b8bf720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.226+0000 7f7c23e00640 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bf6fcb0 0x562b8bf720e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad8cf28f4b0a9b3 server_cookie=15797bbcdf6fd350 in_seq=0 out_seq=0 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.226+0000 7f7c0d600640 10 client.15624 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c0d600640 1 -- 172.21.15.86:0/388519143 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b8bece8b0 con 0x562b8bf6fcb0 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c0d600640 10 client.15624 handle_command_reply: tid=0 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 client.15624 shutdown 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 msgr2=0x562b8bf5ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf5ce00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7c10000c00 tx=0x7f7c1009c9f0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 shutdown_connections 2024-04-25T19:06:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bf6fcb0 0x562b8bf720e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c0d600640 0 client.15624 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf5ce00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 --2- 172.21.15.86:0/388519143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf5c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.227+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 >> 172.21.15.86:0/388519143 conn(0x562b8bf50d30 msgr2=0x562b8bf56ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.228+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 shutdown_connections 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.228+0000 7f7c2da27740 1 -- 172.21.15.86:0/388519143 wait complete. 2024-04-25T19:06:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.228+0000 7f7c2da27740 20 client.15624 trim_cache size 0 max 16384 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.243+0000 7f7c2da27740 1 Processor -- start 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.243+0000 7f7c2da27740 1 -- start start 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf6b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c2da27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241afe30 con 0x562b8bf55140 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c2da27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241b1970 con 0x562b8bf5fa00 2024-04-25T19:06:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c2da27740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241b0500 con 0x562b8bf54820 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40620/0 (socket says 172.21.15.86:40620) 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c23400640 1 -- 172.21.15.86:0/2512944885 learned_addr learned my addr 172.21.15.86:0/2512944885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c28c00640 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.244+0000 7f7c23e00640 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf6b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c23400640 1 -- 172.21.15.86:0/2512944885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 msgr2=0x562b8bf6b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c23400640 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf6b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c23400640 1 -- 172.21.15.86:0/2512944885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 msgr2=0x562b8bf5cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c23400640 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c23400640 1 -- 172.21.15.86:0/2512944885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8bf54f70 con 0x562b8bf55140 2024-04-25T19:06:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c23400640 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf571e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7c1009caf0 tx=0x7f7c1009ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149ad7f331445dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c0fe00640 1 -- 172.21.15.86:0/2512944885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1009d260 con 0x562b8bf55140 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c0fe00640 1 -- 172.21.15.86:0/2512944885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c100a0f10 con 0x562b8bf55140 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.245+0000 7f7c0fe00640 1 -- 172.21.15.86:0/2512944885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c100988b0 con 0x562b8bf55140 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 -- 172.21.15.86:0/2512944885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 msgr2=0x562b8bf571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf571e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7c1009caf0 tx=0x7f7c1009ca60 comp rx=0 tx=0).stop 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 -- 172.21.15.86:0/2512944885 shutdown_connections 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf5cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf571e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 --2- 172.21.15.86:0/2512944885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf6b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 -- 172.21.15.86:0/2512944885 >> 172.21.15.86:0/2512944885 conn(0x562b8bf50d30 msgr2=0x562b8bf5c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 -- 172.21.15.86:0/2512944885 shutdown_connections 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 -- 172.21.15.86:0/2512944885 wait complete. 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 -- start start 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.246+0000 7f7c2da27740 1 Processor -- start 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf67eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf65240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf65980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241b12a0 con 0x562b8bf55140 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241b0bd0 con 0x562b8bf5fa00 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241ade10 con 0x562b8bf54820 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c2da27740 10 client.0 _ll_register_callbacks cb 0x7f7c10098e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf65240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf65240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40636/0 (socket says 172.21.15.86:40636) 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c23400640 1 -- 172.21.15.86:0/476676430 learned_addr learned my addr 172.21.15.86:0/476676430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c23e00640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.247+0000 7f7c28c00640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf65980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c23400640 1 -- 172.21.15.86:0/476676430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 msgr2=0x562b8bf67eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c23400640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf67eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c23400640 1 -- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 msgr2=0x562b8bf65980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c23400640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf65980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c23400640 1 -- 172.21.15.86:0/476676430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8bfd5270 con 0x562b8bf55140 2024-04-25T19:06:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c28c00640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf65980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c23400640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf65240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c10098fc0 tx=0x7f7c100ce3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7919785409f188be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c0d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c0d600640 1 -- 172.21.15.86:0/476676430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1009d260 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c0d600640 1 -- 172.21.15.86:0/476676430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7c10018b20 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c100a0f10 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c0d600640 1 -- 172.21.15.86:0/476676430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c100a7080 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c2da27740 10 client.15630 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.248+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b8bfd63c0 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c0d600640 1 -- 172.21.15.86:0/476676430 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c100a0f10 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c0d600640 1 -- 172.21.15.86:0/476676430 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7c100988b0 con 0x562b8bf55140 2024-04-25T19:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c2da27740 10 client.15630 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c2da27740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c2da27740 20 client.15630 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c2da27740 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bfd6950 0x562b8bfd8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c2da27740 4 client.15630 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b8bec8320 con 0x562b8bfd6950 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.249+0000 7f7c23e00640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bfd6950 0x562b8bfd8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.250+0000 7f7c23e00640 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bfd6950 0x562b8bfd8d70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96b23896c2b7439 server_cookie=27cda31d3be9a982 in_seq=0 out_seq=0 2024-04-25T19:06:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.250+0000 7f7c0d600640 10 client.15630 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c0d600640 1 -- 172.21.15.86:0/476676430 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b8bec8320 con 0x562b8bfd6950 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c0d600640 10 client.15630 handle_command_reply: tid=0 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 client.15630 shutdown 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 msgr2=0x562b8bf65240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf65240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c10098fc0 tx=0x7f7c100ce3f0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 shutdown_connections 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562b8bfd6950 0x562b8bfd8d70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8bf5fa00 0x562b8bf65980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8bf55140 0x562b8bf65240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c0d600640 0 client.15630 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 --2- 172.21.15.86:0/476676430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b8bf54820 0x562b8bf67eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 >> 172.21.15.86:0/476676430 conn(0x562b8bf50d30 msgr2=0x562b8bf4fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 shutdown_connections 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 1 -- 172.21.15.86:0/476676430 wait complete. 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.257+0000 7f7c2da27740 20 client.15630 trim_cache size 0 max 16384 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:06:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:06:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7c080a3e70 msgr2=0x7f7c080a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7c080a3e70 0x7f7c080a6330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c2406b860 tx=0x7f7c1c011040 comp rx=0 tx=0).stop 2024-04-25T19:06:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 msgr2=0x7f7c24101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 0x7f7c24101390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000e9f0 tx=0x7f7c1000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 shutdown_connections 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7c080a3e70 0x7f7c080a6330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2410c5d0 0x7f7c241018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24103b20 0x7f7c24101390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 --2- 172.21.15.86:0/2319981233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c24103200 0x7f7c24100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 >> 172.21.15.86:0/2319981233 conn(0x7f7c240771d0 msgr2=0x7f7c240fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 shutdown_connections 2024-04-25T19:06:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:31.258+0000 7f7c2ac00640 1 -- 172.21.15.86:0/2319981233 wait complete. 2024-04-25T19:06:32.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:32 smithi086 ceph-mon[27783]: pgmap v110: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:32.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:32 smithi194 ceph-mon[32740]: pgmap v110: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.771+0000 7f4b9bc00700 1 -- 172.21.15.86:0/488075860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b9406b0f0 msgr2=0x7f4b9410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.771+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/488075860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b9406b0f0 0x7f4b9410f900 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009f90 tx=0x7f4b8c00b330 comp rx=0 tx=0).stop 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 -- 172.21.15.86:0/488075860 shutdown_connections 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/488075860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b9406b0f0 0x7f4b9410f900 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/488075860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b94108a30 0x7f4b94108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/488075860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b9406a750 0x7f4b9406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 -- 172.21.15.86:0/488075860 >> 172.21.15.86:0/488075860 conn(0x7f4b940788f0 msgr2=0x7f4b94078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:32.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 -- 172.21.15.86:0/488075860 shutdown_connections 2024-04-25T19:06:32.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.772+0000 7f4b9bc00700 1 -- 172.21.15.86:0/488075860 wait complete. 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 Processor -- start 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 -- start start 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b9406a750 0x7f4b94101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9406b0f0 0x7f4b94101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 0x7f4b94102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94112bd0 con 0x7f4b9406b0f0 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94112a50 con 0x7f4b94108a30 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.773+0000 7f4b9bc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94112d50 con 0x7f4b9406a750 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9406b0f0 0x7f4b94101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 0x7f4b94102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 0x7f4b94102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58228/0 (socket says 172.21.15.86:58228) 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b9406a750 0x7f4b94101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 -- 172.21.15.86:0/1396873767 learned_addr learned my addr 172.21.15.86:0/1396873767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 -- 172.21.15.86:0/1396873767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b9406a750 msgr2=0x7f4b94101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b9406a750 0x7f4b94101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 -- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9406b0f0 msgr2=0x7f4b94101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9406b0f0 0x7f4b94101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 -- 172.21.15.86:0/1396873767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b941a75e0 con 0x7f4b94108a30 2024-04-25T19:06:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.774+0000 7f4b99c00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 0x7f4b94102490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009790 tx=0x7f4b8c00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.775+0000 7f4b91600700 1 -- 172.21.15.86:0/1396873767 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c021070 con 0x7f4b94108a30 2024-04-25T19:06:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.775+0000 7f4b91600700 1 -- 172.21.15.86:0/1396873767 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c007ac0 con 0x7f4b94108a30 2024-04-25T19:06:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.775+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b941a7870 con 0x7f4b94108a30 2024-04-25T19:06:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.775+0000 7f4b91600700 1 -- 172.21.15.86:0/1396873767 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c005040 con 0x7f4b94108a30 2024-04-25T19:06:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.775+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941a7d20 con 0x7f4b94108a30 2024-04-25T19:06:32.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.778+0000 7f4b91600700 1 -- 172.21.15.86:0/1396873767 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b8c024070 con 0x7f4b94108a30 2024-04-25T19:06:32.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.780+0000 7f4ba6527b80 1 Processor -- start 2024-04-25T19:06:32.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.780+0000 7f4ba6527b80 1 -- start start 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d87d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b91600700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4b7c0a3db0 0x7f4b7c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563991cdaf90 con 0x563991d87920 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563991c81830 con 0x563991d88240 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563991cd6f50 con 0x563991d930f0 2024-04-25T19:06:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b91600700 1 -- 172.21.15.86:0/1396873767 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f4b8c00f020 con 0x7f4b94108a30 2024-04-25T19:06:32.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d87d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d87d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40650/0 (socket says 172.21.15.86:40650) 2024-04-25T19:06:32.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b93e00700 1 -- 172.21.15.86:0/872333015 learned_addr learned my addr 172.21.15.86:0/872333015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:32.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b99c00700 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b99200700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4b7c0a3db0 0x7f4b7c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.781+0000 7f4b93e00700 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d92bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99c00700 1 -- 172.21.15.86:0/872333015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 msgr2=0x563991d92bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99c00700 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d92bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99c00700 1 -- 172.21.15.86:0/872333015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 msgr2=0x563991d87d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99c00700 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d87d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99c00700 1 -- 172.21.15.86:0/872333015 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563991c77780 con 0x563991d930f0 2024-04-25T19:06:32.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b93e00700 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d92bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99c00700 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d954e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0cf2c0 tx=0x7f4b8c09d580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48d60870945c26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b86000700 1 -- 172.21.15.86:0/872333015 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0b2040 con 0x563991d930f0 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b99200700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4b7c0a3db0 0x7f4b7c0a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005fd0 tx=0x7f4b88009310 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b86000700 1 -- 172.21.15.86:0/872333015 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c09e2a0 con 0x563991d930f0 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.782+0000 7f4b86000700 1 -- 172.21.15.86:0/872333015 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c09bea0 con 0x563991d930f0 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 -- 172.21.15.86:0/872333015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 msgr2=0x563991d954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d954e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0cf2c0 tx=0x7f4b8c09d580 comp rx=0 tx=0).stop 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 -- 172.21.15.86:0/872333015 shutdown_connections 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d954e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d87d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/872333015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d92bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 -- 172.21.15.86:0/872333015 >> 172.21.15.86:0/872333015 conn(0x563991cb66a0 msgr2=0x563991d83dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:32.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 -- 172.21.15.86:0/872333015 shutdown_connections 2024-04-25T19:06:32.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.783+0000 7f4ba6527b80 1 -- 172.21.15.86:0/872333015 wait complete. 2024-04-25T19:06:32.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.784+0000 7f4ba6527b80 1 -- start start 2024-04-25T19:06:32.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.784+0000 7f4ba6527b80 1 Processor -- start 2024-04-25T19:06:32.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d87920 0x563991d927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 0x563991d8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563991d95a20 con 0x563991d88240 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563991c94960 con 0x563991d87920 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563991d95ba0 con 0x563991d930f0 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4ba6527b80 10 client.0 _ll_register_callbacks cb 0x563991c73820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 0x563991d8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4b99200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d87920 0x563991d927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 0x563991d8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40656/0 (socket says 172.21.15.86:40656) 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4b93e00700 1 -- 172.21.15.86:0/653665099 learned_addr learned my addr 172.21.15.86:0/653665099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:32.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.785+0000 7f4b99c00700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b93e00700 1 -- 172.21.15.86:0/653665099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 msgr2=0x563991d900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b93e00700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b93e00700 1 -- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d87920 msgr2=0x563991d927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b93e00700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d87920 0x563991d927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b93e00700 1 -- 172.21.15.86:0/653665099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563991c289d0 con 0x563991d88240 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b99200700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d87920 0x563991d927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b93e00700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 0x563991d8fb80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4b80012b80 tx=0x7f4b80012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364fc9e48761048f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b7aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b7aa00700 1 -- 172.21.15.86:0/653665099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80010cd0 con 0x563991d88240 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4b7aa00700 1 -- 172.21.15.86:0/653665099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f4b80004030 con 0x563991d88240 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.786+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b8c09e2a0 con 0x563991d88240 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4b7aa00700 1 -- 172.21.15.86:0/653665099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b80004230 con 0x563991d88240 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 10 client.15642 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563991c71bf0 con 0x563991d88240 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4b7aa00700 1 -- 172.21.15.86:0/653665099 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c09e2a0 con 0x563991d88240 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4b7aa00700 1 -- 172.21.15.86:0/653665099 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4b8001f480 con 0x563991d88240 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 10 client.15642 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 20 client.15642 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991da2620 0x563991da4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 4 client.15642 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.787+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563991ce8780 con 0x563991da2620 2024-04-25T19:06:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.788+0000 7f4b99200700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991da2620 0x563991da4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.788+0000 7f4b99200700 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991da2620 0x563991da4a10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade25af717fd6ff3 server_cookie=5f6cc480be0038c5 in_seq=0 out_seq=0 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.788+0000 7f4b7aa00700 10 client.15642 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.789+0000 7f4b7aa00700 1 -- 172.21.15.86:0/653665099 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563991ce8780 con 0x563991da2620 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.789+0000 7f4b7aa00700 10 client.15642 handle_command_reply: tid=0 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.789+0000 7f4ba6527b80 1 client.15642 shutdown 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.789+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 msgr2=0x563991d8fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.789+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 0x563991d8fb80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4b80012b80 tx=0x7f4b80012e90 comp rx=0 tx=0).stop 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 shutdown_connections 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991da2620 0x563991da4a10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d87920 0x563991d927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/653665099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d88240 0x563991d8fb80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 >> 172.21.15.86:0/653665099 conn(0x563991cb66a0 msgr2=0x563991d839c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:32.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4b7aa00700 0 client.15642 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:32.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 shutdown_connections 2024-04-25T19:06:32.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 1 -- 172.21.15.86:0/653665099 wait complete. 2024-04-25T19:06:32.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.790+0000 7f4ba6527b80 20 client.15642 trim_cache size 0 max 16384 2024-04-25T19:06:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 Processor -- start 2024-04-25T19:06:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 -- start start 2024-04-25T19:06:32.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d83b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d8b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941af560 con 0x563991d87920 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941b10a0 con 0x563991d88240 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941afc30 con 0x563991d930f0 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.811+0000 7f4b99200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d83b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d9e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58248/0 (socket says 172.21.15.86:58248) 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 -- 172.21.15.86:0/2108695742 learned_addr learned my addr 172.21.15.86:0/2108695742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b99c00700 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d8b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 -- 172.21.15.86:0/2108695742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 msgr2=0x563991d8b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d8b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 -- 172.21.15.86:0/2108695742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 msgr2=0x563991d83b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d83b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 -- 172.21.15.86:0/2108695742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563991d84a10 con 0x563991d88240 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.812+0000 7f4b93e00700 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d9e8e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b80013fd0 tx=0x7f4b800096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=418887064e90638d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.813+0000 7f4b85600700 1 -- 172.21.15.86:0/2108695742 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80009ea0 con 0x563991d88240 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.813+0000 7f4b85600700 1 -- 172.21.15.86:0/2108695742 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b80010710 con 0x563991d88240 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.813+0000 7f4b85600700 1 -- 172.21.15.86:0/2108695742 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8001fd20 con 0x563991d88240 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.813+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2108695742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 msgr2=0x563991d9e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.813+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d9e8e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b80013fd0 tx=0x7f4b800096d0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2108695742 shutdown_connections 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d930f0 0x563991d8b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d83b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2108695742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d88240 0x563991d9e8e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2108695742 >> 172.21.15.86:0/2108695742 conn(0x563991cb66a0 msgr2=0x563991d8aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2108695742 shutdown_connections 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2108695742 wait complete. 2024-04-25T19:06:32.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 -- start start 2024-04-25T19:06:32.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.814+0000 7f4ba6527b80 1 Processor -- start 2024-04-25T19:06:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d88240 0x563991d97dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d930f0 0x563991d98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941b09d0 con 0x563991d87920 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941b0300 con 0x563991d930f0 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941ad500 con 0x563991d88240 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4ba6527b80 10 client.0 _ll_register_callbacks cb 0x563991c73820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4b99200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4b99c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d930f0 0x563991d98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4b99200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d9afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40674/0 (socket says 172.21.15.86:40674) 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4b99200700 1 -- 172.21.15.86:0/2124186551 learned_addr learned my addr 172.21.15.86:0/2124186551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.815+0000 7f4b93e00700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d88240 0x563991d97dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99200700 1 -- 172.21.15.86:0/2124186551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d88240 msgr2=0x563991d97dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99200700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d88240 0x563991d97dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99200700 1 -- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d930f0 msgr2=0x563991d98300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99200700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d930f0 0x563991d98300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99200700 1 -- 172.21.15.86:0/2124186551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563991d988c0 con 0x563991d87920 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99c00700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d930f0 0x563991d98300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:32.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b99200700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d9afd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4b880044c0 tx=0x7f4b880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55155ba0863bcf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b7aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b7aa00700 1 -- 172.21.15.86:0/2124186551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8802b070 con 0x563991d87920 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4b7aa00700 1 -- 172.21.15.86:0/2124186551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f4b88003b10 con 0x563991d87920 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.816+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b80010710 con 0x563991d87920 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4b7aa00700 1 -- 172.21.15.86:0/2124186551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b88003d10 con 0x563991d87920 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 10 client.15648 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563991e09960 con 0x563991d87920 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4b7aa00700 1 -- 172.21.15.86:0/2124186551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b80010710 con 0x563991d87920 2024-04-25T19:06:32.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4b7aa00700 1 -- 172.21.15.86:0/2124186551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4b88027450 con 0x563991d87920 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 10 client.15648 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 20 client.15648 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991e09fc0 0x563991e0c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 4 client.15648 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ff9b5964", "release": true, "await": true}] 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.817+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ff9b5964", "release": true, "await": true}) -- 0x563991ce2820 con 0x563991e09fc0 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.818+0000 7f4b93e00700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991e09fc0 0x563991e0c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.818+0000 7f4b93e00700 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991e09fc0 0x563991e0c3e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8da40031ddb1651 server_cookie=119f485d1a12d59c in_seq=0 out_seq=0 2024-04-25T19:06:32.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.818+0000 7f4b7aa00700 10 client.15648 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:32.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.821+0000 7f4b7aa00700 1 -- 172.21.15.86:0/2124186551 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563991ce2820 con 0x563991e09fc0 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4b7aa00700 10 client.15648 handle_command_reply: tid=0 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 client.15648 shutdown 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 msgr2=0x563991d9afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d9afd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4b880044c0 tx=0x7f4b880047c0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 shutdown_connections 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563991e09fc0 0x563991e0c3e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563991d930f0 0x563991d98300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563991d87920 0x563991d9afd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 --2- 172.21.15.86:0/2124186551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563991d88240 0x563991d97dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 >> 172.21.15.86:0/2124186551 conn(0x563991cb66a0 msgr2=0x563991d91080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4b7aa00700 0 client.15648 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 shutdown_connections 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 1 -- 172.21.15.86:0/2124186551 wait complete. 2024-04-25T19:06:32.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.822+0000 7f4ba6527b80 20 client.15648 trim_cache size 0 max 16384 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 12, 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "ff9b5964": { 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 12, 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:06:32.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 90.0, 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:06:32.826 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:06:32.827 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:06:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4b7c0a3db0 msgr2=0x7f4b7c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4b7c0a3db0 0x7f4b7c0a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b88005fd0 tx=0x7f4b88009310 comp rx=0 tx=0).stop 2024-04-25T19:06:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 msgr2=0x7f4b94102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 0x7f4b94102490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009790 tx=0x7f4b8c00cc30 comp rx=0 tx=0).stop 2024-04-25T19:06:32.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 shutdown_connections 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b94108a30 0x7f4b94102490 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4b7c0a3db0 0x7f4b7c0a6270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b9406a750 0x7f4b94101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 --2- 172.21.15.86:0/1396873767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b9406b0f0 0x7f4b94101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 >> 172.21.15.86:0/1396873767 conn(0x7f4b940788f0 msgr2=0x7f4b940ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 shutdown_connections 2024-04-25T19:06:32.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:06:32.823+0000 7f4b9bc00700 1 -- 172.21.15.86:0/1396873767 wait complete. 2024-04-25T19:06:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:32 smithi183 ceph-mon[32344]: pgmap v110: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-25T19:06:33.814 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ff9b5964', seconds elapsed: 45.2 2024-04-25T19:06:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:34 smithi194 ceph-mon[32740]: pgmap v111: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-04-25T19:06:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:34 smithi086 ceph-mon[27783]: pgmap v111: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-04-25T19:06:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:34 smithi183 ceph-mon[32344]: pgmap v111: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-04-25T19:06:36.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:36 smithi086 ceph-mon[27783]: pgmap v112: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-25T19:06:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:36 smithi183 ceph-mon[32344]: pgmap v112: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-25T19:06:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:36 smithi194 ceph-mon[32740]: pgmap v112: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-25T19:06:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:38 smithi086 ceph-mon[27783]: pgmap v113: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-25T19:06:38.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:38 smithi183 ceph-mon[32344]: pgmap v113: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-25T19:06:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:38 smithi194 ceph-mon[32740]: pgmap v113: 97 pgs: 97 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-25T19:06:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:40 smithi086 ceph-mon[27783]: pgmap v114: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-25T19:06:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:40 smithi183 ceph-mon[32344]: pgmap v114: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-25T19:06:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:40 smithi194 ceph-mon[32740]: pgmap v114: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-25T19:06:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:42 smithi086 ceph-mon[27783]: pgmap v115: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T19:06:42.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:42 smithi183 ceph-mon[32344]: pgmap v115: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T19:06:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:42 smithi194 ceph-mon[32740]: pgmap v115: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T19:06:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 -- 172.21.15.86:0/2079552959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108560 msgr2=0x7f77a81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 --2- 172.21.15.86:0/2079552959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108560 0x7f77a81089e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7790009930 tx=0x7f779002a420 comp rx=0 tx=0).stop 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 -- 172.21.15.86:0/2079552959 shutdown_connections 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 --2- 172.21.15.86:0/2079552959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a8108f20 0x7f77a81153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 --2- 172.21.15.86:0/2079552959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108560 0x7f77a81089e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 --2- 172.21.15.86:0/2079552959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a810e620 0x7f77a810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 -- 172.21.15.86:0/2079552959 >> 172.21.15.86:0/2079552959 conn(0x7f77a80764e0 msgr2=0x7f77a8078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 -- 172.21.15.86:0/2079552959 shutdown_connections 2024-04-25T19:06:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.398+0000 7f77acc00640 1 -- 172.21.15.86:0/2079552959 wait complete. 2024-04-25T19:06:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.399+0000 7f77acc00640 1 Processor -- start 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.399+0000 7f77acc00640 1 -- start start 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77acc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 0x7f77a81a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77acc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108f20 0x7f77a81a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a810e620 0x7f77a819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77acc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8118800 con 0x7f77a810e620 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77acc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8118680 con 0x7f77a8108560 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77acc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8118980 con 0x7f77a8108f20 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a810e620 0x7f77a819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 0x7f77a81a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f779d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108f20 0x7f77a81a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 0x7f77a81a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:47242/0 (socket says 172.21.15.86:47242) 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 -- 172.21.15.86:0/2877657630 learned_addr learned my addr 172.21.15.86:0/2877657630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f779d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108f20 0x7f77a81a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:38540/0 (socket says 172.21.15.86:38540) 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 -- 172.21.15.86:0/2877657630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108f20 msgr2=0x7f77a81a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108f20 0x7f77a81a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 -- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a810e620 msgr2=0x7f77a819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a810e620 0x7f77a819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.400+0000 7f77a5e00640 1 -- 172.21.15.86:0/2877657630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a819f1e0 con 0x7f77a8108560 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.401+0000 7f77a5e00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 0x7f77a81a48b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f779800e9f0 tx=0x7f779800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.401+0000 7f779f400640 1 -- 172.21.15.86:0/2877657630 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779800cd80 con 0x7f77a8108560 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.401+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a819f4d0 con 0x7f77a8108560 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.401+0000 7f779f400640 1 -- 172.21.15.86:0/2877657630 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77980054d0 con 0x7f77a8108560 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.401+0000 7f779f400640 1 -- 172.21.15.86:0/2877657630 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798010690 con 0x7f77a8108560 2024-04-25T19:06:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.401+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81ab770 con 0x7f77a8108560 2024-04-25T19:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.404+0000 7f77afa35740 1 Processor -- start 2024-04-25T19:06:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.404+0000 7f77afa35740 1 -- start start 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f779f400640 1 -- 172.21.15.86:0/2877657630 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7798004e90 con 0x7f77a8108560 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77afa35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 0x561707dd7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77afa35740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707de24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77afa35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707de4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77afa35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561707d3d900 con 0x561707dd7820 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77afa35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561707ce3ef0 con 0x561707de2a00 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77afa35740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561707d398c0 con 0x561707dd8140 2024-04-25T19:06:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 0x561707dd7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 0x561707dd7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53370/0 (socket says 172.21.15.86:53370) 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77a6800640 1 -- 172.21.15.86:0/1510075137 learned_addr learned my addr 172.21.15.86:0/1510075137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f77a5e00640 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707de24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.405+0000 7f779d400640 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707de4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f77a6800640 1 -- 172.21.15.86:0/1510075137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 msgr2=0x561707de24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f77a6800640 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707de24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f77a6800640 1 -- 172.21.15.86:0/1510075137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 msgr2=0x561707de4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f77a6800640 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707de4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f77a6800640 1 -- 172.21.15.86:0/1510075137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561707a50210 con 0x561707dd7820 2024-04-25T19:06:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f77a6800640 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 0x561707dd7c00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009fd0 tx=0x7f77a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc3241e0be5201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f7797000640 1 -- 172.21.15.86:0/1510075137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0004d20 con 0x561707dd7820 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f7797000640 1 -- 172.21.15.86:0/1510075137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a0004050 con 0x561707dd7820 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.406+0000 7f7797000640 1 -- 172.21.15.86:0/1510075137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0005260 con 0x561707dd7820 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f779f400640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f778c0a3dc0 0x7f778c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77a5e00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f778c0a3dc0 0x7f778c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f779f400640 1 -- 172.21.15.86:0/2877657630 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f77980d0cb0 con 0x7f77a8108560 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 -- 172.21.15.86:0/1510075137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 msgr2=0x561707dd7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 0x561707dd7c00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009fd0 tx=0x7f77a00118d0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 -- 172.21.15.86:0/1510075137 shutdown_connections 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707de4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707de24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 --2- 172.21.15.86:0/1510075137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd7820 0x561707dd7c00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.407+0000 7f77afa35740 1 -- 172.21.15.86:0/1510075137 >> 172.21.15.86:0/1510075137 conn(0x561707dd3d30 msgr2=0x561707dd4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.408+0000 7f77afa35740 1 -- 172.21.15.86:0/1510075137 shutdown_connections 2024-04-25T19:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.408+0000 7f77afa35740 1 -- 172.21.15.86:0/1510075137 wait complete. 2024-04-25T19:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.408+0000 7f77a5e00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f778c0a3dc0 0x7f778c0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x561707dd8dc0 tx=0x7f77980d4760 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.408+0000 7f77afa35740 1 -- start start 2024-04-25T19:06:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.408+0000 7f77afa35740 1 Processor -- start 2024-04-25T19:06:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd7820 0x561707ddf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707ddfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707de0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561707de5630 con 0x561707de2a00 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561707cf7310 con 0x561707dd7820 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561707de57b0 con 0x561707dd8140 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77afa35740 10 client.0 _ll_register_callbacks cb 0x561707cc5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f779d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707ddfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77a6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd7820 0x561707ddf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f779d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707ddfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:38556/0 (socket says 172.21.15.86:38556) 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f779d400640 1 -- 172.21.15.86:0/2501341663 learned_addr learned my addr 172.21.15.86:0/2501341663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.409+0000 7f77a6800640 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707de0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f779d400640 1 -- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd7820 msgr2=0x561707ddf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f779d400640 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd7820 0x561707ddf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f779d400640 1 -- 172.21.15.86:0/2501341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 msgr2=0x561707de0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f779d400640 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707de0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f779d400640 1 -- 172.21.15.86:0/2501341663 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561707b1c620 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f779d400640 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707ddfe50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7790000c00 tx=0x7f77900043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9ae8ed0ff8d0711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f778b400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f778b400640 1 -- 172.21.15.86:0/2501341663 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790035040 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f778b400640 1 -- 172.21.15.86:0/2501341663 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f7790005450 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.410+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a0004050 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f778b400640 1 -- 172.21.15.86:0/2501341663 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7790005650 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 10 client.25753 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561707df1810 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f778b400640 1 -- 172.21.15.86:0/2501341663 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a0004050 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f778b400640 1 -- 172.21.15.86:0/2501341663 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7790003ae0 con 0x561707dd8140 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 10 client.25753 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 20 client.25753 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707df2cd0 0x561707df50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 4 client.25753 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.411+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561707d518b0 con 0x561707df2cd0 2024-04-25T19:06:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.412+0000 7f77a6800640 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707df2cd0 0x561707df50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.412+0000 7f77a6800640 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707df2cd0 0x561707df50e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ec64518bac0575 server_cookie=cca690cc64411e2 in_seq=0 out_seq=0 2024-04-25T19:06:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.412+0000 7f778b400640 10 client.25753 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.413+0000 7f778b400640 1 -- 172.21.15.86:0/2501341663 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561707d518b0 con 0x561707df2cd0 2024-04-25T19:06:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.413+0000 7f778b400640 10 client.25753 handle_command_reply: tid=0 2024-04-25T19:06:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.413+0000 7f77afa35740 1 client.25753 shutdown 2024-04-25T19:06:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.413+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 msgr2=0x561707ddfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.413+0000 7f77afa35740 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707ddfe50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7790000c00 tx=0x7f77900043b0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 shutdown_connections 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707df2cd0 0x561707df50e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707de0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd8140 0x561707ddfe50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 --2- 172.21.15.86:0/2501341663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd7820 0x561707ddf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f778b400640 0 client.25753 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 >> 172.21.15.86:0/2501341663 conn(0x561707dd3d30 msgr2=0x561707dd9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 shutdown_connections 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 1 -- 172.21.15.86:0/2501341663 wait complete. 2024-04-25T19:06:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.414+0000 7f77afa35740 20 client.25753 trim_cache size 0 max 16384 2024-04-25T19:06:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 Processor -- start 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 -- start start 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707dee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd8140 0x561707de01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707dfa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b41f0 con 0x561707de2a00 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b5d30 con 0x561707dd8140 2024-04-25T19:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.430+0000 7f77afa35740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b48c0 con 0x561707dd7820 2024-04-25T19:06:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f77a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707dfa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd8140 0x561707de01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f77a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707dfa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53376/0 (socket says 172.21.15.86:53376) 2024-04-25T19:06:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f77a6800640 1 -- 172.21.15.86:0/628780647 learned_addr learned my addr 172.21.15.86:0/628780647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f77a6800640 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707dee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 -- 172.21.15.86:0/628780647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 msgr2=0x561707dee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707dee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 -- 172.21.15.86:0/628780647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 msgr2=0x561707dfa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707dfa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 -- 172.21.15.86:0/628780647 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561707dd88b0 con 0x561707dd8140 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f77a6800640 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707dfa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.431+0000 7f779d400640 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd8140 0x561707de01b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7790005ec0 tx=0x7f779002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a7193090336c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f7796600640 1 -- 172.21.15.86:0/628780647 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790035030 con 0x561707dd8140 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f7796600640 1 -- 172.21.15.86:0/628780647 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7790041070 con 0x561707dd8140 2024-04-25T19:06:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f7796600640 1 -- 172.21.15.86:0/628780647 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790052070 con 0x561707dd8140 2024-04-25T19:06:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 -- 172.21.15.86:0/628780647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd8140 msgr2=0x561707de01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd8140 0x561707de01b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7790005ec0 tx=0x7f779002a420 comp rx=0 tx=0).stop 2024-04-25T19:06:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 -- 172.21.15.86:0/628780647 shutdown_connections 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707de2a00 0x561707dfa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707dd8140 0x561707de01b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 --2- 172.21.15.86:0/628780647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707dee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.432+0000 7f77afa35740 1 -- 172.21.15.86:0/628780647 >> 172.21.15.86:0/628780647 conn(0x561707dd3d30 msgr2=0x561707ddac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.433+0000 7f77afa35740 1 -- 172.21.15.86:0/628780647 shutdown_connections 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.433+0000 7f77afa35740 1 -- 172.21.15.86:0/628780647 wait complete. 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.433+0000 7f77afa35740 1 -- start start 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.433+0000 7f77afa35740 1 Processor -- start 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707de7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 0x561707de8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707dea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b5660 con 0x561707dd8140 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b4f90 con 0x561707de2a00 2024-04-25T19:06:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b21d0 con 0x561707dd7820 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77afa35740 10 client.0 _ll_register_callbacks cb 0x7f7790004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 0x561707de8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 0x561707de8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53392/0 (socket says 172.21.15.86:53392) 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 -- 172.21.15.86:0/117634673 learned_addr learned my addr 172.21.15.86:0/117634673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77a6800640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707de7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77a6800640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707dea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 -- 172.21.15.86:0/117634673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 msgr2=0x561707de7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707de7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 -- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 msgr2=0x561707dea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707dea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f779d400640 1 -- 172.21.15.86:0/117634673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561707deaa80 con 0x561707dd8140 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.434+0000 7f77a6800640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707dea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f779d400640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 0x561707de8510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f779002a940 tx=0x7f7790031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c91adc0ce065bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f778b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f778b400640 1 -- 172.21.15.86:0/117634673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790035030 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7790041070 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f778b400640 1 -- 172.21.15.86:0/117634673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f77900035f0 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f778b400640 1 -- 172.21.15.86:0/117634673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7790052070 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f77afa35740 10 client.15666 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561707e58de0 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.435+0000 7f778b400640 1 -- 172.21.15.86:0/117634673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7790041070 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f778b400640 1 -- 172.21.15.86:0/117634673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7790004a60 con 0x561707dd8140 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77afa35740 10 client.15666 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77afa35740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77afa35740 20 client.15666 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77afa35740 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707e593f0 0x561707e5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77afa35740 4 client.15666 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561707d4b320 con 0x561707e593f0 2024-04-25T19:06:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.436+0000 7f77a6800640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707e593f0 0x561707e5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.437+0000 7f77a6800640 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707e593f0 0x561707e5b810 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3538ccff737d8d9 server_cookie=181cf175c7ed887f in_seq=0 out_seq=0 2024-04-25T19:06:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.437+0000 7f778b400640 10 client.15666 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.448+0000 7f778b400640 1 -- 172.21.15.86:0/117634673 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561707d4b320 con 0x561707e593f0 2024-04-25T19:06:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.448+0000 7f778b400640 10 client.15666 handle_command_reply: tid=0 2024-04-25T19:06:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.448+0000 7f77afa35740 1 client.15666 shutdown 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 msgr2=0x561707de8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 0x561707de8510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f779002a940 tx=0x7f7790031510 comp rx=0 tx=0).stop 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 shutdown_connections 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561707e593f0 0x561707e5b810 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561707de2a00 0x561707dea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561707dd8140 0x561707de8510 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f778b400640 0 client.15666 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 --2- 172.21.15.86:0/117634673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561707dd7820 0x561707de7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 >> 172.21.15.86:0/117634673 conn(0x561707dd3d30 msgr2=0x561707dd2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 shutdown_connections 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 1 -- 172.21.15.86:0/117634673 wait complete. 2024-04-25T19:06:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.449+0000 7f77afa35740 20 client.15666 trim_cache size 0 max 16384 2024-04-25T19:06:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:06:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:06:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:06:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f778c0a3dc0 msgr2=0x7f778c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f778c0a3dc0 0x7f778c0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x561707dd8dc0 tx=0x7f77980d4760 comp rx=0 tx=0).stop 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 msgr2=0x7f77a81a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 0x7f77a81a48b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f779800e9f0 tx=0x7f779800edf0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 shutdown_connections 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f778c0a3dc0 0x7f778c0a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a810e620 0x7f77a819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a8108f20 0x7f77a81a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 --2- 172.21.15.86:0/2877657630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a8108560 0x7f77a81a48b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 >> 172.21.15.86:0/2877657630 conn(0x7f77a80764e0 msgr2=0x7f77a8077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 shutdown_connections 2024-04-25T19:06:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:44.450+0000 7f77acc00640 1 -- 172.21.15.86:0/2877657630 wait complete. 2024-04-25T19:06:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:44 smithi086 ceph-mon[27783]: pgmap v116: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 18 op/s 2024-04-25T19:06:44.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:44 smithi183 ceph-mon[32344]: pgmap v116: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 18 op/s 2024-04-25T19:06:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:44 smithi194 ceph-mon[32740]: pgmap v116: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 18 op/s 2024-04-25T19:06:46.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:46 smithi086 ceph-mon[27783]: pgmap v117: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T19:06:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:46 smithi183 ceph-mon[32344]: pgmap v117: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T19:06:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:46 smithi194 ceph-mon[32740]: pgmap v117: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T19:06:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:48 smithi086 ceph-mon[27783]: pgmap v118: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T19:06:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:48 smithi183 ceph-mon[32344]: pgmap v118: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T19:06:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:48 smithi194 ceph-mon[32740]: pgmap v118: 97 pgs: 97 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T19:06:50.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:50 smithi194 ceph-mon[32740]: pgmap v119: 97 pgs: 97 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-04-25T19:06:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:50 smithi086 ceph-mon[27783]: pgmap v119: 97 pgs: 97 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-04-25T19:06:50.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:50 smithi183 ceph-mon[32344]: pgmap v119: 97 pgs: 97 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-04-25T19:06:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:52 smithi086 ceph-mon[27783]: pgmap v120: 97 pgs: 97 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:06:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:52 smithi183 ceph-mon[32344]: pgmap v120: 97 pgs: 97 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:06:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:52 smithi194 ceph-mon[32740]: pgmap v120: 97 pgs: 97 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:1/600: open fd = 3 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:2/600: open fd = 3 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:3/600: open fd = 3 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.007 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:4/600: open fd = 3 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:5/600: open fd = 3 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:6/600: open fd = 3 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.008 INFO:tasks.workunit.client.0.smithi086.stdout:7/600: open fd = 3 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:8/600: open fd = 3 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:9/600: open fd = 3 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:10/600: open fd = 3 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.009 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:11/600: open fd = 3 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:12/600: open fd = 3 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:13/600: open fd = 3 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.010 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:14/600: open fd = 3 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:15/600: open fd = 3 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:16/600: open fd = 3 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:17/600: open fd = 3 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.011 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:18/600: open fd = 3 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:19/600: open fd = 3 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:20/600: open fd = 3 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:21/600: open fd = 3 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.012 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:22/600: open fd = 3 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:23/600: open fd = 3 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:24/600: open fd = 3 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:25/600: open fd = 3 2024-04-25T19:06:54.013 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:26/600: open fd = 3 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:27/600: open fd = 3 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:28/600: open fd = 3 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.014 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:29/600: open fd = 3 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:30/600: open fd = 3 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:31/600: open fd = 3 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.015 INFO:tasks.workunit.client.0.smithi086.stdout:32/600: open fd = 3 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:33/600: open fd = 3 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:34/600: open fd = 3 2024-04-25T19:06:54.016 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:35/600: open fd = 3 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:36/600: open fd = 3 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:37/600: open fd = 3 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.017 INFO:tasks.workunit.client.0.smithi086.stdout:38/600: open fd = 3 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:39/600: open fd = 3 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:40/600: open fd = 3 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:41/600: open fd = 3 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.018 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:42/600: open fd = 3 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:43/600: open fd = 3 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:44/600: open fd = 3 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:45/600: open fd = 3 2024-04-25T19:06:54.019 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:46/600: open fd = 3 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:47/600: open fd = 3 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:48/600: open fd = 3 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.020 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:49/600: open fd = 3 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:50/600: open fd = 3 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:51/600: open fd = 3 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:52/600: open fd = 3 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.021 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:53/600: open fd = 3 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:54/600: open fd = 3 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:55/600: open fd = 3 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:56/600: open fd = 3 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.022 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:57/600: open fd = 3 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:58/600: open fd = 3 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:59/600: open fd = 3 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:60/600: open fd = 3 2024-04-25T19:06:54.023 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:61/600: open fd = 3 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:62/600: open fd = 3 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:63/600: open fd = 3 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:06:54.024 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:06:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:54 smithi086 ceph-mon[27783]: pgmap v121: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 22 op/s 2024-04-25T19:06:54.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:54 smithi183 ceph-mon[32344]: pgmap v121: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 22 op/s 2024-04-25T19:06:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:54 smithi194 ceph-mon[32740]: pgmap v121: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 22 op/s 2024-04-25T19:06:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:56 smithi086 ceph-mon[27783]: pgmap v122: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T19:06:56.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:56 smithi183 ceph-mon[32344]: pgmap v122: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T19:06:56.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:56 smithi194 ceph-mon[32740]: pgmap v122: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 -- 172.21.15.86:0/267143193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 msgr2=0x7ff2801153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 --2- 172.21.15.86:0/267143193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801153e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff278009ca0 tx=0x7ff27802a8b0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 -- 172.21.15.86:0/267143193 shutdown_connections 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 --2- 172.21.15.86:0/267143193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801153e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 --2- 172.21.15.86:0/267143193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 0x7ff2801089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 --2- 172.21.15.86:0/267143193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 0x7ff28010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 -- 172.21.15.86:0/267143193 >> 172.21.15.86:0/267143193 conn(0x7ff2800764e0 msgr2=0x7ff280078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 -- 172.21.15.86:0/267143193 shutdown_connections 2024-04-25T19:06:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.595+0000 7ff286600640 1 -- 172.21.15.86:0/267143193 wait complete. 2024-04-25T19:06:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.596+0000 7ff286600640 1 Processor -- start 2024-04-25T19:06:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 -- start start 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 0x7ff2801a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 0x7ff28019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801187e0 con 0x7ff280108f20 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff280118660 con 0x7ff28010e620 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff286600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff280118960 con 0x7ff280108560 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff277e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 0x7ff28019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff277e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41906/0 (socket says 172.21.15.86:41906) 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff277e00640 1 -- 172.21.15.86:0/4096892654 learned_addr learned my addr 172.21.15.86:0/4096892654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.597+0000 7ff27f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 0x7ff2801a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27fe00640 1 -- 172.21.15.86:0/4096892654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 msgr2=0x7ff2801a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27fe00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 0x7ff2801a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27fe00640 1 -- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 msgr2=0x7ff2801a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27fe00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27fe00640 1 -- 172.21.15.86:0/4096892654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28019f140 con 0x7ff28010e620 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff277e00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27f400640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 0x7ff2801a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.598+0000 7ff27fe00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 0x7ff28019e9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff278005e30 tx=0x7ff278030df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.599+0000 7ff27cc00640 1 -- 172.21.15.86:0/4096892654 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278003e00 con 0x7ff28010e620 2024-04-25T19:06:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.599+0000 7ff27cc00640 1 -- 172.21.15.86:0/4096892654 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff278040070 con 0x7ff28010e620 2024-04-25T19:06:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.599+0000 7ff27cc00640 1 -- 172.21.15.86:0/4096892654 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278045b10 con 0x7ff28010e620 2024-04-25T19:06:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.599+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28019f3d0 con 0x7ff28010e620 2024-04-25T19:06:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.599+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801ab6d0 con 0x7ff28010e620 2024-04-25T19:06:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.603+0000 7ff2895ec740 1 Processor -- start 2024-04-25T19:06:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.603+0000 7ff27cc00640 1 -- 172.21.15.86:0/4096892654 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff278045cb0 con 0x7ff28010e620 2024-04-25T19:06:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 -- start start 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e2b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e2ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9d86900 con 0x55c0c9e21140 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9d2cef0 con 0x55c0c9e2ba00 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff2895ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9d828c0 con 0x55c0c9e20820 2024-04-25T19:06:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff27f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e2b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff27f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e2b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41908/0 (socket says 172.21.15.86:41908) 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.604+0000 7ff27f400640 1 -- 172.21.15.86:0/3117996686 learned_addr learned my addr 172.21.15.86:0/3117996686 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff277e00640 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff277e00640 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e2ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27f400640 1 -- 172.21.15.86:0/3117996686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 msgr2=0x55c0c9e20c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27f400640 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e20c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27f400640 1 -- 172.21.15.86:0/3117996686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 msgr2=0x55c0c9e2ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27f400640 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e2ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27f400640 1 -- 172.21.15.86:0/3117996686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c9a99210 con 0x55c0c9e21140 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff277e00640 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e2ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27f400640 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e2b4c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff2700114f0 tx=0x7ff2700119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2110cead2c4b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27cc00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff2640a3e40 0x7ff2640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff274c00640 1 -- 172.21.15.86:0/3117996686 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27001e070 con 0x55c0c9e21140 2024-04-25T19:06:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff274c00640 1 -- 172.21.15.86:0/3117996686 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2700035f0 con 0x55c0c9e21140 2024-04-25T19:06:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff274c00640 1 -- 172.21.15.86:0/3117996686 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27000f820 con 0x55c0c9e21140 2024-04-25T19:06:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.605+0000 7ff27cc00640 1 -- 172.21.15.86:0/4096892654 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ff2780f1620 con 0x7ff28010e620 2024-04-25T19:06:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff277e00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff2640a3e40 0x7ff2640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 -- 172.21.15.86:0/3117996686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 msgr2=0x55c0c9e2b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e2b4c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff2700114f0 tx=0x7ff2700119b0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff277e00640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff2640a3e40 0x7ff2640a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff268009fd0 tx=0x7ff268005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 -- 172.21.15.86:0/3117996686 shutdown_connections 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e2ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e2b4c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3117996686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e20c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 -- 172.21.15.86:0/3117996686 >> 172.21.15.86:0/3117996686 conn(0x55c0c9e1cd30 msgr2=0x55c0c9e1d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 -- 172.21.15.86:0/3117996686 shutdown_connections 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.606+0000 7ff2895ec740 1 -- 172.21.15.86:0/3117996686 wait complete. 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.607+0000 7ff2895ec740 1 -- start start 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.607+0000 7ff2895ec740 1 Processor -- start 2024-04-25T19:06:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.607+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e28960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e21140 0x55c0c9e28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff2895ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9e2e630 con 0x55c0c9e2ba00 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff2895ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9d40310 con 0x55c0c9e21140 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff2895ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9e2e7b0 con 0x55c0c9e20820 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff2895ec740 10 client.0 _ll_register_callbacks cb 0x55c0c9d0ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e293e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41914/0 (socket says 172.21.15.86:41914) 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 -- 172.21.15.86:0/36217750 learned_addr learned my addr 172.21.15.86:0/36217750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff277e00640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e21140 0x55c0c9e28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27f400640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e28960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 -- 172.21.15.86:0/36217750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 msgr2=0x55c0c9e28960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e28960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 -- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e21140 msgr2=0x55c0c9e28ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e21140 0x55c0c9e28ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.608+0000 7ff27fe00640 1 -- 172.21.15.86:0/36217750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c9b65620 con 0x55c0c9e2ba00 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff27fe00640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e293e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2780c5d60 tx=0x7ff2780bd220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7a318e40b1c4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff26d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff26d600640 1 -- 172.21.15.86:0/36217750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780bd620 con 0x55c0c9e2ba00 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff26d600640 1 -- 172.21.15.86:0/36217750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ff2780bd880 con 0x55c0c9e2ba00 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2700035f0 con 0x55c0c9e2ba00 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff26d600640 1 -- 172.21.15.86:0/36217750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2780c19d0 con 0x55c0c9e2ba00 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff2895ec740 10 client.15678 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0c9e3a800 con 0x55c0c9e2ba00 2024-04-25T19:06:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.609+0000 7ff26d600640 1 -- 172.21.15.86:0/36217750 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2700035f0 con 0x55c0c9e2ba00 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff26d600640 1 -- 172.21.15.86:0/36217750 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff27804d030 con 0x55c0c9e2ba00 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff2895ec740 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff2895ec740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff2895ec740 20 client.15678 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff2895ec740 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9e3bc60 0x55c0c9e3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff2895ec740 4 client.15678 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c9d9a8b0 con 0x55c0c9e3bc60 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.610+0000 7ff27f400640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9e3bc60 0x55c0c9e3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.611+0000 7ff27f400640 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9e3bc60 0x55c0c9e3e0e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d856e337ff66c496 server_cookie=b21a7c57095e35 in_seq=0 out_seq=0 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.611+0000 7ff26d600640 10 client.15678 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.611+0000 7ff26d600640 1 -- 172.21.15.86:0/36217750 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0c9d9a8b0 con 0x55c0c9e3bc60 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.611+0000 7ff26d600640 10 client.15678 handle_command_reply: tid=0 2024-04-25T19:06:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.611+0000 7ff2895ec740 1 client.15678 shutdown 2024-04-25T19:06:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 msgr2=0x55c0c9e293e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e293e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2780c5d60 tx=0x7ff2780bd220 comp rx=0 tx=0).stop 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 shutdown_connections 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9e3bc60 0x55c0c9e3e0e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e293e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e21140 0x55c0c9e28ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 --2- 172.21.15.86:0/36217750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e28960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff26d600640 0 client.15678 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 >> 172.21.15.86:0/36217750 conn(0x55c0c9e1cd30 msgr2=0x55c0c9e22a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 shutdown_connections 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 1 -- 172.21.15.86:0/36217750 wait complete. 2024-04-25T19:06:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.612+0000 7ff2895ec740 20 client.15678 trim_cache size 0 max 16384 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.628+0000 7ff2895ec740 1 Processor -- start 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.628+0000 7ff2895ec740 1 -- start start 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 0x55c0c9e374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e21140 0x55c0c9e439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff2895ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b40e0 con 0x55c0c9e20820 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff2895ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b5c20 con 0x55c0c9e2ba00 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff2895ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b47b0 con 0x55c0c9e21140 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 0x55c0c9e374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 0x55c0c9e374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41918/0 (socket says 172.21.15.86:41918) 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 -- 172.21.15.86:0/1937725709 learned_addr learned my addr 172.21.15.86:0/1937725709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff277e00640 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e21140 0x55c0c9e439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27fe00640 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 -- 172.21.15.86:0/1937725709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e21140 msgr2=0x55c0c9e439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e21140 0x55c0c9e439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 -- 172.21.15.86:0/1937725709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 msgr2=0x55c0c9e43f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e43f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 -- 172.21.15.86:0/1937725709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c9e18690 con 0x55c0c9e20820 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27fe00640 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:06:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.629+0000 7ff27f400640 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 0x55c0c9e374f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff27001c040 tx=0x7ff270004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa11885821fc0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.630+0000 7ff26fe00640 1 -- 172.21.15.86:0/1937725709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27002f070 con 0x55c0c9e20820 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.630+0000 7ff26fe00640 1 -- 172.21.15.86:0/1937725709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270029030 con 0x55c0c9e20820 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.630+0000 7ff26fe00640 1 -- 172.21.15.86:0/1937725709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27001e040 con 0x55c0c9e20820 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.630+0000 7ff2895ec740 1 -- 172.21.15.86:0/1937725709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 msgr2=0x55c0c9e374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.630+0000 7ff2895ec740 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 0x55c0c9e374f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff27001c040 tx=0x7ff270004220 comp rx=0 tx=0).stop 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 -- 172.21.15.86:0/1937725709 shutdown_connections 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e21140 0x55c0c9e439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 --2- 172.21.15.86:0/1937725709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e20820 0x55c0c9e374f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 -- 172.21.15.86:0/1937725709 >> 172.21.15.86:0/1937725709 conn(0x55c0c9e1cd30 msgr2=0x55c0c9e1d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 -- 172.21.15.86:0/1937725709 shutdown_connections 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 -- 172.21.15.86:0/1937725709 wait complete. 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 -- start start 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.631+0000 7ff2895ec740 1 Processor -- start 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b5550 con 0x55c0c9e21140 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b4e80 con 0x55c0c9e2ba00 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b20c0 con 0x55c0c9e20820 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff2895ec740 10 client.0 _ll_register_callbacks cb 0x7ff2780b9200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff277e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff277e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e31970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41926/0 (socket says 172.21.15.86:41926) 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff277e00640 1 -- 172.21.15.86:0/3976441807 learned_addr learned my addr 172.21.15.86:0/3976441807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:06:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27fe00640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27f400640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27fe00640 1 -- 172.21.15.86:0/3976441807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 msgr2=0x55c0c9e31430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27fe00640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e31430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27fe00640 1 -- 172.21.15.86:0/3976441807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 msgr2=0x55c0c9e31970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27fe00640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e31970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff27fe00640 1 -- 172.21.15.86:0/3976441807 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c9ea1270 con 0x55c0c9e2ba00 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.632+0000 7ff277e00640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e31970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff27fe00640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e31eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff2780bb8d0 tx=0x7ff278004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a592e41c0732fb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff26d600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff26d600640 1 -- 172.21.15.86:0/3976441807 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780bdab0 con 0x55c0c9e2ba00 2024-04-25T19:06:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff270029030 con 0x55c0c9e2ba00 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff26d600640 1 -- 172.21.15.86:0/3976441807 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ff2780a6c20 con 0x55c0c9e2ba00 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff26d600640 1 -- 172.21.15.86:0/3976441807 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2780bd620 con 0x55c0c9e2ba00 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff2895ec740 10 client.25781 fetch_fsmap learned FSMap version 29 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0c9ea24d0 con 0x55c0c9e2ba00 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff26d600640 1 -- 172.21.15.86:0/3976441807 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270029030 con 0x55c0c9e2ba00 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.633+0000 7ff26d600640 1 -- 172.21.15.86:0/3976441807 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff27804d070 con 0x55c0c9e2ba00 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff2895ec740 10 client.25781 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff2895ec740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff2895ec740 20 client.25781 populate_metadata read hostname 'smithi086' 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9ea2a60 0x55c0c9ea4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff2895ec740 4 client.25781 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0c9d94320 con 0x55c0c9ea2a60 2024-04-25T19:06:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff27f400640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9ea2a60 0x55c0c9ea4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff27f400640 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9ea2a60 0x55c0c9ea4e80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea507edeec348bf2 server_cookie=e04a2cf14a7408dd in_seq=0 out_seq=0 2024-04-25T19:06:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.634+0000 7ff26d600640 10 client.25781 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.648+0000 7ff26d600640 1 -- 172.21.15.86:0/3976441807 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0c9d94320 con 0x55c0c9ea2a60 2024-04-25T19:06:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.648+0000 7ff26d600640 10 client.25781 handle_command_reply: tid=0 2024-04-25T19:06:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.648+0000 7ff2895ec740 1 client.25781 shutdown 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 msgr2=0x55c0c9e31eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e31eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff2780bb8d0 tx=0x7ff278004990 comp rx=0 tx=0).stop 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 shutdown_connections 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c0c9ea2a60 0x55c0c9ea4e80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0c9e2ba00 0x55c0c9e31eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0c9e21140 0x55c0c9e31970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff26d600640 0 client.25781 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 --2- 172.21.15.86:0/3976441807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0c9e20820 0x55c0c9e31430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 >> 172.21.15.86:0/3976441807 conn(0x55c0c9e1cd30 msgr2=0x55c0c9e1b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 shutdown_connections 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 1 -- 172.21.15.86:0/3976441807 wait complete. 2024-04-25T19:06:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.649+0000 7ff2895ec740 20 client.25781 trim_cache size 0 max 16384 2024-04-25T19:06:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:06:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:06:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:06:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff2640a3e40 msgr2=0x7ff2640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff2640a3e40 0x7ff2640a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff268009fd0 tx=0x7ff268005f90 comp rx=0 tx=0).stop 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 msgr2=0x7ff28019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 0x7ff28019e9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff278005e30 tx=0x7ff278030df0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 shutdown_connections 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff2640a3e40 0x7ff2640a6300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28010e620 0x7ff28019e9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff280108f20 0x7ff2801a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 --2- 172.21.15.86:0/4096892654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff280108560 0x7ff2801a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 >> 172.21.15.86:0/4096892654 conn(0x7ff2800764e0 msgr2=0x7ff280077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 shutdown_connections 2024-04-25T19:06:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:06:57.650+0000 7ff286600640 1 -- 172.21.15.86:0/4096892654 wait complete. 2024-04-25T19:06:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:06:58 smithi086 ceph-mon[27783]: pgmap v123: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T19:06:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:06:58 smithi183 ceph-mon[32344]: pgmap v123: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T19:06:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:06:58 smithi194 ceph-mon[32740]: pgmap v123: 97 pgs: 97 active+clean; 2.8 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T19:07:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:00 smithi086 ceph-mon[27783]: pgmap v124: 97 pgs: 97 active+clean; 3.0 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T19:07:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:00 smithi183 ceph-mon[32344]: pgmap v124: 97 pgs: 97 active+clean; 3.0 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T19:07:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:00 smithi194 ceph-mon[32740]: pgmap v124: 97 pgs: 97 active+clean; 3.0 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T19:07:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:02 smithi086 ceph-mon[27783]: pgmap v125: 97 pgs: 97 active+clean; 3.0 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T19:07:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:02 smithi183 ceph-mon[32344]: pgmap v125: 97 pgs: 97 active+clean; 3.0 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T19:07:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:02 smithi194 ceph-mon[32740]: pgmap v125: 97 pgs: 97 active+clean; 3.0 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T19:07:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:04 smithi086 ceph-mon[27783]: pgmap v126: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-04-25T19:07:04.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:04 smithi183 ceph-mon[32344]: pgmap v126: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-04-25T19:07:04.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:04 smithi194 ceph-mon[32740]: pgmap v126: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-04-25T19:07:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:06 smithi086 ceph-mon[27783]: pgmap v127: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-25T19:07:06.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:06 smithi183 ceph-mon[32344]: pgmap v127: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-25T19:07:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:06 smithi194 ceph-mon[32740]: pgmap v127: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-25T19:07:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:08 smithi086 ceph-mon[27783]: pgmap v128: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-25T19:07:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:08 smithi183 ceph-mon[32344]: pgmap v128: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-25T19:07:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:08 smithi194 ceph-mon[32740]: pgmap v128: 97 pgs: 97 active+clean; 3.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-04-25T19:07:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 -- 172.21.15.86:0/1814595433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b80742f0 msgr2=0x7fb2b80746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 --2- 172.21.15.86:0/1814595433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b80742f0 0x7fb2b80746d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009900 tx=0x7fb2a802a480 comp rx=0 tx=0).stop 2024-04-25T19:07:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 -- 172.21.15.86:0/1814595433 shutdown_connections 2024-04-25T19:07:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 --2- 172.21.15.86:0/1814595433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8112f40 0x7fb2b8115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 --2- 172.21.15.86:0/1814595433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8074c10 0x7fb2b8112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 --2- 172.21.15.86:0/1814595433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b80742f0 0x7fb2b80746d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 -- 172.21.15.86:0/1814595433 >> 172.21.15.86:0/1814595433 conn(0x7fb2b80fda00 msgr2=0x7fb2b80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 -- 172.21.15.86:0/1814595433 shutdown_connections 2024-04-25T19:07:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.785+0000 7fb2bd200640 1 -- 172.21.15.86:0/1814595433 wait complete. 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.786+0000 7fb2bd200640 1 Processor -- start 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 -- start start 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b80742f0 0x7fb2b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 0x7fb2b819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b8112f40 0x7fb2b81a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8118830 con 0x7fb2b8074c10 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81186b0 con 0x7fb2b8112f40 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2bd200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81189b0 con 0x7fb2b80742f0 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 0x7fb2b819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 0x7fb2b819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45674/0 (socket says 172.21.15.86:45674) 2024-04-25T19:07:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2afe00640 1 -- 172.21.15.86:0/4257883787 learned_addr learned my addr 172.21.15.86:0/4257883787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b80742f0 0x7fb2b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.787+0000 7fb2b6800640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b8112f40 0x7fb2b81a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2afe00640 1 -- 172.21.15.86:0/4257883787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b80742f0 msgr2=0x7fb2b819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2afe00640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b80742f0 0x7fb2b819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2afe00640 1 -- 172.21.15.86:0/4257883787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b8112f40 msgr2=0x7fb2b81a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2afe00640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b8112f40 0x7fb2b81a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2afe00640 1 -- 172.21.15.86:0/4257883787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b81a3d10 con 0x7fb2b8074c10 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2afe00640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 0x7fb2b819f2b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e9f0 tx=0x7fb2a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2aea00640 1 -- 172.21.15.86:0/4257883787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000cd80 con 0x7fb2b8074c10 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2aea00640 1 -- 172.21.15.86:0/4257883787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a00054d0 con 0x7fb2b8074c10 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b81a4000 con 0x7fb2b8074c10 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.788+0000 7fb2aea00640 1 -- 172.21.15.86:0/4257883787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0010690 con 0x7fb2b8074c10 2024-04-25T19:07:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.789+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b81ab890 con 0x7fb2b8074c10 2024-04-25T19:07:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2c000e740 1 Processor -- start 2024-04-25T19:07:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2c000e740 1 -- start start 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c2581c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c258c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2aea00640 1 -- 172.21.15.86:0/4257883787 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2a0004e90 con 0x7fb2b8074c10 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c258edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.793+0000 7fb2c000e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5611c24e7900 con 0x5611c258ca00 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2c000e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611c248def0 con 0x5611c2581820 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2c000e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5611c24e38c0 con 0x5611c2582140 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c258c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c258c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53624/0 (socket says 172.21.15.86:53624) 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3739341280 learned_addr learned my addr 172.21.15.86:0/3739341280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b6800640 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c2581c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2afe00640 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c258edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3739341280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 msgr2=0x5611c2581c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c2581c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3739341280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 msgr2=0x5611c258edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c258edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3739341280 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c21fa210 con 0x5611c2582140 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.794+0000 7fb2afe00640 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c258edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.795+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c258c4c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8000c00 tx=0x7fb2a8004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d8c7f9c700d38f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.795+0000 7fb2a7400640 1 -- 172.21.15.86:0/3739341280 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8035040 con 0x5611c2582140 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.795+0000 7fb2a7400640 1 -- 172.21.15.86:0/3739341280 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a80054f0 con 0x5611c2582140 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.795+0000 7fb2a7400640 1 -- 172.21.15.86:0/3739341280 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80035f0 con 0x5611c2582140 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.795+0000 7fb2aea00640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb29c0a3dc0 0x7fb29c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.795+0000 7fb2aea00640 1 -- 172.21.15.86:0/4257883787 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fb2a00d0ca0 con 0x7fb2b8074c10 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2b6800640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb29c0a3dc0 0x7fb29c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 -- 172.21.15.86:0/3739341280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 msgr2=0x5611c258c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c258c4c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8000c00 tx=0x7fb2a8004170 comp rx=0 tx=0).stop 2024-04-25T19:07:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 -- 172.21.15.86:0/3739341280 shutdown_connections 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c258edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c258c4c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3739341280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c2581c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 -- 172.21.15.86:0/3739341280 >> 172.21.15.86:0/3739341280 conn(0x5611c257dd30 msgr2=0x5611c257e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 -- 172.21.15.86:0/3739341280 shutdown_connections 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2c000e740 1 -- 172.21.15.86:0/3739341280 wait complete. 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.796+0000 7fb2b6800640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb29c0a3dc0 0x7fb29c0a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2b806b5d0 tx=0x7fb2b00096a0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 -- start start 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 Processor -- start 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2589a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c2589f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c258a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5611c258f5d0 con 0x5611c2582140 2024-04-25T19:07:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611c24a1310 con 0x5611c258ca00 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5611c258f750 con 0x5611c2581820 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.797+0000 7fb2c000e740 10 client.0 _ll_register_callbacks cb 0x5611c246fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2589a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c2589f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2589a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53626/0 (socket says 172.21.15.86:53626) 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3493409109 learned_addr learned my addr 172.21.15.86:0/3493409109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b6800640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c258a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 msgr2=0x5611c258a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c258a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3493409109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 msgr2=0x5611c2589f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c2589f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 -- 172.21.15.86:0/3493409109 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c22c6620 con 0x5611c2581820 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b6800640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c258a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2afe00640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c2589f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2589a10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005c40 tx=0x7fb2a8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e503a0f60e6e2628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.798+0000 7fb29be00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb29be00640 1 -- 172.21.15.86:0/3493409109 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8053060 con 0x5611c2581820 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb29be00640 1 -- 172.21.15.86:0/3493409109 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fb2a8004770 con 0x5611c2581820 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a80054f0 con 0x5611c2581820 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb29be00640 1 -- 172.21.15.86:0/3493409109 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8035050 con 0x5611c2581820 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb2c000e740 10 client.25789 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611c258f5d0 con 0x5611c2581820 2024-04-25T19:07:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb29be00640 1 -- 172.21.15.86:0/3493409109 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a80054f0 con 0x5611c2581820 2024-04-25T19:07:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.799+0000 7fb29be00640 1 -- 172.21.15.86:0/3493409109 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb2a8003a40 con 0x5611c2581820 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2c000e740 10 client.25789 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2c000e740 10 client.25789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2c000e740 20 client.25789 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c259ca40 0x5611c259ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2c000e740 4 client.25789 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611c24fb8b0 con 0x5611c259ca40 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.800+0000 7fb2afe00640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c259ca40 0x5611c259ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.801+0000 7fb2afe00640 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c259ca40 0x5611c259ee30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35b674b9e4a437d server_cookie=72c4b59555c9bbaf in_seq=0 out_seq=0 2024-04-25T19:07:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.801+0000 7fb29be00640 10 client.25789 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.801+0000 7fb29be00640 1 -- 172.21.15.86:0/3493409109 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611c24fb8b0 con 0x5611c259ca40 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.801+0000 7fb29be00640 10 client.25789 handle_command_reply: tid=0 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.801+0000 7fb2c000e740 1 client.25789 shutdown 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 msgr2=0x5611c2589a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2589a10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005c40 tx=0x7fb2a8005850 comp rx=0 tx=0).stop 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 shutdown_connections 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c259ca40 0x5611c259ee30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c258a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c2589f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 --2- 172.21.15.86:0/3493409109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2589a10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb29be00640 0 client.25789 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 >> 172.21.15.86:0/3493409109 conn(0x5611c257dd30 msgr2=0x5611c2583a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 shutdown_connections 2024-04-25T19:07:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 1 -- 172.21.15.86:0/3493409109 wait complete. 2024-04-25T19:07:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.802+0000 7fb2c000e740 20 client.25789 trim_cache size 0 max 16384 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 Processor -- start 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 -- start start 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2598490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c25838e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c25a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b42a0 con 0x5611c2582140 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b5de0 con 0x5611c258ca00 2024-04-25T19:07:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.818+0000 7fb2c000e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b4970 con 0x5611c2581820 2024-04-25T19:07:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c25838e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2598490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c25838e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45704/0 (socket says 172.21.15.86:45704) 2024-04-25T19:07:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2afe00640 1 -- 172.21.15.86:0/2835035700 learned_addr learned my addr 172.21.15.86:0/2835035700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b6800640 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c25a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 -- 172.21.15.86:0/2835035700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 msgr2=0x5611c25a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c25a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 -- 172.21.15.86:0/2835035700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 msgr2=0x5611c25838e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c25838e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 -- 172.21.15.86:0/2835035700 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c257d610 con 0x5611c2581820 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2afe00640 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c25838e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.819+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2598490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8004650 tx=0x7fb2a8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b44b8e34c9819f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.820+0000 7fb2a6a00640 1 -- 172.21.15.86:0/2835035700 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8053070 con 0x5611c2581820 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.820+0000 7fb2a6a00640 1 -- 172.21.15.86:0/2835035700 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a80035f0 con 0x5611c2581820 2024-04-25T19:07:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.820+0000 7fb2a6a00640 1 -- 172.21.15.86:0/2835035700 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8005330 con 0x5611c2581820 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 -- 172.21.15.86:0/2835035700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 msgr2=0x5611c2598490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2598490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8004650 tx=0x7fb2a8000bc0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 -- 172.21.15.86:0/2835035700 shutdown_connections 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c258ca00 0x5611c25a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c2582140 0x5611c25838e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2835035700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2581820 0x5611c2598490 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 -- 172.21.15.86:0/2835035700 >> 172.21.15.86:0/2835035700 conn(0x5611c257dd30 msgr2=0x5611c25831d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 -- 172.21.15.86:0/2835035700 shutdown_connections 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 -- 172.21.15.86:0/2835035700 wait complete. 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 -- start start 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.821+0000 7fb2c000e740 1 Processor -- start 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c25922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c2592810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c2592d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b5710 con 0x5611c258ca00 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b5040 con 0x5611c2581820 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b2280 con 0x5611c2582140 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2c000e740 10 client.0 _ll_register_callbacks cb 0x7fb2a8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c2592810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c2592810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53644/0 (socket says 172.21.15.86:53644) 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 -- 172.21.15.86:0/2718941063 learned_addr learned my addr 172.21.15.86:0/2718941063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c25922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2b6800640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c2592d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 -- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 msgr2=0x5611c25922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c25922d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 -- 172.21.15.86:0/2718941063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 msgr2=0x5611c2592d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c2592d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2afe00640 1 -- 172.21.15.86:0/2718941063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c2601dc0 con 0x5611c2582140 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.822+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c25922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb2afe00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c2592810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2a009cb20 tx=0x7fb2a00bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ba32413d8d8270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb29be00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb29be00640 1 -- 172.21.15.86:0/2718941063 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0005180 con 0x5611c2582140 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb29be00640 1 -- 172.21.15.86:0/2718941063 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fb2a00046c0 con 0x5611c2582140 2024-04-25T19:07:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a80035f0 con 0x5611c2582140 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb29be00640 1 -- 172.21.15.86:0/2718941063 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2a0005680 con 0x5611c2582140 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb2c000e740 10 client.25795 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.823+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611c2603010 con 0x5611c2582140 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb29be00640 1 -- 172.21.15.86:0/2718941063 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a80035f0 con 0x5611c2582140 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb29be00640 1 -- 172.21.15.86:0/2718941063 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb2a00d6040 con 0x5611c2582140 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2c000e740 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2c000e740 10 client.25795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2c000e740 20 client.25795 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c2603790 0x5611c2605bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2c000e740 4 client.25795 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611c24f5320 con 0x5611c2603790 2024-04-25T19:07:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.824+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c2603790 0x5611c2605bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.825+0000 7fb2b5e00640 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c2603790 0x5611c2605bb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f02601eee0bc08 server_cookie=44f0f37e3a4129bc in_seq=0 out_seq=0 2024-04-25T19:07:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.825+0000 7fb29be00640 10 client.25795 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.836+0000 7fb29be00640 1 -- 172.21.15.86:0/2718941063 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611c24f5320 con 0x5611c2603790 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb29be00640 10 client.25795 handle_command_reply: tid=0 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 client.25795 shutdown 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 msgr2=0x5611c2592810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c2592810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2a009cb20 tx=0x7fb2a00bd040 comp rx=0 tx=0).stop 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 shutdown_connections 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5611c2603790 0x5611c2605bb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5611c258ca00 0x5611c2592d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb29be00640 0 client.25795 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611c2582140 0x5611c2592810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 --2- 172.21.15.86:0/2718941063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611c2581820 0x5611c25922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 >> 172.21.15.86:0/2718941063 conn(0x5611c257dd30 msgr2=0x5611c2583a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 shutdown_connections 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 1 -- 172.21.15.86:0/2718941063 wait complete. 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.837+0000 7fb2c000e740 20 client.25795 trim_cache size 0 max 16384 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:07:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:07:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb29c0a3dc0 msgr2=0x7fb29c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb29c0a3dc0 0x7fb29c0a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2b806b5d0 tx=0x7fb2b00096a0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 msgr2=0x7fb2b819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 0x7fb2b819f2b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e9f0 tx=0x7fb2a000edf0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 shutdown_connections 2024-04-25T19:07:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb29c0a3dc0 0x7fb29c0a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2b8112f40 0x7fb2b81a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b8074c10 0x7fb2b819f2b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 --2- 172.21.15.86:0/4257883787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b80742f0 0x7fb2b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 >> 172.21.15.86:0/4257883787 conn(0x7fb2b80fda00 msgr2=0x7fb2b80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.838+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 shutdown_connections 2024-04-25T19:07:10.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:10.839+0000 7fb2bd200640 1 -- 172.21.15.86:0/4257883787 wait complete. 2024-04-25T19:07:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:10 smithi183 ceph-mon[32344]: pgmap v129: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-25T19:07:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:10 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:07:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:10 smithi194 ceph-mon[32740]: pgmap v129: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-25T19:07:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:10 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:07:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:10 smithi086 ceph-mon[27783]: pgmap v129: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-25T19:07:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:10 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:07:11.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:11 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:07:11.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:11 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:07:11.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:11 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:07:11.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:11 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:07:11.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:11 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:07:11.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:11 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:07:12.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:11 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:07:12.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:11 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:07:12.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:11 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:07:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:12 smithi183 ceph-mon[32344]: pgmap v130: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-25T19:07:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:12 smithi194 ceph-mon[32740]: pgmap v130: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-25T19:07:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:12 smithi086 ceph-mon[27783]: pgmap v130: 97 pgs: 97 active+clean; 3.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-25T19:07:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:14 smithi183 ceph-mon[32344]: pgmap v131: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 36 op/s 2024-04-25T19:07:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:14 smithi194 ceph-mon[32740]: pgmap v131: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 36 op/s 2024-04-25T19:07:15.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:14 smithi086 ceph-mon[27783]: pgmap v131: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 36 op/s 2024-04-25T19:07:16.872 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:16 smithi086 ceph-mon[27783]: pgmap v132: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:16 smithi183 ceph-mon[32344]: pgmap v132: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:16 smithi194 ceph-mon[32740]: pgmap v132: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:18 smithi183 ceph-mon[32344]: pgmap v133: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:18 smithi194 ceph-mon[32740]: pgmap v133: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:19.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:18 smithi086 ceph-mon[27783]: pgmap v133: 97 pgs: 97 active+clean; 3.5 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:20 smithi183 ceph-mon[32344]: pgmap v134: 97 pgs: 97 active+clean; 3.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:20 smithi194 ceph-mon[32740]: pgmap v134: 97 pgs: 97 active+clean; 3.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:20 smithi086 ceph-mon[27783]: pgmap v134: 97 pgs: 97 active+clean; 3.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:22 smithi183 ceph-mon[32344]: pgmap v135: 97 pgs: 97 active+clean; 3.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:22 smithi194 ceph-mon[32740]: pgmap v135: 97 pgs: 97 active+clean; 3.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:22 smithi086 ceph-mon[27783]: pgmap v135: 97 pgs: 97 active+clean; 3.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 -- 172.21.15.86:0/2767781047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 msgr2=0x7f010c1153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 --2- 172.21.15.86:0/2767781047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 0x7f010c1153e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0108009c10 tx=0x7f010802a6e0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 -- 172.21.15.86:0/2767781047 shutdown_connections 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 --2- 172.21.15.86:0/2767781047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 0x7f010c1153e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 --2- 172.21.15.86:0/2767781047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 --2- 172.21.15.86:0/2767781047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f010c10e620 0x7f010c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 -- 172.21.15.86:0/2767781047 >> 172.21.15.86:0/2767781047 conn(0x7f010c0764e0 msgr2=0x7f010c078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 -- 172.21.15.86:0/2767781047 shutdown_connections 2024-04-25T19:07:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.985+0000 7f0114800640 1 -- 172.21.15.86:0/2767781047 wait complete. 2024-04-25T19:07:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.986+0000 7f0114800640 1 Processor -- start 2024-04-25T19:07:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.986+0000 7f0114800640 1 -- start start 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0114800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0114800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 0x7f010c1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0114800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f010c10e620 0x7f010c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0114800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1187b0 con 0x7f010c108560 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0114800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f010c118630 con 0x7f010c10e620 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0114800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f010c118930 con 0x7f010c108f20 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0111e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0111e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53068/0 (socket says 172.21.15.86:53068) 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0111e00640 1 -- 172.21.15.86:0/1490474446 learned_addr learned my addr 172.21.15.86:0/1490474446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0111400640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 0x7f010c1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.987+0000 7f0112800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f010c10e620 0x7f010c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0111e00640 1 -- 172.21.15.86:0/1490474446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 msgr2=0x7f010c1a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0111e00640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 0x7f010c1a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0111e00640 1 -- 172.21.15.86:0/1490474446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f010c10e620 msgr2=0x7f010c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0111e00640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f010c10e620 0x7f010c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0111e00640 1 -- 172.21.15.86:0/1490474446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c19f170 con 0x7f010c108560 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0111e00640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1a4840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f010000e9f0 tx=0x7f010000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0106a00640 1 -- 172.21.15.86:0/1490474446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010000cd80 con 0x7f010c108560 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0106a00640 1 -- 172.21.15.86:0/1490474446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01000054d0 con 0x7f010c108560 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c19f460 con 0x7f010c108560 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.988+0000 7f0106a00640 1 -- 172.21.15.86:0/1490474446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100010690 con 0x7f010c108560 2024-04-25T19:07:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.989+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1ab700 con 0x7f010c108560 2024-04-25T19:07:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.992+0000 7f0117322740 1 Processor -- start 2024-04-25T19:07:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.992+0000 7f0106a00640 1 -- 172.21.15.86:0/1490474446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0100004e90 con 0x7f010c108560 2024-04-25T19:07:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.992+0000 7f0117322740 1 -- start start 2024-04-25T19:07:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.992+0000 7f0117322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b77c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0117322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0117322740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462b84df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0117322740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560462add900 con 0x560462b82a00 2024-04-25T19:07:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0117322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560462a83ef0 con 0x560462b78140 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0117322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560462ad98c0 con 0x560462b77820 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0111e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0112800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b77c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0111e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b824c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48602/0 (socket says 172.21.15.86:48602) 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0112800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b77c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51492/0 (socket says 172.21.15.86:51492) 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0111e00640 1 -- 172.21.15.86:0/1916168047 learned_addr learned my addr 172.21.15.86:0/1916168047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.993+0000 7f0111400640 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462b84df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111e00640 1 -- 172.21.15.86:0/1916168047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 msgr2=0x560462b77c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111e00640 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b77c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111e00640 1 -- 172.21.15.86:0/1916168047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 msgr2=0x560462b84df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111e00640 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462b84df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111e00640 1 -- 172.21.15.86:0/1916168047 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604627f0210 con 0x560462b78140 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0112800640 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b77c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111400640 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462b84df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0106a00640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f00f40a3e40 0x7f00f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0111e00640 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b824c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01000ce200 tx=0x7f01000d4cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d5d720dae6d023 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0106a00640 1 -- 172.21.15.86:0/1490474446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f01000d0c20 con 0x7f010c108560 2024-04-25T19:07:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f0112800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f00f40a3e40 0x7f00f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f00fb000640 1 -- 172.21.15.86:0/1916168047 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100098db0 con 0x560462b78140 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f00fb000640 1 -- 172.21.15.86:0/1916168047 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0100005680 con 0x560462b78140 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.994+0000 7f00fb000640 1 -- 172.21.15.86:0/1916168047 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100005180 con 0x560462b78140 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 -- 172.21.15.86:0/1916168047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 msgr2=0x560462b824c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0112800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f00f40a3e40 0x7f00f40a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f010802ac00 tx=0x7f0108005fd0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b824c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01000ce200 tx=0x7f01000d4cc0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 -- 172.21.15.86:0/1916168047 shutdown_connections 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462b84df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b824c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 --2- 172.21.15.86:0/1916168047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b77c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 -- 172.21.15.86:0/1916168047 >> 172.21.15.86:0/1916168047 conn(0x560462b73d30 msgr2=0x560462b74110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 -- 172.21.15.86:0/1916168047 shutdown_connections 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.995+0000 7f0117322740 1 -- 172.21.15.86:0/1916168047 wait complete. 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.996+0000 7f0117322740 1 -- start start 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.996+0000 7f0117322740 1 Processor -- start 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.996+0000 7f0117322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 0x560462b7f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.996+0000 7f0117322740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b78140 0x560462b7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.996+0000 7f0117322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b80320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0117322740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560462b85630 con 0x560462b78140 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0117322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560462a97310 con 0x560462b77820 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0117322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560462b857b0 con 0x560462b82a00 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0117322740 10 client.0 _ll_register_callbacks cb 0x560462a65cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 0x560462b7f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 0x560462b7f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48618/0 (socket says 172.21.15.86:48618) 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 -- 172.21.15.86:0/4203102281 learned_addr learned my addr 172.21.15.86:0/4203102281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111400640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b78140 0x560462b7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0112800640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b80320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 -- 172.21.15.86:0/4203102281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 msgr2=0x560462b80320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b80320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 -- 172.21.15.86:0/4203102281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b78140 msgr2=0x560462b7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b78140 0x560462b7fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.997+0000 7f0111e00640 1 -- 172.21.15.86:0/4203102281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604628bc620 con 0x560462b77820 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f0111e00640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 0x560462b7f8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0100004ad0 tx=0x7f0100009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f80d8c7ea2e59f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f00ef400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f00ef400640 1 -- 172.21.15.86:0/4203102281 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01000b5020 con 0x560462b77820 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f00ef400640 1 -- 172.21.15.86:0/4203102281 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f01000046c0 con 0x560462b77820 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0100005680 con 0x560462b77820 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f00ef400640 1 -- 172.21.15.86:0/4203102281 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01000aa050 con 0x560462b77820 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f0117322740 10 client.25811 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560462b91820 con 0x560462b77820 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f00ef400640 1 -- 172.21.15.86:0/4203102281 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0100005680 con 0x560462b77820 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.998+0000 7f00ef400640 1 -- 172.21.15.86:0/4203102281 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01000d7070 con 0x560462b77820 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0117322740 10 client.25811 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0117322740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0117322740 20 client.25811 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0117322740 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462b92b70 0x560462b94f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0117322740 4 client.25811 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560462af18b0 con 0x560462b92b70 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0111400640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462b92b70 0x560462b94f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f0111400640 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462b92b70 0x560462b94f60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a696346f054ff1 server_cookie=58700bb266611a69 in_seq=0 out_seq=0 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:23.999+0000 7f00ef400640 10 client.25811 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f00ef400640 1 -- 172.21.15.86:0/4203102281 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560462af18b0 con 0x560462b92b70 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f00ef400640 10 client.25811 handle_command_reply: tid=0 2024-04-25T19:07:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 client.25811 shutdown 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 msgr2=0x560462b7f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 0x560462b7f8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0100004ad0 tx=0x7f0100009720 comp rx=0 tx=0).stop 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 shutdown_connections 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462b92b70 0x560462b94f60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b80320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f00ef400640 0 client.25811 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b78140 0x560462b7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 --2- 172.21.15.86:0/4203102281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b77820 0x560462b7f8a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.000+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 >> 172.21.15.86:0/4203102281 conn(0x560462b73d30 msgr2=0x560462b79a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.001+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 shutdown_connections 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.001+0000 7f0117322740 1 -- 172.21.15.86:0/4203102281 wait complete. 2024-04-25T19:07:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.001+0000 7f0117322740 20 client.25811 trim_cache size 0 max 16384 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.017+0000 7f0117322740 1 Processor -- start 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.017+0000 7f0117322740 1 -- start start 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.017+0000 7f0117322740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b77820 0x560462b7f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.017+0000 7f0117322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.017+0000 7f0117322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.017+0000 7f0117322740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1b42b0 con 0x560462b77820 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0117322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1b5df0 con 0x560462b78140 2024-04-25T19:07:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0117322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1b4980 con 0x560462b82a00 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48626/0 (socket says 172.21.15.86:48626) 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 -- 172.21.15.86:0/2644116733 learned_addr learned my addr 172.21.15.86:0/2644116733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111e00640 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b77820 0x560462b7f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0112800640 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 -- 172.21.15.86:0/2644116733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 msgr2=0x560462b984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 -- 172.21.15.86:0/2644116733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b77820 msgr2=0x560462b7f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b77820 0x560462b7f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.018+0000 7f0111400640 1 -- 172.21.15.86:0/2644116733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560462b9ada0 con 0x560462b78140 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.019+0000 7f0111400640 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005c80 tx=0x7f00fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997b3d56ee05a920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.019+0000 7f00fa600640 1 -- 172.21.15.86:0/2644116733 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc01e710 con 0x560462b78140 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.019+0000 7f00fa600640 1 -- 172.21.15.86:0/2644116733 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00fc01e8b0 con 0x560462b78140 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.019+0000 7f00fa600640 1 -- 172.21.15.86:0/2644116733 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc004a00 con 0x560462b78140 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 -- 172.21.15.86:0/2644116733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 msgr2=0x560462b97f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005c80 tx=0x7f00fc0046c0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 -- 172.21.15.86:0/2644116733 shutdown_connections 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b82a00 0x560462b984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97f80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 --2- 172.21.15.86:0/2644116733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b77820 0x560462b7f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 -- 172.21.15.86:0/2644116733 >> 172.21.15.86:0/2644116733 conn(0x560462b73d30 msgr2=0x560462b8e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 -- 172.21.15.86:0/2644116733 shutdown_connections 2024-04-25T19:07:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 -- 172.21.15.86:0/2644116733 wait complete. 2024-04-25T19:07:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 -- start start 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.020+0000 7f0117322740 1 Processor -- start 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b96d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462bf6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1b5720 con 0x560462b82a00 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1b5050 con 0x560462b78140 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1b2290 con 0x560462b77820 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0117322740 10 client.0 _ll_register_callbacks cb 0x7f01000d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0111400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0112800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462bf6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0111400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48640/0 (socket says 172.21.15.86:48640) 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0111400640 1 -- 172.21.15.86:0/3491086457 learned_addr learned my addr 172.21.15.86:0/3491086457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.021+0000 7f0111e00640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b96d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111400640 1 -- 172.21.15.86:0/3491086457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 msgr2=0x560462b96d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111400640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b96d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111400640 1 -- 172.21.15.86:0/3491086457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 msgr2=0x560462bf6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111400640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462bf6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111400640 1 -- 172.21.15.86:0/3491086457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560462bf7290 con 0x560462b78140 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111e00640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b96d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0111400640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0092a0 tx=0x7f00fc009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92412d3820fd1bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f00ef400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f00ef400640 1 -- 172.21.15.86:0/3491086457 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc02d030 con 0x560462b78140 2024-04-25T19:07:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00fc01e8b0 con 0x560462b78140 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.022+0000 7f00ef400640 1 -- 172.21.15.86:0/3491086457 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f00fc01e690 con 0x560462b78140 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f00ef400640 1 -- 172.21.15.86:0/3491086457 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00fc037050 con 0x560462b78140 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560462bf8440 con 0x560462b78140 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f00ef400640 1 -- 172.21.15.86:0/3491086457 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00fc01e8b0 con 0x560462b78140 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f00ef400640 1 -- 172.21.15.86:0/3491086457 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f00fc029470 con 0x560462b78140 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 20 client.25817 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462bf8b90 0x560462bfafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 4 client.25817 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.023+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560462aeb320 con 0x560462bf8b90 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.024+0000 7f0111e00640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462bf8b90 0x560462bfafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.024+0000 7f00ef400640 10 client.25817 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.024+0000 7f0111e00640 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462bf8b90 0x560462bfafb0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1369d32943a59c10 server_cookie=fb1a264ef72f1f2 in_seq=0 out_seq=0 2024-04-25T19:07:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.038+0000 7f00ef400640 1 -- 172.21.15.86:0/3491086457 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560462aeb320 con 0x560462bf8b90 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.038+0000 7f00ef400640 10 client.25817 handle_command_reply: tid=0 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.038+0000 7f0117322740 1 client.25817 shutdown 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 msgr2=0x560462b97280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0092a0 tx=0x7f00fc009be0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 shutdown_connections 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560462bf8b90 0x560462bfafb0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560462b82a00 0x560462bf6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560462b78140 0x560462b97280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f00ef400640 0 client.25817 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 --2- 172.21.15.86:0/3491086457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560462b77820 0x560462b96d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 >> 172.21.15.86:0/3491086457 conn(0x560462b73d30 msgr2=0x560462b80180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 shutdown_connections 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 1 -- 172.21.15.86:0/3491086457 wait complete. 2024-04-25T19:07:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.039+0000 7f0117322740 20 client.25817 trim_cache size 0 max 16384 2024-04-25T19:07:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:07:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:07:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:07:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f00f40a3e40 msgr2=0x7f00f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f00f40a3e40 0x7f00f40a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f010802ac00 tx=0x7f0108005fd0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 msgr2=0x7f010c1a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1a4840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f010000e9f0 tx=0x7f010000edf0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 shutdown_connections 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f00f40a3e40 0x7f00f40a6300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f010c10e620 0x7f010c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f010c108f20 0x7f010c1a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 --2- 172.21.15.86:0/1490474446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c108560 0x7f010c1a4840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 >> 172.21.15.86:0/1490474446 conn(0x7f010c0764e0 msgr2=0x7f010c077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 shutdown_connections 2024-04-25T19:07:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:24.040+0000 7f0114800640 1 -- 172.21.15.86:0/1490474446 wait complete. 2024-04-25T19:07:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:24 smithi183 ceph-mon[32344]: pgmap v136: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:07:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:24 smithi194 ceph-mon[32740]: pgmap v136: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:07:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:24 smithi086 ceph-mon[27783]: pgmap v136: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:07:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:26 smithi183 ceph-mon[32344]: pgmap v137: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:07:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:26 smithi194 ceph-mon[32740]: pgmap v137: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:07:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:26 smithi086 ceph-mon[27783]: pgmap v137: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:07:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:28 smithi183 ceph-mon[32344]: pgmap v138: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:07:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:28 smithi194 ceph-mon[32740]: pgmap v138: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:07:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:28 smithi086 ceph-mon[27783]: pgmap v138: 97 pgs: 97 active+clean; 3.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:07:30.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:30 smithi183 ceph-mon[32344]: pgmap v139: 97 pgs: 97 active+clean; 4.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:07:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:30 smithi194 ceph-mon[32740]: pgmap v139: 97 pgs: 97 active+clean; 4.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:07:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:30 smithi086 ceph-mon[27783]: pgmap v139: 97 pgs: 97 active+clean; 4.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:07:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:32 smithi183 ceph-mon[32344]: pgmap v140: 97 pgs: 97 active+clean; 4.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:32.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:32 smithi194 ceph-mon[32740]: pgmap v140: 97 pgs: 97 active+clean; 4.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:32 smithi086 ceph-mon[27783]: pgmap v140: 97 pgs: 97 active+clean; 4.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-25T19:07:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:34 smithi183 ceph-mon[32344]: pgmap v141: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:34 smithi194 ceph-mon[32740]: pgmap v141: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:34 smithi086 ceph-mon[27783]: pgmap v141: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:36 smithi183 ceph-mon[32344]: pgmap v142: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:36 smithi194 ceph-mon[32740]: pgmap v142: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:37.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:36 smithi086 ceph-mon[27783]: pgmap v142: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 -- 172.21.15.86:0/2582308157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 msgr2=0x7f1ce010e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/2582308157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce010e7d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009ca0 tx=0x7f1cd802a950 comp rx=0 tx=0).stop 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 -- 172.21.15.86:0/2582308157 shutdown_connections 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/2582308157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce010e7d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/2582308157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0103830 0x7f1ce010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/2582308157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce0102f10 0x7f1ce01032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 -- 172.21.15.86:0/2582308157 >> 172.21.15.86:0/2582308157 conn(0x7f1ce00fc770 msgr2=0x7f1ce00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 -- 172.21.15.86:0/2582308157 shutdown_connections 2024-04-25T19:07:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.185+0000 7f1ce6e00640 1 -- 172.21.15.86:0/2582308157 wait complete. 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.186+0000 7f1ce6e00640 1 Processor -- start 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.186+0000 7f1ce6e00640 1 -- start start 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0102f10 0x7f1ce0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce0103830 0x7f1ce0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0112010 con 0x7f1ce010c330 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0111e90 con 0x7f1ce0103830 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0112190 con 0x7f1ce0102f10 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40216/0 (socket says 172.21.15.86:40216) 2024-04-25T19:07:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0102f10 0x7f1ce0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6400640 1 -- 172.21.15.86:0/4052650170 learned_addr learned my addr 172.21.15.86:0/4052650170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce5000640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce0103830 0x7f1ce0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6400640 1 -- 172.21.15.86:0/4052650170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0102f10 msgr2=0x7f1ce0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.187+0000 7f1ce6400640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0102f10 0x7f1ce0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce6400640 1 -- 172.21.15.86:0/4052650170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce0103830 msgr2=0x7f1ce0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce6400640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce0103830 0x7f1ce0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce6400640 1 -- 172.21.15.86:0/4052650170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce019d530 con 0x7f1ce010c330 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0102f10 0x7f1ce0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce6400640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce019ce00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009dc0 tx=0x7f1cd8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1cde000640 1 -- 172.21.15.86:0/4052650170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8034040 con 0x7f1ce010c330 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1cde000640 1 -- 172.21.15.86:0/4052650170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd8003e50 con 0x7f1ce010c330 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce019d7c0 con 0x7f1ce010c330 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1cde000640 1 -- 172.21.15.86:0/4052650170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8004600 con 0x7f1ce010c330 2024-04-25T19:07:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.188+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce01a50b0 con 0x7f1ce010c330 2024-04-25T19:07:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1cde000640 1 -- 172.21.15.86:0/4052650170 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1cd8037020 con 0x7f1ce010c330 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 Processor -- start 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 -- start start 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c82dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56282c793900 con 0x56282c83cfa0 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56282c739ef0 con 0x56282c82d8f0 2024-04-25T19:07:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.192+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56282c78f8c0 con 0x56282c82e210 2024-04-25T19:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47862/0 (socket says 172.21.15.86:47862) 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 -- 172.21.15.86:0/400402404 learned_addr learned my addr 172.21.15.86:0/400402404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5000640 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c82dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1cde000640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1cc80a3d90 0x7f1cc80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 -- 172.21.15.86:0/400402404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 msgr2=0x56282c82dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c82dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 -- 172.21.15.86:0/400402404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 msgr2=0x56282c83f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 -- 172.21.15.86:0/400402404 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282c4a6210 con 0x56282c82e210 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5000640 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1cde000640 1 -- 172.21.15.86:0/4052650170 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1cd80f13e0 con 0x7f1ce010c330 2024-04-25T19:07:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce6400640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1cc80a3d90 0x7f1cc80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.193+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83ca60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40115f0 tx=0x7f1cd4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23770f89ad5e81ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.194+0000 7f1cd2a00640 1 -- 172.21.15.86:0/400402404 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd401e070 con 0x56282c82e210 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.194+0000 7f1cd2a00640 1 -- 172.21.15.86:0/400402404 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd400faf0 con 0x56282c82e210 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.194+0000 7f1cd2a00640 1 -- 172.21.15.86:0/400402404 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4019a50 con 0x56282c82e210 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/400402404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 msgr2=0x56282c83ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ce6400640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1cc80a3d90 0x7f1cc80a6250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80eebf0 tx=0x7f1cd80040d0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83ca60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40115f0 tx=0x7f1cd4011ab0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/400402404 shutdown_connections 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83ca60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/400402404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c82dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/400402404 >> 172.21.15.86:0/400402404 conn(0x56282c829ec0 msgr2=0x56282c82a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/400402404 shutdown_connections 2024-04-25T19:07:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/400402404 wait complete. 2024-04-25T19:07:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 -- start start 2024-04-25T19:07:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.195+0000 7f1ceb3d5740 1 Processor -- start 2024-04-25T19:07:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c839f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56282c83fb70 con 0x56282c83cfa0 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56282c74d310 con 0x56282c82d8f0 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56282c83fcf0 con 0x56282c82e210 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ceb3d5740 10 client.0 _ll_register_callbacks cb 0x56282c71bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47872/0 (socket says 172.21.15.86:47872) 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ce5a00640 1 -- 172.21.15.86:0/4232422306 learned_addr learned my addr 172.21.15.86:0/4232422306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ce5000640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c839f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.196+0000 7f1ce5000640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5a00640 1 -- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 msgr2=0x56282c839f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c839f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5a00640 1 -- 172.21.15.86:0/4232422306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 msgr2=0x56282c83a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5a00640 1 -- 172.21.15.86:0/4232422306 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282c572620 con 0x56282c82e210 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5000640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c839f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83a470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40057c0 tx=0x7f1cd400f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a53e21708fc419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1cc7400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1cc7400640 1 -- 172.21.15.86:0/4232422306 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd401e040 con 0x56282c82e210 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd400faf0 con 0x56282c82e210 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1cc7400640 1 -- 172.21.15.86:0/4232422306 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1cd4004040 con 0x56282c82e210 2024-04-25T19:07:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1cc7400640 1 -- 172.21.15.86:0/4232422306 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cd402f050 con 0x56282c82e210 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ceb3d5740 10 client.25825 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.197+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56282c84be80 con 0x56282c82e210 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1cc7400640 1 -- 172.21.15.86:0/4232422306 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd400faf0 con 0x56282c82e210 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1cc7400640 1 -- 172.21.15.86:0/4232422306 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd402a030 con 0x56282c82e210 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1ceb3d5740 10 client.25825 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1ceb3d5740 10 client.25825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1ceb3d5740 20 client.25825 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c84d290 0x56282c84f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1ceb3d5740 4 client.25825 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.198+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56282c7a78b0 con 0x56282c84d290 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.199+0000 7f1ce5000640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c84d290 0x56282c84f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.199+0000 7f1ce5000640 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c84d290 0x56282c84f680 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae6dde44a4b3085 server_cookie=991918f8990ac9ea in_seq=0 out_seq=0 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.199+0000 7f1cc7400640 10 client.25825 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.200+0000 7f1cc7400640 1 -- 172.21.15.86:0/4232422306 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56282c7a78b0 con 0x56282c84d290 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.200+0000 7f1cc7400640 10 client.25825 handle_command_reply: tid=0 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 client.25825 shutdown 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 msgr2=0x56282c83a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83a470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40057c0 tx=0x7f1cd400f8f0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 shutdown_connections 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c84d290 0x56282c84f680 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c83a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c83a470 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1cc7400640 0 client.25825 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/4232422306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c839f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 >> 172.21.15.86:0/4232422306 conn(0x56282c829ec0 msgr2=0x56282c82fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 shutdown_connections 2024-04-25T19:07:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/4232422306 wait complete. 2024-04-25T19:07:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.201+0000 7f1ceb3d5740 20 client.25825 trim_cache size 0 max 16384 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.217+0000 7f1ceb3d5740 1 Processor -- start 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.217+0000 7f1ceb3d5740 1 -- start start 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.217+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c848aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.217+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 0x56282c82fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.217+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c83cfa0 0x56282c854e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01ab910 con 0x56282c82e210 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01ad450 con 0x56282c82d8f0 2024-04-25T19:07:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01abfe0 con 0x56282c83cfa0 2024-04-25T19:07:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 0x56282c82fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 0x56282c82fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40250/0 (socket says 172.21.15.86:40250) 2024-04-25T19:07:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 -- 172.21.15.86:0/312959099 learned_addr learned my addr 172.21.15.86:0/312959099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5000640 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c848aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5000640 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c83cfa0 0x56282c854e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 -- 172.21.15.86:0/312959099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c83cfa0 msgr2=0x56282c854e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c83cfa0 0x56282c854e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 -- 172.21.15.86:0/312959099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 msgr2=0x56282c848aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c848aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5a00640 1 -- 172.21.15.86:0/312959099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282c8384e0 con 0x56282c82e210 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.218+0000 7f1ce5000640 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c848aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 0x56282c82fac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009250 tx=0x7f1cd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a276171b63f832a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1cd2000640 1 -- 172.21.15.86:0/312959099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd401e070 con 0x56282c82e210 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1cd2000640 1 -- 172.21.15.86:0/312959099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd4005c50 con 0x56282c82e210 2024-04-25T19:07:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1cd2000640 1 -- 172.21.15.86:0/312959099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4003850 con 0x56282c82e210 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/312959099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 msgr2=0x56282c82fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 0x56282c82fac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009250 tx=0x7f1cd4009b50 comp rx=0 tx=0).stop 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/312959099 shutdown_connections 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c83cfa0 0x56282c854e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c82e210 0x56282c82fac0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/312959099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c848aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.219+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/312959099 >> 172.21.15.86:0/312959099 conn(0x56282c829ec0 msgr2=0x56282c82f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.220+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/312959099 shutdown_connections 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.220+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/312959099 wait complete. 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.220+0000 7f1ceb3d5740 1 -- start start 2024-04-25T19:07:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.220+0000 7f1ceb3d5740 1 Processor -- start 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c8454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c842830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c842e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01acd80 con 0x56282c83cfa0 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01ac6b0 con 0x56282c82d8f0 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce01a98f0 con 0x56282c82e210 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ceb3d5740 10 client.0 _ll_register_callbacks cb 0x7f1ccc009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c842e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c842830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c842e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40260/0 (socket says 172.21.15.86:40260) 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5000640 1 -- 172.21.15.86:0/3364991639 learned_addr learned my addr 172.21.15.86:0/3364991639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5000640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c8454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5a00640 1 -- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 msgr2=0x56282c8454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c8454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5a00640 1 -- 172.21.15.86:0/3364991639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 msgr2=0x56282c842e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c842e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5a00640 1 -- 172.21.15.86:0/3364991639 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282c843670 con 0x56282c82e210 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.221+0000 7f1ce5000640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c842e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1ce5a00640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c842830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40198a0 tx=0x7f1cd400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce3efe785a709c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1cc7400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1cc7400640 1 -- 172.21.15.86:0/3364991639 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4015040 con 0x56282c82e210 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1cc7400640 1 -- 172.21.15.86:0/3364991639 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1cd4005040 con 0x56282c82e210 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd4005c50 con 0x56282c82e210 2024-04-25T19:07:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1cc7400640 1 -- 172.21.15.86:0/3364991639 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cd4042050 con 0x56282c82e210 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1ceb3d5740 10 client.25831 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.222+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56282c8b37d0 con 0x56282c82e210 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1cc7400640 1 -- 172.21.15.86:0/3364991639 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd4005c50 con 0x56282c82e210 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1cc7400640 1 -- 172.21.15.86:0/3364991639 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1cd403d030 con 0x56282c82e210 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ceb3d5740 10 client.25831 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ceb3d5740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ceb3d5740 20 client.25831 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c8b3e60 0x56282c8b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ceb3d5740 4 client.25831 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56282c7a1320 con 0x56282c8b3e60 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.223+0000 7f1ce5000640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c8b3e60 0x56282c8b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.224+0000 7f1ce5000640 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c8b3e60 0x56282c8b6280 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f600f2b99c1eb1 server_cookie=32e200f75bd3a589 in_seq=0 out_seq=0 2024-04-25T19:07:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.224+0000 7f1cc7400640 10 client.25831 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.239+0000 7f1cc7400640 1 -- 172.21.15.86:0/3364991639 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56282c7a1320 con 0x56282c8b3e60 2024-04-25T19:07:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.239+0000 7f1cc7400640 10 client.25831 handle_command_reply: tid=0 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.239+0000 7f1ceb3d5740 1 client.25831 shutdown 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 msgr2=0x56282c842830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c842830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40198a0 tx=0x7f1cd400f3c0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 shutdown_connections 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56282c8b3e60 0x56282c8b6280 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56282c83cfa0 0x56282c842e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56282c82e210 0x56282c842830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 --2- 172.21.15.86:0/3364991639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56282c82d8f0 0x56282c8454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1cc7400640 0 client.25831 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 >> 172.21.15.86:0/3364991639 conn(0x56282c829ec0 msgr2=0x56282c829020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 shutdown_connections 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 1 -- 172.21.15.86:0/3364991639 wait complete. 2024-04-25T19:07:37.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.240+0000 7f1ceb3d5740 20 client.25831 trim_cache size 0 max 16384 2024-04-25T19:07:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:07:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:07:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:07:37.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1cc80a3d90 msgr2=0x7f1cc80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1cc80a3d90 0x7f1cc80a6250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80eebf0 tx=0x7f1cd80040d0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 msgr2=0x7f1ce019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce019ce00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009dc0 tx=0x7f1cd8003ce0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 shutdown_connections 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1cc80a3d90 0x7f1cc80a6250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce010c330 0x7f1ce019ce00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce0103830 0x7f1ce0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 --2- 172.21.15.86:0/4052650170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ce0102f10 0x7f1ce0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 >> 172.21.15.86:0/4052650170 conn(0x7f1ce00fc770 msgr2=0x7f1ce00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 shutdown_connections 2024-04-25T19:07:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:37.241+0000 7f1ce6e00640 1 -- 172.21.15.86:0/4052650170 wait complete. 2024-04-25T19:07:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:38 smithi183 ceph-mon[32344]: pgmap v143: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:38 smithi194 ceph-mon[32740]: pgmap v143: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:38 smithi086 ceph-mon[27783]: pgmap v143: 97 pgs: 97 active+clean; 4.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:40 smithi183 ceph-mon[32344]: pgmap v144: 97 pgs: 97 active+clean; 4.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 24 op/s 2024-04-25T19:07:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:40 smithi194 ceph-mon[32740]: pgmap v144: 97 pgs: 97 active+clean; 4.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 24 op/s 2024-04-25T19:07:41.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:40 smithi086 ceph-mon[27783]: pgmap v144: 97 pgs: 97 active+clean; 4.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 24 op/s 2024-04-25T19:07:42.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:42 smithi183 ceph-mon[32344]: pgmap v145: 97 pgs: 97 active+clean; 4.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-25T19:07:42.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:42 smithi194 ceph-mon[32740]: pgmap v145: 97 pgs: 97 active+clean; 4.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-25T19:07:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:42 smithi086 ceph-mon[27783]: pgmap v145: 97 pgs: 97 active+clean; 4.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-25T19:07:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:44 smithi194 ceph-mon[32740]: pgmap v146: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-25T19:07:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:44 smithi086 ceph-mon[27783]: pgmap v146: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-25T19:07:45.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:44 smithi183 ceph-mon[32344]: pgmap v146: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-25T19:07:46.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:46 smithi194 ceph-mon[32740]: pgmap v147: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:47.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:46 smithi086 ceph-mon[27783]: pgmap v147: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:47.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:46 smithi183 ceph-mon[32344]: pgmap v147: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:48 smithi194 ceph-mon[32740]: pgmap v148: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:49.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:48 smithi086 ceph-mon[27783]: pgmap v148: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:49.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:48 smithi183 ceph-mon[32344]: pgmap v148: 97 pgs: 97 active+clean; 4.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.383+0000 7fbb27600640 1 -- 172.21.15.86:0/1076696368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb2010e620 msgr2=0x7fbb2010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.383+0000 7fbb27600640 1 --2- 172.21.15.86:0/1076696368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb2010e620 0x7fbb2010ea00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009890 tx=0x7fbb1002a370 comp rx=0 tx=0).stop 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 -- 172.21.15.86:0/1076696368 shutdown_connections 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 --2- 172.21.15.86:0/1076696368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb20108f20 0x7fbb201153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 --2- 172.21.15.86:0/1076696368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb20108560 0x7fbb201089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 --2- 172.21.15.86:0/1076696368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb2010e620 0x7fbb2010ea00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 -- 172.21.15.86:0/1076696368 >> 172.21.15.86:0/1076696368 conn(0x7fbb200764e0 msgr2=0x7fbb20078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 -- 172.21.15.86:0/1076696368 shutdown_connections 2024-04-25T19:07:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.384+0000 7fbb27600640 1 -- 172.21.15.86:0/1076696368 wait complete. 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 Processor -- start 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 -- start start 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb20108560 0x7fbb2019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb20108f20 0x7fbb2019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 0x7fbb201a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201187b0 con 0x7fbb2010e620 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20118630 con 0x7fbb20108560 2024-04-25T19:07:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb27600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20118930 con 0x7fbb20108f20 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb25600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 0x7fbb201a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb25600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 0x7fbb201a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58554/0 (socket says 172.21.15.86:58554) 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb20108f20 0x7fbb2019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb25600640 1 -- 172.21.15.86:0/1537965916 learned_addr learned my addr 172.21.15.86:0/1537965916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.385+0000 7fbb24c00640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb20108560 0x7fbb2019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb17e00640 1 -- 172.21.15.86:0/1537965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb20108560 msgr2=0x7fbb2019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb17e00640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb20108560 0x7fbb2019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb17e00640 1 -- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 msgr2=0x7fbb201a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb17e00640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 0x7fbb201a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb17e00640 1 -- 172.21.15.86:0/1537965916 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201a3d20 con 0x7fbb20108f20 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb25600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 0x7fbb201a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb17e00640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb20108f20 0x7fbb2019f270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e9f0 tx=0x7fbb0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb1e000640 1 -- 172.21.15.86:0/1537965916 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800ccc0 con 0x7fbb20108f20 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb1e000640 1 -- 172.21.15.86:0/1537965916 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08005020 con 0x7fbb20108f20 2024-04-25T19:07:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201a4010 con 0x7fbb20108f20 2024-04-25T19:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.386+0000 7fbb1e000640 1 -- 172.21.15.86:0/1537965916 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08004a00 con 0x7fbb20108f20 2024-04-25T19:07:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.387+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201ab8a0 con 0x7fbb20108f20 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 Processor -- start 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 -- start start 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb1e000640 1 -- 172.21.15.86:0/1537965916 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb08004ba0 con 0x7fbb20108f20 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633d7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565363330900 con 0x5653633cb140 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653632d6ef0 con 0x5653633ca820 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56536332c8c0 con 0x5653633d5a00 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb24c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb25600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633d7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb25600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58196/0 (socket says 172.21.15.86:58196) 2024-04-25T19:07:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb25600640 1 -- 172.21.15.86:0/2759641475 learned_addr learned my addr 172.21.15.86:0/2759641475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.390+0000 7fbb24c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58556/0 (socket says 172.21.15.86:58556) 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb25600640 1 -- 172.21.15.86:0/2759641475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 msgr2=0x5653633d7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb25600640 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633d7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb25600640 1 -- 172.21.15.86:0/2759641475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 msgr2=0x5653633d54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb25600640 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb25600640 1 -- 172.21.15.86:0/2759641475 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565363043210 con 0x5653633ca820 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb25600640 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633cac00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e8d0 tx=0x7fbb1800ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf2feb776974ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.391+0000 7fbb15c00640 1 -- 172.21.15.86:0/2759641475 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800cba0 con 0x5653633ca820 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb15c00640 1 -- 172.21.15.86:0/2759641475 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1800cd40 con 0x5653633ca820 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb15c00640 1 -- 172.21.15.86:0/2759641475 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180057f0 con 0x5653633ca820 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb1e000640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb040a3dc0 0x7fbb040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb1e000640 1 -- 172.21.15.86:0/1537965916 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fbb080d0840 con 0x7fbb20108f20 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2759641475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 msgr2=0x5653633cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633cac00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e8d0 tx=0x7fbb1800ed90 comp rx=0 tx=0).stop 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb24c00640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb040a3dc0 0x7fbb040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2759641475 shutdown_connections 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633d7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2759641475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633cac00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2759641475 >> 172.21.15.86:0/2759641475 conn(0x5653633c6d30 msgr2=0x5653633c7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.392+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2759641475 shutdown_connections 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.393+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2759641475 wait complete. 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.393+0000 7fbb2a3f8740 1 -- start start 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.393+0000 7fbb24c00640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb040a3dc0 0x7fbb040a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb10035040 tx=0x7fbb10030340 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.393+0000 7fbb2a3f8740 1 Processor -- start 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x5653633d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653633d8630 con 0x5653633cb140 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653632ea310 con 0x5653633d5a00 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5653633d87b0 con 0x5653633ca820 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb2a3f8740 10 client.0 _ll_register_callbacks cb 0x5653632b8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633d2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:55226/0 (socket says 172.21.15.86:55226) 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 -- 172.21.15.86:0/3421662509 learned_addr learned my addr 172.21.15.86:0/3421662509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb25600640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x5653633d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 -- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 msgr2=0x5653633d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x5653633d3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 -- 172.21.15.86:0/3421662509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 msgr2=0x5653633d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb24c00640 1 -- 172.21.15.86:0/3421662509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56536310f620 con 0x5653633ca820 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.394+0000 7fbb17e00640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb24c00640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633d2980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb100095f0 tx=0x7fbb1002a370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db5ba4f14dff034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb0e000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb0e000640 1 -- 172.21.15.86:0/3421662509 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10037040 con 0x5653633ca820 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1800cd40 con 0x5653633ca820 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb0e000640 1 -- 172.21.15.86:0/3421662509 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fbb10003a00 con 0x5653633ca820 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb0e000640 1 -- 172.21.15.86:0/3421662509 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb10058050 con 0x5653633ca820 2024-04-25T19:07:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb2a3f8740 10 client.25843 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.395+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653633e48c0 con 0x5653633ca820 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb0e000640 1 -- 172.21.15.86:0/3421662509 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1800cd40 con 0x5653633ca820 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb0e000640 1 -- 172.21.15.86:0/3421662509 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbb1003d040 con 0x5653633ca820 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb2a3f8740 10 client.25843 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb2a3f8740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb2a3f8740 20 client.25843 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653633e5d30 0x5653633e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb2a3f8740 4 client.25843 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.396+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653633448b0 con 0x5653633e5d30 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.397+0000 7fbb17e00640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653633e5d30 0x5653633e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.397+0000 7fbb0e000640 10 client.25843 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.397+0000 7fbb17e00640 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653633e5d30 0x5653633e81c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e071a713097d4e4 server_cookie=322275ab202d3b27 in_seq=0 out_seq=0 2024-04-25T19:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb0e000640 1 -- 172.21.15.86:0/3421662509 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653633448b0 con 0x5653633e5d30 2024-04-25T19:07:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb0e000640 10 client.25843 handle_command_reply: tid=0 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 client.25843 shutdown 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 msgr2=0x5653633d2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633d2980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb100095f0 tx=0x7fbb1002a370 comp rx=0 tx=0).stop 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 shutdown_connections 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653633e5d30 0x5653633e81c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x5653633d3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/3421662509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633d2980 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 >> 172.21.15.86:0/3421662509 conn(0x5653633c6d30 msgr2=0x5653633ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.398+0000 7fbb0e000640 0 client.25843 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.399+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 shutdown_connections 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.399+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/3421662509 wait complete. 2024-04-25T19:07:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.399+0000 7fbb2a3f8740 20 client.25843 trim_cache size 0 max 16384 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.414+0000 7fbb2a3f8740 1 Processor -- start 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 -- start start 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633e14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633edb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b4250 con 0x5653633cb140 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b5d90 con 0x5653633ca820 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b4920 con 0x5653633d5a00 2024-04-25T19:07:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58572/0 (socket says 172.21.15.86:58572) 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 -- 172.21.15.86:0/2719516437 learned_addr learned my addr 172.21.15.86:0/2719516437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb24c00640 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633e14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb25600640 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 -- 172.21.15.86:0/2719516437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 msgr2=0x5653633edb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633edb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 -- 172.21.15.86:0/2719516437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 msgr2=0x5653633e14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.415+0000 7fbb17e00640 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633e14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.416+0000 7fbb17e00640 1 -- 172.21.15.86:0/2719516437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653633c2690 con 0x5653633cb140 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.416+0000 7fbb17e00640 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633c5d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb08099100 tx=0x7fbb080d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de808e3219fbf10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.416+0000 7fbb15200640 1 -- 172.21.15.86:0/2719516437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0801c040 con 0x5653633cb140 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.416+0000 7fbb15200640 1 -- 172.21.15.86:0/2719516437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0809ce30 con 0x5653633cb140 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.416+0000 7fbb15200640 1 -- 172.21.15.86:0/2719516437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0809a7d0 con 0x5653633cb140 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2719516437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 msgr2=0x5653633c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633c5d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb08099100 tx=0x7fbb080d2040 comp rx=0 tx=0).stop 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2719516437 shutdown_connections 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633d5a00 0x5653633edb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633c5d10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/2719516437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633ca820 0x5653633e14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2719516437 >> 172.21.15.86:0/2719516437 conn(0x5653633c6d30 msgr2=0x5653633c5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2719516437 shutdown_connections 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/2719516437 wait complete. 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 -- start start 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.417+0000 7fbb2a3f8740 1 Processor -- start 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x56536344b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b56c0 con 0x5653633cb140 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b4ff0 con 0x5653633d5a00 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b2230 con 0x5653633ca820 2024-04-25T19:07:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb2a3f8740 10 client.0 _ll_register_callbacks cb 0x7fbb10004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633dbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58586/0 (socket says 172.21.15.86:58586) 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb17e00640 1 -- 172.21.15.86:0/702780807 learned_addr learned my addr 172.21.15.86:0/702780807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb24c00640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.418+0000 7fbb25600640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x56536344b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb17e00640 1 -- 172.21.15.86:0/702780807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 msgr2=0x5653633db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb17e00640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633db850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb17e00640 1 -- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 msgr2=0x56536344b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb17e00640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x56536344b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb17e00640 1 -- 172.21.15.86:0/702780807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56536344b8e0 con 0x5653633cb140 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb17e00640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633dbd90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb080990b0 tx=0x7fbb080cde80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c29a231f223827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb0e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb0e000640 1 -- 172.21.15.86:0/702780807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0801c070 con 0x5653633cb140 2024-04-25T19:07:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb0e000640 1 -- 172.21.15.86:0/702780807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fbb080051d0 con 0x5653633cb140 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0809ce30 con 0x5653633cb140 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb0e000640 1 -- 172.21.15.86:0/702780807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb0809c730 con 0x5653633cb140 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb2a3f8740 10 client.15756 fetch_fsmap learned FSMap version 29 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.419+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56536344cab0 con 0x5653633cb140 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb0e000640 1 -- 172.21.15.86:0/702780807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0809ce30 con 0x5653633cb140 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb0e000640 1 -- 172.21.15.86:0/702780807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbb080ce1f0 con 0x5653633cb140 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb2a3f8740 10 client.15756 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb2a3f8740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb2a3f8740 20 client.15756 populate_metadata read hostname 'smithi086' 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56536344d200 0x56536344f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb2a3f8740 4 client.15756 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56536333e320 con 0x56536344d200 2024-04-25T19:07:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.420+0000 7fbb24c00640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56536344d200 0x56536344f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.421+0000 7fbb24c00640 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56536344d200 0x56536344f620 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af820a62470bc13 server_cookie=aa1b17ff891f6a08 in_seq=0 out_seq=0 2024-04-25T19:07:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.421+0000 7fbb0e000640 10 client.15756 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb0e000640 1 -- 172.21.15.86:0/702780807 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56536333e320 con 0x56536344d200 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb0e000640 10 client.15756 handle_command_reply: tid=0 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 client.15756 shutdown 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 msgr2=0x5653633dbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633dbd90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb080990b0 tx=0x7fbb080cde80 comp rx=0 tx=0).stop 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 shutdown_connections 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56536344d200 0x56536344f620 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653633d5a00 0x56536344b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653633cb140 0x5653633dbd90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 --2- 172.21.15.86:0/702780807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653633ca820 0x5653633db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb0e000640 0 client.15756 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 >> 172.21.15.86:0/702780807 conn(0x5653633c6d30 msgr2=0x5653633d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 shutdown_connections 2024-04-25T19:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 1 -- 172.21.15.86:0/702780807 wait complete. 2024-04-25T19:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.437+0000 7fbb2a3f8740 20 client.15756 trim_cache size 0 max 16384 2024-04-25T19:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:07:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb040a3dc0 msgr2=0x7fbb040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb040a3dc0 0x7fbb040a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb10035040 tx=0x7fbb10030340 comp rx=0 tx=0).stop 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb20108f20 msgr2=0x7fbb2019f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb20108f20 0x7fbb2019f270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e9f0 tx=0x7fbb0800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 shutdown_connections 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb040a3dc0 0x7fbb040a6280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb2010e620 0x7fbb201a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb20108f20 0x7fbb2019f270 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 --2- 172.21.15.86:0/1537965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb20108560 0x7fbb2019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 >> 172.21.15.86:0/1537965916 conn(0x7fbb200764e0 msgr2=0x7fbb20077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 shutdown_connections 2024-04-25T19:07:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:07:50.438+0000 7fbb27600640 1 -- 172.21.15.86:0/1537965916 wait complete. 2024-04-25T19:07:50.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:50 smithi194 ceph-mon[32740]: pgmap v149: 97 pgs: 97 active+clean; 4.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:50 smithi086 ceph-mon[27783]: pgmap v149: 97 pgs: 97 active+clean; 4.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:51.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:50 smithi183 ceph-mon[32344]: pgmap v149: 97 pgs: 97 active+clean; 4.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:52 smithi194 ceph-mon[32740]: pgmap v150: 97 pgs: 97 active+clean; 4.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-25T19:07:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:52 smithi086 ceph-mon[27783]: pgmap v150: 97 pgs: 97 active+clean; 4.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-25T19:07:53.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:52 smithi183 ceph-mon[32344]: pgmap v150: 97 pgs: 97 active+clean; 4.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-25T19:07:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:54 smithi194 ceph-mon[32740]: pgmap v151: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:54 smithi086 ceph-mon[27783]: pgmap v151: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:55.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:54 smithi183 ceph-mon[32344]: pgmap v151: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T19:07:56.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:56 smithi194 ceph-mon[32740]: pgmap v152: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:57.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:56 smithi086 ceph-mon[27783]: pgmap v152: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:57.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:56 smithi183 ceph-mon[32344]: pgmap v152: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:64/600: open fd = 3 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:65/600: open fd = 3 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:66/600: open fd = 3 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:67/600: open fd = 3 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:68/600: open fd = 3 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:69/600: open fd = 3 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:70/600: open fd = 3 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:71/600: open fd = 3 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:72/600: open fd = 3 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:73/600: open fd = 3 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:74/600: open fd = 3 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:75/600: open fd = 3 2024-04-25T19:07:57.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:76/600: open fd = 3 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:77/600: open fd = 3 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.558 INFO:tasks.workunit.client.0.smithi086.stdout:78/600: open fd = 3 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:79/600: open fd = 3 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:80/600: open fd = 3 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:81/600: open fd = 3 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:82/600: open fd = 3 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:83/600: open fd = 3 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:84/600: open fd = 3 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.560 INFO:tasks.workunit.client.0.smithi086.stdout:85/600: open fd = 3 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:86/600: open fd = 3 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:87/600: open fd = 3 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:88/600: open fd = 3 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.561 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:89/600: open fd = 3 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:90/600: open fd = 3 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:91/600: open fd = 3 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:92/600: open fd = 3 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.562 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:93/600: open fd = 3 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:94/600: open fd = 3 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:95/600: open fd = 3 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:96/600: open fd = 3 2024-04-25T19:07:57.563 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:97/600: open fd = 3 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:98/600: open fd = 3 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:99/600: open fd = 3 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:100/600: open fd = 3 2024-04-25T19:07:57.564 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:101/600: open fd = 3 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:102/600: open fd = 3 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:103/600: open fd = 3 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.565 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:104/600: open fd = 3 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:105/600: open fd = 3 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:106/600: open fd = 3 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:107/600: open fd = 3 2024-04-25T19:07:57.566 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:108/600: open fd = 3 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:109/600: open fd = 3 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:110/600: open fd = 3 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:111/600: open fd = 3 2024-04-25T19:07:57.567 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:112/600: open fd = 3 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:113/600: open fd = 3 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:114/600: open fd = 3 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.568 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:115/600: open fd = 3 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:116/600: open fd = 3 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:117/600: open fd = 3 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:118/600: open fd = 3 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.569 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:119/600: open fd = 3 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:120/600: open fd = 3 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:121/600: open fd = 3 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.570 INFO:tasks.workunit.client.0.smithi086.stdout:122/600: open fd = 3 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:123/600: open fd = 3 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:124/600: open fd = 3 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:125/600: open fd = 3 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:57.571 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:07:57.572 INFO:tasks.workunit.client.0.smithi086.stdout:126/600: open fd = 3 2024-04-25T19:07:57.572 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:07:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:07:58 smithi194 ceph-mon[32740]: pgmap v153: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:07:58 smithi086 ceph-mon[27783]: pgmap v153: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:07:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:07:58 smithi183 ceph-mon[32344]: pgmap v153: 97 pgs: 97 active+clean; 4.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:08:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:00 smithi194 ceph-mon[32740]: pgmap v154: 97 pgs: 97 active+clean; 5.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-25T19:08:01.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:00 smithi086 ceph-mon[27783]: pgmap v154: 97 pgs: 97 active+clean; 5.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-25T19:08:01.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:00 smithi183 ceph-mon[32344]: pgmap v154: 97 pgs: 97 active+clean; 5.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-04-25T19:08:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:02 smithi194 ceph-mon[32740]: pgmap v155: 97 pgs: 97 active+clean; 5.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:08:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:02 smithi086 ceph-mon[27783]: pgmap v155: 97 pgs: 97 active+clean; 5.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:08:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:02 smithi183 ceph-mon[32344]: pgmap v155: 97 pgs: 97 active+clean; 5.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-25T19:08:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.581+0000 7fecf2000640 1 -- 172.21.15.86:0/253695881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec108f20 msgr2=0x7fecec1153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.581+0000 7fecf2000640 1 --2- 172.21.15.86:0/253695881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec108f20 0x7fecec1153e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fece4009ca0 tx=0x7fece402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 -- 172.21.15.86:0/253695881 shutdown_connections 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 --2- 172.21.15.86:0/253695881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec108f20 0x7fecec1153e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 --2- 172.21.15.86:0/253695881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecec108560 0x7fecec1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 --2- 172.21.15.86:0/253695881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec10e620 0x7fecec10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 -- 172.21.15.86:0/253695881 >> 172.21.15.86:0/253695881 conn(0x7fecec0764e0 msgr2=0x7fecec078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 -- 172.21.15.86:0/253695881 shutdown_connections 2024-04-25T19:08:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.582+0000 7fecf2000640 1 -- 172.21.15.86:0/253695881 wait complete. 2024-04-25T19:08:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 Processor -- start 2024-04-25T19:08:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 -- start start 2024-04-25T19:08:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecec108560 0x7fecec1a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec108f20 0x7fecec1a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec10e620 0x7fecec19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecec118800 con 0x7fecec10e620 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecec118680 con 0x7fecec108560 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.583+0000 7fecf2000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fecec118980 con 0x7fecec108f20 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fecebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec10e620 0x7fecec19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec108f20 0x7fecec1a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fecebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec10e620 0x7fecec19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48436/0 (socket says 172.21.15.86:48436) 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7feceb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecec108560 0x7fecec1a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fecebe00640 1 -- 172.21.15.86:0/267428712 learned_addr learned my addr 172.21.15.86:0/267428712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 -- 172.21.15.86:0/267428712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecec108560 msgr2=0x7fecec1a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecec108560 0x7fecec1a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 -- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec10e620 msgr2=0x7fecec19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec10e620 0x7fecec19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 -- 172.21.15.86:0/267428712 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec19f1a0 con 0x7fecec108f20 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.584+0000 7fece2a00640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec108f20 0x7fecec1a4db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e9f0 tx=0x7fecd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.585+0000 7fece8c00640 1 -- 172.21.15.86:0/267428712 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd400ccc0 con 0x7fecec108f20 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.585+0000 7fece8c00640 1 -- 172.21.15.86:0/267428712 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd40108e0 con 0x7fecec108f20 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.585+0000 7fece8c00640 1 -- 172.21.15.86:0/267428712 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4018880 con 0x7fecec108f20 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.585+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec19f490 con 0x7fecec108f20 2024-04-25T19:08:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.585+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1ab730 con 0x7fecec108f20 2024-04-25T19:08:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 Processor -- start 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 -- start start 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fece8c00640 1 -- 172.21.15.86:0/267428712 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fecd4010430 con 0x7fecec108f20 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x557022017c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 0x5570220224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022024df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557021f7d900 con 0x557022018140 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557021f23ef0 con 0x557022022a00 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.588+0000 7fecf4f66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557021f798c0 con 0x557022017820 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 0x5570220224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 0x5570220224c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48446/0 (socket says 172.21.15.86:48446) 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 -- 172.21.15.86:0/3640424483 learned_addr learned my addr 172.21.15.86:0/3640424483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7fecebe00640 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x557022017c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7fece2a00640 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022024df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 -- 172.21.15.86:0/3640424483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 msgr2=0x557022017c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x557022017c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 -- 172.21.15.86:0/3640424483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 msgr2=0x557022024df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022024df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7feceb400640 1 -- 172.21.15.86:0/3640424483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557021c90210 con 0x557022018140 2024-04-25T19:08:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.589+0000 7fece2a00640 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022024df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7feceb400640 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 0x5570220224c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009c20 tx=0x7fecdc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf31dafd677c6acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7fecdbe00640 1 -- 172.21.15.86:0/3640424483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc004230 con 0x557022018140 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7fecdbe00640 1 -- 172.21.15.86:0/3640424483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecdc0043d0 con 0x557022018140 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7fecdbe00640 1 -- 172.21.15.86:0/3640424483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc004d40 con 0x557022018140 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7fece8c00640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fecd00a3dc0 0x7fecd00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7fece8c00640 1 -- 172.21.15.86:0/267428712 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fecd40d0bb0 con 0x7fecec108f20 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.590+0000 7fecebe00640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fecd00a3dc0 0x7fecd00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 -- 172.21.15.86:0/3640424483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 msgr2=0x5570220224c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 0x5570220224c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009c20 tx=0x7fecdc011810 comp rx=0 tx=0).stop 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 -- 172.21.15.86:0/3640424483 shutdown_connections 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022024df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022018140 0x5570220224c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 --2- 172.21.15.86:0/3640424483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x557022017c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 -- 172.21.15.86:0/3640424483 >> 172.21.15.86:0/3640424483 conn(0x557022013d30 msgr2=0x557022014110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 -- 172.21.15.86:0/3640424483 shutdown_connections 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecebe00640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fecd00a3dc0 0x7fecd00a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fece4009a60 tx=0x7fece4005d80 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 -- 172.21.15.86:0/3640424483 wait complete. 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.591+0000 7fecf4f66740 1 -- start start 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 Processor -- start 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702201f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x55702201ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x557022020440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557022025630 con 0x557022022a00 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557021f37310 con 0x557022018140 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5570220257b0 con 0x557022017820 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecf4f66740 10 client.0 _ll_register_callbacks cb 0x557021f05cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7fecebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x557022020440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.592+0000 7feceb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702201f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7fecebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x557022020440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48462/0 (socket says 172.21.15.86:48462) 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702201f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51740/0 (socket says 172.21.15.86:51740) 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7fecebe00640 1 -- 172.21.15.86:0/771074801 learned_addr learned my addr 172.21.15.86:0/771074801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7fece2a00640 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x55702201ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 -- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 msgr2=0x55702201ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x55702201ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 -- 172.21.15.86:0/771074801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 msgr2=0x557022020440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x557022020440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 -- 172.21.15.86:0/771074801 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557021d5c620 con 0x557022017820 2024-04-25T19:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7feceb400640 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702201f9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fecdc011d10 tx=0x7fecdc019b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e03b528f2f17c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.593+0000 7fecd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecd8c00640 1 -- 172.21.15.86:0/771074801 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00e7c0 con 0x557022017820 2024-04-25T19:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecd8c00640 1 -- 172.21.15.86:0/771074801 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fecdc00e960 con 0x557022017820 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecdc0043d0 con 0x557022017820 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecd8c00640 1 -- 172.21.15.86:0/771074801 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fecdc032050 con 0x557022017820 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecf4f66740 10 client.25861 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570220318c0 con 0x557022017820 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecd8c00640 1 -- 172.21.15.86:0/771074801 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecdc0043d0 con 0x557022017820 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.594+0000 7fecd8c00640 1 -- 172.21.15.86:0/771074801 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fecdc018070 con 0x557022017820 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecf4f66740 10 client.25861 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecf4f66740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecf4f66740 20 client.25861 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecf4f66740 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557022032cc0 0x5570220350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecf4f66740 4 client.25861 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557021f918b0 con 0x557022032cc0 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fece2a00640 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557022032cc0 0x5570220350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fece2a00640 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557022032cc0 0x5570220350b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329bbfa8d4199910 server_cookie=e0229f6501967a3d in_seq=0 out_seq=0 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.595+0000 7fecd8c00640 10 client.25861 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.596+0000 7fecd8c00640 1 -- 172.21.15.86:0/771074801 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557021f918b0 con 0x557022032cc0 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.596+0000 7fecd8c00640 10 client.25861 handle_command_reply: tid=0 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 client.25861 shutdown 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 msgr2=0x55702201f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702201f9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fecdc011d10 tx=0x7fecdc019b00 comp rx=0 tx=0).stop 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 shutdown_connections 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557022032cc0 0x5570220350b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x557022020440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x55702201ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 --2- 172.21.15.86:0/771074801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702201f9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 >> 172.21.15.86:0/771074801 conn(0x557022013d30 msgr2=0x557022019a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecd8c00640 0 client.25861 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 shutdown_connections 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 1 -- 172.21.15.86:0/771074801 wait complete. 2024-04-25T19:08:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.597+0000 7fecf4f66740 20 client.25861 trim_cache size 0 max 16384 2024-04-25T19:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.613+0000 7fecf4f66740 1 Processor -- start 2024-04-25T19:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.613+0000 7fecf4f66740 1 -- start start 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702202e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x5570220198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x55702203a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecf4f66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b4270 con 0x557022022a00 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecf4f66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b5db0 con 0x557022018140 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecf4f66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b4940 con 0x557022017820 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7feceb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702202e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fece2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x5570220198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7fecebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x55702203a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7feceb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702202e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51756/0 (socket says 172.21.15.86:51756) 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.614+0000 7feceb400640 1 -- 172.21.15.86:0/4276965015 learned_addr learned my addr 172.21.15.86:0/4276965015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7feceb400640 1 -- 172.21.15.86:0/4276965015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 msgr2=0x5570220198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7feceb400640 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x5570220198e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7feceb400640 1 -- 172.21.15.86:0/4276965015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 msgr2=0x55702203a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7feceb400640 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x55702203a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7feceb400640 1 -- 172.21.15.86:0/4276965015 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570220251b0 con 0x557022017820 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7fecebe00640 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x55702203a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7feceb400640 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702202e4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0059f0 tx=0x7fecdc019860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46964088ca75cff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7fecdb400640 1 -- 172.21.15.86:0/4276965015 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc03b070 con 0x557022017820 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7fecdb400640 1 -- 172.21.15.86:0/4276965015 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecdc029070 con 0x557022017820 2024-04-25T19:08:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.615+0000 7fecdb400640 1 -- 172.21.15.86:0/4276965015 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0248d0 con 0x557022017820 2024-04-25T19:08:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 -- 172.21.15.86:0/4276965015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 msgr2=0x55702202e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702202e4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0059f0 tx=0x7fecdc019860 comp rx=0 tx=0).stop 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 -- 172.21.15.86:0/4276965015 shutdown_connections 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022022a00 0x55702203a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022018140 0x5570220198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 --2- 172.21.15.86:0/4276965015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022017820 0x55702202e4f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 -- 172.21.15.86:0/4276965015 >> 172.21.15.86:0/4276965015 conn(0x557022013d30 msgr2=0x5570220191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 -- 172.21.15.86:0/4276965015 shutdown_connections 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 -- 172.21.15.86:0/4276965015 wait complete. 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.616+0000 7fecf4f66740 1 -- start start 2024-04-25T19:08:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 Processor -- start 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022017820 0x55702202af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022018140 0x5570220282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022028a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b56e0 con 0x557022017820 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b5010 con 0x557022022a00 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b2250 con 0x557022018140 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecf4f66740 10 client.0 _ll_register_callbacks cb 0x7fecdc019790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022028a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7feceb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022017820 0x55702202af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022028a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37530/0 (socket says 172.21.15.86:37530) 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fecebe00640 1 -- 172.21.15.86:0/1856073513 learned_addr learned my addr 172.21.15.86:0/1856073513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.617+0000 7fece2a00640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022018140 0x5570220282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecebe00640 1 -- 172.21.15.86:0/1856073513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022018140 msgr2=0x5570220282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecebe00640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022018140 0x5570220282d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecebe00640 1 -- 172.21.15.86:0/1856073513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022017820 msgr2=0x55702202af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecebe00640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022017820 0x55702202af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecebe00640 1 -- 172.21.15.86:0/1856073513 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557022098070 con 0x557022022a00 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7feceb400640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022017820 0x55702202af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecebe00640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022028a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fece40316f0 tx=0x7fece402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53652720de7e834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecd8c00640 1 -- 172.21.15.86:0/1856073513 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece402f610 con 0x557022022a00 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.618+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecdc029070 con 0x557022022a00 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecd8c00640 1 -- 172.21.15.86:0/1856073513 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fece4042030 con 0x557022022a00 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecd8c00640 1 -- 172.21.15.86:0/1856073513 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fece402f7b0 con 0x557022022a00 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 10 client.25871 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557022099270 con 0x557022022a00 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecd8c00640 1 -- 172.21.15.86:0/1856073513 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecdc029070 con 0x557022022a00 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecd8c00640 1 -- 172.21.15.86:0/1856073513 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fece4030a90 con 0x557022022a00 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 10 client.25871 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 20 client.25871 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570220999f0 0x55702209be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 4 client.25871 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.619+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557021f8b320 con 0x5570220999f0 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.620+0000 7feceb400640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570220999f0 0x55702209be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.620+0000 7feceb400640 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570220999f0 0x55702209be10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110c03031303bb01 server_cookie=8ef1c6fd8ab99bf2 in_seq=0 out_seq=0 2024-04-25T19:08:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.620+0000 7fecd8c00640 10 client.25871 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecd8c00640 1 -- 172.21.15.86:0/1856073513 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557021f8b320 con 0x5570220999f0 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecd8c00640 10 client.25871 handle_command_reply: tid=0 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 client.25871 shutdown 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 msgr2=0x557022028a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022028a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fece40316f0 tx=0x7fece402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 shutdown_connections 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570220999f0 0x55702209be10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557022022a00 0x557022028a10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557022018140 0x5570220282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecd8c00640 0 client.25871 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 --2- 172.21.15.86:0/1856073513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557022017820 0x55702202af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 >> 172.21.15.86:0/1856073513 conn(0x557022013d30 msgr2=0x557022019a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 shutdown_connections 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.635+0000 7fecf4f66740 1 -- 172.21.15.86:0/1856073513 wait complete. 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf4f66740 20 client.25871 trim_cache size 0 max 16384 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:08:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:08:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:08:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fecd00a3dc0 msgr2=0x7fecd00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fecd00a3dc0 0x7fecd00a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fece4009a60 tx=0x7fece4005d80 comp rx=0 tx=0).stop 2024-04-25T19:08:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec108f20 msgr2=0x7fecec1a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec108f20 0x7fecec1a4db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e9f0 tx=0x7fecd400eeb0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 shutdown_connections 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fecd00a3dc0 0x7fecd00a6280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecec10e620 0x7fecec19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecec108f20 0x7fecec1a4db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 --2- 172.21.15.86:0/267428712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecec108560 0x7fecec1a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.636+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 >> 172.21.15.86:0/267428712 conn(0x7fecec0764e0 msgr2=0x7fecec077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.637+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 shutdown_connections 2024-04-25T19:08:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:03.637+0000 7fecf2000640 1 -- 172.21.15.86:0/267428712 wait complete. 2024-04-25T19:08:04.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:04 smithi194 ceph-mon[32740]: pgmap v156: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-25T19:08:05.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:04 smithi086 ceph-mon[27783]: pgmap v156: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-25T19:08:05.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:04 smithi183 ceph-mon[32344]: pgmap v156: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-25T19:08:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:06 smithi194 ceph-mon[32740]: pgmap v157: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-04-25T19:08:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:06 smithi086 ceph-mon[27783]: pgmap v157: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-04-25T19:08:07.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:06 smithi183 ceph-mon[32344]: pgmap v157: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-04-25T19:08:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:08 smithi194 ceph-mon[32740]: pgmap v158: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-04-25T19:08:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:08 smithi086 ceph-mon[27783]: pgmap v158: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-04-25T19:08:09.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:08 smithi183 ceph-mon[32344]: pgmap v158: 97 pgs: 97 active+clean; 5.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-04-25T19:08:11.073 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:10 smithi086 ceph-mon[27783]: pgmap v159: 97 pgs: 97 active+clean; 5.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-25T19:08:11.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:10 smithi183 ceph-mon[32344]: pgmap v159: 97 pgs: 97 active+clean; 5.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-25T19:08:11.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:10 smithi194 ceph-mon[32740]: pgmap v159: 97 pgs: 97 active+clean; 5.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-25T19:08:12.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:11 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:08:12.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:11 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:08:12.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:11 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:08:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:12 smithi086 ceph-mon[27783]: pgmap v160: 97 pgs: 97 active+clean; 5.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 KiB/s wr, 30 op/s 2024-04-25T19:08:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:12 smithi183 ceph-mon[32344]: pgmap v160: 97 pgs: 97 active+clean; 5.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 KiB/s wr, 30 op/s 2024-04-25T19:08:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:12 smithi194 ceph-mon[32740]: pgmap v160: 97 pgs: 97 active+clean; 5.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 KiB/s wr, 30 op/s 2024-04-25T19:08:14.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:08:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:08:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:13 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:08:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:13 smithi086 ceph-mon[27783]: pgmap v161: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-04-25T19:08:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:08:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:08:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:13 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:08:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:13 smithi183 ceph-mon[32344]: pgmap v161: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-04-25T19:08:14.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:08:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:08:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:13 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:08:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:13 smithi194 ceph-mon[32740]: pgmap v161: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-04-25T19:08:16.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:16 smithi086 ceph-mon[27783]: pgmap v162: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:08:16.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:16 smithi183 ceph-mon[32344]: pgmap v162: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:08:16.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:16 smithi194 ceph-mon[32740]: pgmap v162: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.778+0000 7f1f1f200640 1 -- 172.21.15.86:0/3630760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108e10 msgr2=0x7f1f20115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.778+0000 7f1f1f200640 1 --2- 172.21.15.86:0/3630760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108e10 0x7f1f20115250 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009930 tx=0x7f1f1802a420 comp rx=0 tx=0).stop 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 -- 172.21.15.86:0/3630760679 shutdown_connections 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 --2- 172.21.15.86:0/3630760679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108e10 0x7f1f20115250 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 --2- 172.21.15.86:0/3630760679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f20108490 0x7f1f201088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 --2- 172.21.15.86:0/3630760679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2010e420 0x7f1f2010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 -- 172.21.15.86:0/3630760679 >> 172.21.15.86:0/3630760679 conn(0x7f1f20077d30 msgr2=0x7f1f20106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 -- 172.21.15.86:0/3630760679 shutdown_connections 2024-04-25T19:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.779+0000 7f1f1f200640 1 -- 172.21.15.86:0/3630760679 wait complete. 2024-04-25T19:08:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.780+0000 7f1f1f200640 1 Processor -- start 2024-04-25T19:08:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.780+0000 7f1f1f200640 1 -- start start 2024-04-25T19:08:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.780+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 0x7f1f201a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.780+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f20108e10 0x7f1f201a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.780+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2010e420 0x7f1f2019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20118670 con 0x7f1f20108490 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201184f0 con 0x7f1f2010e420 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201187f0 con 0x7f1f20108e10 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 0x7f1f201a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 0x7f1f201a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60214/0 (socket says 172.21.15.86:60214) 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 -- 172.21.15.86:0/2992354880 learned_addr learned my addr 172.21.15.86:0/2992354880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f20108e10 0x7f1f201a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1e800640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2010e420 0x7f1f2019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 -- 172.21.15.86:0/2992354880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f20108e10 msgr2=0x7f1f201a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f20108e10 0x7f1f201a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 -- 172.21.15.86:0/2992354880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2010e420 msgr2=0x7f1f2019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2010e420 0x7f1f2019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 -- 172.21.15.86:0/2992354880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2019f110 con 0x7f1f20108490 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.781+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 0x7f1f201a47e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000eb90 tx=0x7f1f1000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.782+0000 7f1f16a00640 1 -- 172.21.15.86:0/2992354880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10004d20 con 0x7f1f20108490 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.782+0000 7f1f16a00640 1 -- 172.21.15.86:0/2992354880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1000ee90 con 0x7f1f20108490 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.782+0000 7f1f16a00640 1 -- 172.21.15.86:0/2992354880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10010670 con 0x7f1f20108490 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.782+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2019f400 con 0x7f1f20108490 2024-04-25T19:08:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.782+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f201ab6a0 con 0x7f1f20108490 2024-04-25T19:08:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.785+0000 7f1f16a00640 1 -- 172.21.15.86:0/2992354880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f10005020 con 0x7f1f20108490 2024-04-25T19:08:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.785+0000 7f1f27c16740 1 Processor -- start 2024-04-25T19:08:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.785+0000 7f1f27c16740 1 -- start start 2024-04-25T19:08:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db96cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96d2d0 0x55a1db977650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db977b90 0x55a1db979f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f27c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1db8d2900 con 0x55a1db977b90 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f27c16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1db878ef0 con 0x55a1db96d2d0 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f27c16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a1db8ce8c0 con 0x55a1db96c9b0 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db977b90 0x55a1db979f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f1de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96d2d0 0x55a1db977650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f1de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96d2d0 0x55a1db977650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:39240/0 (socket says 172.21.15.86:39240) 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f1de00640 1 -- 172.21.15.86:0/2402927044 learned_addr learned my addr 172.21.15.86:0/2402927044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.786+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db96cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1d400640 1 -- 172.21.15.86:0/2402927044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 msgr2=0x55a1db96cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db96cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1d400640 1 -- 172.21.15.86:0/2402927044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96d2d0 msgr2=0x55a1db977650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96d2d0 0x55a1db977650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1d400640 1 -- 172.21.15.86:0/2402927044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1db5e5210 con 0x55a1db977b90 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db977b90 0x55a1db979f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f201a0180 tx=0x7f1f0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59485555d2950f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f16a00640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1f040a3d10 0x7f1f040a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f0b400640 1 -- 172.21.15.86:0/2402927044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c004230 con 0x55a1db977b90 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f16a00640 1 -- 172.21.15.86:0/2992354880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1f100d0f20 con 0x7f1f20108490 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f0b400640 1 -- 172.21.15.86:0/2402927044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c009e10 con 0x55a1db977b90 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f0b400640 1 -- 172.21.15.86:0/2402927044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c004c60 con 0x55a1db977b90 2024-04-25T19:08:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.787+0000 7f1f1e800640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1f040a3d10 0x7f1f040a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 -- 172.21.15.86:0/2402927044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db977b90 msgr2=0x55a1db979f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f1e800640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1f040a3d10 0x7f1f040a61d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f18006010 tx=0x7f1f18005b50 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db977b90 0x55a1db979f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f201a0180 tx=0x7f1f0c011810 comp rx=0 tx=0).stop 2024-04-25T19:08:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 -- 172.21.15.86:0/2402927044 shutdown_connections 2024-04-25T19:08:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db977b90 0x55a1db979f80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96d2d0 0x55a1db977650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2402927044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db96cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 -- 172.21.15.86:0/2402927044 >> 172.21.15.86:0/2402927044 conn(0x55a1db968ec0 msgr2=0x55a1db9692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 -- 172.21.15.86:0/2402927044 shutdown_connections 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.788+0000 7f1f27c16740 1 -- 172.21.15.86:0/2402927044 wait complete. 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.789+0000 7f1f27c16740 1 -- start start 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.789+0000 7f1f27c16740 1 Processor -- start 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.789+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db974a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.789+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db974fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db977b90 0x55a1db9754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f27c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1db97a7c0 con 0x55a1db96d2d0 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f27c16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1db88c310 con 0x55a1db977b90 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f27c16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a1db97a940 con 0x55a1db96c9b0 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f27c16740 10 client.0 _ll_register_callbacks cb 0x55a1db85acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db974fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db974fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60226/0 (socket says 172.21.15.86:60226) 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 -- 172.21.15.86:0/2203238383 learned_addr learned my addr 172.21.15.86:0/2203238383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db974a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1e800640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db977b90 0x55a1db9754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 -- 172.21.15.86:0/2203238383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 msgr2=0x55a1db974a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db974a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 -- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db977b90 msgr2=0x55a1db9754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db977b90 0x55a1db9754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1d400640 1 -- 172.21.15.86:0/2203238383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1db6b1620 con 0x55a1db96d2d0 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.790+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db974a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1f1d400640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db974fb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005890 tx=0x7f1f0c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccc3763b3634fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1effe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:08:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1effe00640 1 -- 172.21.15.86:0/2203238383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c027020 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1effe00640 1 -- 172.21.15.86:0/2203238383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1f0c003c40 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0c009e10 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1effe00640 1 -- 172.21.15.86:0/2203238383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1f0c032050 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1f27c16740 10 client.15786 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1db9869a0 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.791+0000 7f1effe00640 1 -- 172.21.15.86:0/2203238383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0c009e10 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1effe00640 1 -- 172.21.15.86:0/2203238383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1f0c018070 con 0x55a1db96d2d0 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f27c16740 10 client.15786 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f27c16740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f27c16740 20 client.15786 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db987da0 0x55a1db98a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f27c16740 4 client.15786 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1db8e68b0 con 0x55a1db987da0 2024-04-25T19:08:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.792+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db987da0 0x55a1db98a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.793+0000 7f1f1de00640 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db987da0 0x55a1db98a190 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0074e9f1f64d90 server_cookie=6a5fcfc94707a075 in_seq=0 out_seq=0 2024-04-25T19:08:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.793+0000 7f1effe00640 10 client.15786 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1effe00640 1 -- 172.21.15.86:0/2203238383 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1db8e68b0 con 0x55a1db987da0 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1effe00640 10 client.15786 handle_command_reply: tid=0 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 client.15786 shutdown 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 msgr2=0x55a1db974fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db974fb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005890 tx=0x7f1f0c003820 comp rx=0 tx=0).stop 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 shutdown_connections 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db987da0 0x55a1db98a190 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db977b90 0x55a1db9754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db974fb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 --2- 172.21.15.86:0/2203238383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db974a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 >> 172.21.15.86:0/2203238383 conn(0x55a1db968ec0 msgr2=0x55a1db96ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1effe00640 0 client.15786 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 shutdown_connections 2024-04-25T19:08:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.794+0000 7f1f27c16740 1 -- 172.21.15.86:0/2203238383 wait complete. 2024-04-25T19:08:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.795+0000 7f1f27c16740 20 client.15786 trim_cache size 0 max 16384 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.810+0000 7f1f27c16740 1 Processor -- start 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 -- start start 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db968ec0 0x55a1db983680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96c9b0 0x55a1db98f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db98fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201b41c0 con 0x55a1db96d2d0 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201b5d00 con 0x55a1db96c9b0 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f27c16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201b4890 con 0x55a1db968ec0 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db98fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96c9b0 0x55a1db98f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db98fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60228/0 (socket says 172.21.15.86:60228) 2024-04-25T19:08:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f1e800640 1 -- 172.21.15.86:0/1509951163 learned_addr learned my addr 172.21.15.86:0/1509951163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.811+0000 7f1f1de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db968ec0 0x55a1db983680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1e800640 1 -- 172.21.15.86:0/1509951163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db968ec0 msgr2=0x55a1db983680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1e800640 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db968ec0 0x55a1db983680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1e800640 1 -- 172.21.15.86:0/1509951163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96c9b0 msgr2=0x55a1db98f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1e800640 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96c9b0 0x55a1db98f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1e800640 1 -- 172.21.15.86:0/1509951163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1db972fc0 con 0x55a1db96d2d0 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1d400640 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96c9b0 0x55a1db98f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f1e800640 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db98fe80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009900 tx=0x7f1f1803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17157595a76018b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f0aa00640 1 -- 172.21.15.86:0/1509951163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18035040 con 0x55a1db96d2d0 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f0aa00640 1 -- 172.21.15.86:0/1509951163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f180324b0 con 0x55a1db96d2d0 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.812+0000 7f1f0aa00640 1 -- 172.21.15.86:0/1509951163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1803c660 con 0x55a1db96d2d0 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 -- 172.21.15.86:0/1509951163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 msgr2=0x55a1db98fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db98fe80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009900 tx=0x7f1f1803e000 comp rx=0 tx=0).stop 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 -- 172.21.15.86:0/1509951163 shutdown_connections 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db98fe80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db96c9b0 0x55a1db98f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 --2- 172.21.15.86:0/1509951163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db968ec0 0x55a1db983680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 -- 172.21.15.86:0/1509951163 >> 172.21.15.86:0/1509951163 conn(0x55a1db98a500 msgr2=0x55a1db9692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 -- 172.21.15.86:0/1509951163 shutdown_connections 2024-04-25T19:08:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 -- 172.21.15.86:0/1509951163 wait complete. 2024-04-25T19:08:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.813+0000 7f1f27c16740 1 -- start start 2024-04-25T19:08:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 Processor -- start 2024-04-25T19:08:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db968ec0 0x55a1db97d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db97de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db9ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201b5630 con 0x55a1db96d2d0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201b4f60 con 0x55a1db968ec0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201b21a0 con 0x55a1db96c9b0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f27c16740 10 client.0 _ll_register_callbacks cb 0x7f1f0c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db9ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.814+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db97de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db9ed1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60230/0 (socket says 172.21.15.86:60230) 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 -- 172.21.15.86:0/3090026299 learned_addr learned my addr 172.21.15.86:0/3090026299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1de00640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db968ec0 0x55a1db97d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 -- 172.21.15.86:0/3090026299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 msgr2=0x55a1db97de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db97de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 -- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db968ec0 msgr2=0x55a1db97d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db968ec0 0x55a1db97d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 -- 172.21.15.86:0/3090026299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1db97e6e0 con 0x55a1db96d2d0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1d400640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db97de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1de00640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db968ec0 0x55a1db97d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1f1e800640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db9ed1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009930 tx=0x7f1f18032ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7af4c0bcca952a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.815+0000 7f1effe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1effe00640 1 -- 172.21.15.86:0/3090026299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18060060 con 0x55a1db96d2d0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f180324b0 con 0x55a1db96d2d0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1effe00640 1 -- 172.21.15.86:0/3090026299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1f18004ad0 con 0x55a1db96d2d0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1effe00640 1 -- 172.21.15.86:0/3090026299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1f18035070 con 0x55a1db96d2d0 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 10 client.15792 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1db9ee7e0 con 0x55a1db96d2d0 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1effe00640 1 -- 172.21.15.86:0/3090026299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f180324b0 con 0x55a1db96d2d0 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1effe00640 1 -- 172.21.15.86:0/3090026299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1f1803ca10 con 0x55a1db96d2d0 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 20 client.15792 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db977b90 0x55a1db9f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 4 client.15792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.816+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1db8e0320 con 0x55a1db977b90 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.817+0000 7f1f1de00640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db977b90 0x55a1db9f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.817+0000 7f1f1de00640 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db977b90 0x55a1db9f0e20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0c3f770a059307 server_cookie=75acddf5798e8a6a in_seq=0 out_seq=0 2024-04-25T19:08:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.817+0000 7f1effe00640 10 client.15792 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.836+0000 7f1effe00640 1 -- 172.21.15.86:0/3090026299 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1db8e0320 con 0x55a1db977b90 2024-04-25T19:08:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.836+0000 7f1effe00640 10 client.15792 handle_command_reply: tid=0 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 client.15792 shutdown 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 msgr2=0x55a1db9ed1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db9ed1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009930 tx=0x7f1f18032ac0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 shutdown_connections 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a1db977b90 0x55a1db9f0e20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1db96d2d0 0x55a1db9ed1c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1db96c9b0 0x55a1db97de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 --2- 172.21.15.86:0/3090026299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1db968ec0 0x55a1db97d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1effe00640 0 client.15792 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 >> 172.21.15.86:0/3090026299 conn(0x55a1db98a500 msgr2=0x55a1db974a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 shutdown_connections 2024-04-25T19:08:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 1 -- 172.21.15.86:0/3090026299 wait complete. 2024-04-25T19:08:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.837+0000 7f1f27c16740 20 client.15792 trim_cache size 0 max 16384 2024-04-25T19:08:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:08:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1f040a3d10 msgr2=0x7f1f040a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1f040a3d10 0x7f1f040a61d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f18006010 tx=0x7f1f18005b50 comp rx=0 tx=0).stop 2024-04-25T19:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 msgr2=0x7f1f201a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 0x7f1f201a47e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000eb90 tx=0x7f1f1000c6a0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 shutdown_connections 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1f040a3d10 0x7f1f040a61d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2010e420 0x7f1f2019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f20108e10 0x7f1f201a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 --2- 172.21.15.86:0/2992354880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f20108490 0x7f1f201a47e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.838+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 >> 172.21.15.86:0/2992354880 conn(0x7f1f20077d30 msgr2=0x7f1f20078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.839+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 shutdown_connections 2024-04-25T19:08:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:16.839+0000 7f1f1f200640 1 -- 172.21.15.86:0/2992354880 wait complete. 2024-04-25T19:08:18.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:18 smithi086 ceph-mon[27783]: pgmap v163: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:08:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:18 smithi183 ceph-mon[32344]: pgmap v163: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:08:18.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:18 smithi194 ceph-mon[32740]: pgmap v163: 97 pgs: 97 active+clean; 5.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:08:20.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:20 smithi086 ceph-mon[27783]: pgmap v164: 97 pgs: 97 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 26 op/s 2024-04-25T19:08:20.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:20 smithi183 ceph-mon[32344]: pgmap v164: 97 pgs: 97 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 26 op/s 2024-04-25T19:08:20.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:20 smithi194 ceph-mon[32740]: pgmap v164: 97 pgs: 97 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 26 op/s 2024-04-25T19:08:22.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:22 smithi086 ceph-mon[27783]: pgmap v165: 97 pgs: 97 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-25T19:08:22.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:22 smithi183 ceph-mon[32344]: pgmap v165: 97 pgs: 97 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-25T19:08:22.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:22 smithi194 ceph-mon[32740]: pgmap v165: 97 pgs: 97 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-25T19:08:24.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:24 smithi086 ceph-mon[27783]: pgmap v166: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:08:24.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:24 smithi183 ceph-mon[32344]: pgmap v166: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:08:24.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:24 smithi194 ceph-mon[32740]: pgmap v166: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:08:25.244 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T19:08:25.244 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.6 2024-04-25T19:08:25.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:08:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:08:26.179 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:26 smithi086 ceph-mon[27783]: pgmap v167: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:26.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:26 smithi183 ceph-mon[32344]: pgmap v167: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:26.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:26 smithi194 ceph-mon[32740]: pgmap v167: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:27.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 -- 172.21.15.86:0/4220225305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 msgr2=0x7f894c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 --2- 172.21.15.86:0/4220225305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 0x7f894c115f80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f894400b0d0 tx=0x7f894400b3e0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 -- 172.21.15.86:0/4220225305 shutdown_connections 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 --2- 172.21.15.86:0/4220225305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 0x7f894c115f80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 --2- 172.21.15.86:0/4220225305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 --2- 172.21.15.86:0/4220225305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894c109c80 0x7f894c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 -- 172.21.15.86:0/4220225305 >> 172.21.15.86:0/4220225305 conn(0x7f894c076210 msgr2=0x7f894c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 -- 172.21.15.86:0/4220225305 shutdown_connections 2024-04-25T19:08:27.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.911+0000 7f8954000700 1 -- 172.21.15.86:0/4220225305 wait complete. 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.912+0000 7f8954000700 1 Processor -- start 2024-04-25T19:08:27.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.912+0000 7f8954000700 1 -- start start 2024-04-25T19:08:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8954000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8954000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894c109c80 0x7f894c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8954000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 0x7f894c1a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8954000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1192a0 con 0x7f894c10e3b0 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8954000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f894c119120 con 0x7f894c1092d0 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8954000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f894c119420 con 0x7f894c109c80 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8951600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8951600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:41672/0 (socket says 172.21.15.86:41672) 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8951600700 1 -- 172.21.15.86:0/3487962693 learned_addr learned my addr 172.21.15.86:0/3487962693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8950c00700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894c109c80 0x7f894c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.913+0000 7f8952000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 0x7f894c1a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8951600700 1 -- 172.21.15.86:0/3487962693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894c109c80 msgr2=0x7f894c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8951600700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894c109c80 0x7f894c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8951600700 1 -- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 msgr2=0x7f894c1a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8951600700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 0x7f894c1a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8951600700 1 -- 172.21.15.86:0/3487962693 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c1a4730 con 0x7f894c1092d0 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8951600700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c19f770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f894000e9a0 tx=0x7f894000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894000ca10 con 0x7f894c1092d0 2024-04-25T19:08:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c1a4a20 con 0x7f894c1092d0 2024-04-25T19:08:27.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f894000cbb0 con 0x7f894c1092d0 2024-04-25T19:08:27.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940018680 con 0x7f894c1092d0 2024-04-25T19:08:27.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.914+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c1ac2a0 con 0x7f894c1092d0 2024-04-25T19:08:27.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.915+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c069730 con 0x7f894c1092d0 2024-04-25T19:08:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.918+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8940010430 con 0x7f894c1092d0 2024-04-25T19:08:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.919+0000 7f894a000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89340a3e30 0x7f89340a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.919+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f89400d0cd0 con 0x7f894c1092d0 2024-04-25T19:08:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.919+0000 7f8950c00700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89340a3e30 0x7f89340a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.920+0000 7f8950c00700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89340a3e30 0x7f89340a62f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f894c108830 tx=0x7f8938009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:27.922+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f894009d2c0 con 0x7f894c1092d0 2024-04-25T19:08:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.200+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f894c02df80 con 0x7f894c1092d0 2024-04-25T19:08:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.201+0000 7f894a000700 1 -- 172.21.15.86:0/3487962693 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f894009cbc0 con 0x7f894c1092d0 2024-04-25T19:08:28.204 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:08:28.204 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:08:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89340a3e30 msgr2=0x7f89340a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89340a3e30 0x7f89340a62f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f894c108830 tx=0x7f8938009260 comp rx=0 tx=0).stop 2024-04-25T19:08:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 msgr2=0x7f894c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c19f770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f894000e9a0 tx=0x7f894000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 shutdown_connections 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c10e3b0 0x7f894c1a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c1092d0 0x7f894c19f770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89340a3e30 0x7f89340a62f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.204+0000 7f8954000700 1 --2- 172.21.15.86:0/3487962693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894c109c80 0x7f894c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.205+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 >> 172.21.15.86:0/3487962693 conn(0x7f894c076210 msgr2=0x7f894c0767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.205+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 shutdown_connections 2024-04-25T19:08:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:28.205+0000 7f8954000700 1 -- 172.21.15.86:0/3487962693 wait complete. 2024-04-25T19:08:28.209 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:08:28.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:28 smithi086 ceph-mon[27783]: pgmap v168: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:28.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:28 smithi183 ceph-mon[32344]: pgmap v168: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:28.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:28 smithi194 ceph-mon[32740]: pgmap v168: 97 pgs: 97 active+clean; 1.9 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:29.173 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:29 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3487962693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:29.178 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 78.6 --await-for 42.0' 2024-04-25T19:08:29.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 78.6 --await-for 42.0 2024-04-25T19:08:29.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:29 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3487962693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:29.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:29 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3487962693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:29.608 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:08:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 -- 172.21.15.86:0/4222271388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc10e910 msgr2=0x7fd7fc110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 --2- 172.21.15.86:0/4222271388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc10e910 0x7fd7fc110d00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009be0 tx=0x7fd7f402a6c0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 -- 172.21.15.86:0/4222271388 shutdown_connections 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 --2- 172.21.15.86:0/4222271388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc10e910 0x7fd7fc110d00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 --2- 172.21.15.86:0/4222271388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7fc101680 0x7fd7fc10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 --2- 172.21.15.86:0/4222271388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7fc100d60 0x7fd7fc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 -- 172.21.15.86:0/4222271388 >> 172.21.15.86:0/4222271388 conn(0x7fd7fc077fd0 msgr2=0x7fd7fc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 -- 172.21.15.86:0/4222271388 shutdown_connections 2024-04-25T19:08:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.969+0000 7fd802000640 1 -- 172.21.15.86:0/4222271388 wait complete. 2024-04-25T19:08:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.970+0000 7fd802000640 1 Processor -- start 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.970+0000 7fd802000640 1 -- start start 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd802000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 0x7fd7fc1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd802000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7fc101680 0x7fd7fc1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd802000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7fc10e910 0x7fd7fc19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd802000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1143b0 con 0x7fd7fc100d60 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd802000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc114230 con 0x7fd7fc10e910 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd802000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc114530 con 0x7fd7fc101680 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 0x7fd7fc1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7faa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7fc101680 0x7fd7fc1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 0x7fd7fc1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37708/0 (socket says 172.21.15.86:37708) 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 -- 172.21.15.86:0/4290021003 learned_addr learned my addr 172.21.15.86:0/4290021003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7fc10e910 0x7fd7fc19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 -- 172.21.15.86:0/4290021003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7fc101680 msgr2=0x7fd7fc1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7fc101680 0x7fd7fc1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 -- 172.21.15.86:0/4290021003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7fc10e910 msgr2=0x7fd7fc19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7fc10e910 0x7fd7fc19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.971+0000 7fd7fb400640 1 -- 172.21.15.86:0/4290021003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc19af60 con 0x7fd7fc100d60 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.972+0000 7fd7fb400640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 0x7fd7fc1a0620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e9f0 tx=0x7fd7ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.972+0000 7fd7f3e00640 1 -- 172.21.15.86:0/4290021003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00cd80 con 0x7fd7fc100d60 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.972+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc19b250 con 0x7fd7fc100d60 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.972+0000 7fd7f3e00640 1 -- 172.21.15.86:0/4290021003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7ec0108e0 con 0x7fd7fc100d60 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.972+0000 7fd7f3e00640 1 -- 172.21.15.86:0/4290021003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec018980 con 0x7fd7fc100d60 2024-04-25T19:08:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.972+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc1a7470 con 0x7fd7fc100d60 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.975+0000 7fd7f3e00640 1 -- 172.21.15.86:0/4290021003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7ec010430 con 0x7fd7fc100d60 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.975+0000 7fd804fce740 1 Processor -- start 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 -- start start 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6d5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bedd63b900 con 0x55bedd6d5820 2024-04-25T19:08:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bedd5e1ef0 con 0x55bedd6e0a00 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd804fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bedd6378c0 con 0x55bedd6d6140 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6d5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37714/0 (socket says 172.21.15.86:37714) 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd7faa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.976+0000 7fd7fbe00640 1 -- 172.21.15.86:0/1155253109 learned_addr learned my addr 172.21.15.86:0/1155253109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7faa00640 1 -- 172.21.15.86:0/1155253109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 msgr2=0x55bedd6e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7faa00640 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7faa00640 1 -- 172.21.15.86:0/1155253109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 msgr2=0x55bedd6d5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7faa00640 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6d5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7faa00640 1 -- 172.21.15.86:0/1155253109 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedd34e210 con 0x55bedd6e0a00 2024-04-25T19:08:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7fb400640 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6e04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7f3e00640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd7e00a3dc0 0x7fd7e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7faa00640 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6e2df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc19bae0 tx=0x7fd7e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0bc7a8a2959d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7f3e00640 1 -- 172.21.15.86:0/4290021003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd7ec0d0b90 con 0x7fd7fc100d60 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd7e00a3dc0 0x7fd7e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7e7e00640 1 -- 172.21.15.86:0/1155253109 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8009db0 con 0x55bedd6e0a00 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7e7e00640 1 -- 172.21.15.86:0/1155253109 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e80035f0 con 0x55bedd6e0a00 2024-04-25T19:08:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.977+0000 7fd7e7e00640 1 -- 172.21.15.86:0/1155253109 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8019610 con 0x55bedd6e0a00 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 -- 172.21.15.86:0/1155253109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 msgr2=0x55bedd6e2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd7e00a3dc0 0x7fd7e00a6280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004af0 tx=0x7fd7f4005fd0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6e2df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc19bae0 tx=0x7fd7e8011810 comp rx=0 tx=0).stop 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 -- 172.21.15.86:0/1155253109 shutdown_connections 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6e2df0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6e04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 --2- 172.21.15.86:0/1155253109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6d5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.978+0000 7fd804fce740 1 -- 172.21.15.86:0/1155253109 >> 172.21.15.86:0/1155253109 conn(0x55bedd6d1d30 msgr2=0x55bedd6d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.979+0000 7fd804fce740 1 -- 172.21.15.86:0/1155253109 shutdown_connections 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.979+0000 7fd804fce740 1 -- 172.21.15.86:0/1155253109 wait complete. 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.979+0000 7fd804fce740 1 -- start start 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.979+0000 7fd804fce740 1 Processor -- start 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.979+0000 7fd804fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.979+0000 7fd804fce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd804fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6e0a00 0x55bedd6de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd804fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bedd6e3630 con 0x55bedd6d5820 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd804fce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bedd5f5310 con 0x55bedd6d6140 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd804fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bedd6e37b0 con 0x55bedd6e0a00 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd804fce740 10 client.0 _ll_register_callbacks cb 0x55bedd5c3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6dd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37724/0 (socket says 172.21.15.86:37724) 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 -- 172.21.15.86:0/1848945573 learned_addr learned my addr 172.21.15.86:0/1848945573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7faa00640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6e0a00 0x55bedd6de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 -- 172.21.15.86:0/1848945573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6e0a00 msgr2=0x55bedd6de320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6e0a00 0x55bedd6de320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 -- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 msgr2=0x55bedd6ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.980+0000 7fd7fb400640 1 -- 172.21.15.86:0/1848945573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedd41a620 con 0x55bedd6d5820 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7fb400640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6dd8a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c6a0 tx=0x7fd7ec018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a229d6284afcd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7e4c00640 1 -- 172.21.15.86:0/1848945573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0d5040 con 0x55bedd6d5820 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7e4c00640 1 -- 172.21.15.86:0/1848945573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd7ec09bf80 con 0x55bedd6d5820 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7e80035f0 con 0x55bedd6d5820 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7e4c00640 1 -- 172.21.15.86:0/1848945573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7ec0ce6b0 con 0x55bedd6d5820 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd804fce740 10 client.15810 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bedd6ef820 con 0x55bedd6d5820 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7e4c00640 1 -- 172.21.15.86:0/1848945573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e80035f0 con 0x55bedd6d5820 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd7e4c00640 1 -- 172.21.15.86:0/1848945573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd7ec09b9f0 con 0x55bedd6d5820 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd804fce740 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.981+0000 7fd804fce740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.982+0000 7fd804fce740 20 client.15810 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.982+0000 7fd804fce740 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd6f0ce0 0x55bedd6f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.982+0000 7fd804fce740 4 client.15810 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.982+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bedd64f8b0 con 0x55bedd6f0ce0 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.982+0000 7fd7faa00640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd6f0ce0 0x55bedd6f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.983+0000 7fd7faa00640 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd6f0ce0 0x55bedd6f30f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f72d2b65904ae2a server_cookie=69dd49bcd6b47ad2 in_seq=0 out_seq=0 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.983+0000 7fd7e4c00640 10 client.15810 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.983+0000 7fd7e4c00640 1 -- 172.21.15.86:0/1848945573 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bedd64f8b0 con 0x55bedd6f0ce0 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.983+0000 7fd7e4c00640 10 client.15810 handle_command_reply: tid=0 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 client.15810 shutdown 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 msgr2=0x55bedd6dd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6dd8a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c6a0 tx=0x7fd7ec018420 comp rx=0 tx=0).stop 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 shutdown_connections 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd6f0ce0 0x55bedd6f30f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6e0a00 0x55bedd6de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd7e4c00640 0 client.15810 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 --2- 172.21.15.86:0/1848945573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6dd8a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 >> 172.21.15.86:0/1848945573 conn(0x55bedd6d1d30 msgr2=0x55bedd6d7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 shutdown_connections 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 1 -- 172.21.15.86:0/1848945573 wait complete. 2024-04-25T19:08:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:29.984+0000 7fd804fce740 20 client.15810 trim_cache size 0 max 16384 2024-04-25T19:08:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 Processor -- start 2024-04-25T19:08:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 -- start start 2024-04-25T19:08:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6ec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1aff10 con 0x55bedd6d5820 2024-04-25T19:08:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1b1a50 con 0x55bedd6e0a00 2024-04-25T19:08:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd804fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1b05e0 con 0x55bedd6d6140 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.000+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6ec520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37738/0 (socket says 172.21.15.86:37738) 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 -- 172.21.15.86:0/158195457 learned_addr learned my addr 172.21.15.86:0/158195457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7faa00640 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 -- 172.21.15.86:0/158195457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 msgr2=0x55bedd6dca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6dca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 -- 172.21.15.86:0/158195457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 msgr2=0x55bedd6f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 -- 172.21.15.86:0/158195457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedd6d6eb0 con 0x55bedd6d5820 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7fb400640 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6ec520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec09c830 tx=0x7fd7ec00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6441ec0d36da91d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7e7400640 1 -- 172.21.15.86:0/158195457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0a56f0 con 0x55bedd6d5820 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7e7400640 1 -- 172.21.15.86:0/158195457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7ec0a0e30 con 0x55bedd6d5820 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.001+0000 7fd7e7400640 1 -- 172.21.15.86:0/158195457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0ce4b0 con 0x55bedd6d5820 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 -- 172.21.15.86:0/158195457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 msgr2=0x55bedd6ec520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6ec520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec09c830 tx=0x7fd7ec00cf20 comp rx=0 tx=0).stop 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 -- 172.21.15.86:0/158195457 shutdown_connections 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6e0a00 0x55bedd6f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d6140 0x55bedd6dca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 --2- 172.21.15.86:0/158195457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6d5820 0x55bedd6ec520 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 -- 172.21.15.86:0/158195457 >> 172.21.15.86:0/158195457 conn(0x55bedd6d1d30 msgr2=0x55bedd6d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 -- 172.21.15.86:0/158195457 shutdown_connections 2024-04-25T19:08:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 -- 172.21.15.86:0/158195457 wait complete. 2024-04-25T19:08:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.002+0000 7fd804fce740 1 -- start start 2024-04-25T19:08:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 Processor -- start 2024-04-25T19:08:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d5820 0x55bedd6e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 0x55bedd6e6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1b1380 con 0x55bedd6e0a00 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1b0cb0 con 0x55bedd6d6140 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1adef0 con 0x55bedd6d5820 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd804fce740 10 client.0 _ll_register_callbacks cb 0x7fd7ec098dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 0x55bedd6e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 0x55bedd6e6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37746/0 (socket says 172.21.15.86:37746) 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd7fbe00640 1 -- 172.21.15.86:0/4176033168 learned_addr learned my addr 172.21.15.86:0/4176033168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd7faa00640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.003+0000 7fd7fb400640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d5820 0x55bedd6e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7fbe00640 1 -- 172.21.15.86:0/4176033168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d5820 msgr2=0x55bedd6e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d5820 0x55bedd6e8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7fbe00640 1 -- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 msgr2=0x55bedd6e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7fbe00640 1 -- 172.21.15.86:0/4176033168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedd6e6a00 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 0x55bedd6e6200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc06b730 tx=0x7fd7f4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf6d143827148f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7e4c00640 1 -- 172.21.15.86:0/4176033168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4036040 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7ec0a0e30 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.004+0000 7fd7e4c00640 1 -- 172.21.15.86:0/4176033168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd7f4004040 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd7e4c00640 1 -- 172.21.15.86:0/4176033168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7f4004240 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 10 client.15816 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bedd756f10 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd7e4c00640 1 -- 172.21.15.86:0/4176033168 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7ec0a0e30 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd7e4c00640 1 -- 172.21.15.86:0/4176033168 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd7f4053020 con 0x55bedd6e0a00 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 10 client.15816 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 20 client.15816 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd7576f0 0x55bedd759b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 4 client.15816 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.005+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bedd649320 con 0x55bedd7576f0 2024-04-25T19:08:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.006+0000 7fd7fb400640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd7576f0 0x55bedd759b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.006+0000 7fd7fb400640 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd7576f0 0x55bedd759b10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6693331cfca68 server_cookie=d1a6b7846a1d4157 in_seq=0 out_seq=0 2024-04-25T19:08:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.006+0000 7fd7e4c00640 10 client.15816 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.019+0000 7fd7e4c00640 1 -- 172.21.15.86:0/4176033168 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bedd649320 con 0x55bedd7576f0 2024-04-25T19:08:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.019+0000 7fd7e4c00640 10 client.15816 handle_command_reply: tid=0 2024-04-25T19:08:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.019+0000 7fd804fce740 1 client.15816 shutdown 2024-04-25T19:08:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 msgr2=0x55bedd6e6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 0x55bedd6e6200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc06b730 tx=0x7fd7f4009500 comp rx=0 tx=0).stop 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 shutdown_connections 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bedd7576f0 0x55bedd759b10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bedd6e0a00 0x55bedd6e6200 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bedd6d6140 0x55bedd6e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 --2- 172.21.15.86:0/4176033168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bedd6d5820 0x55bedd6e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd7e4c00640 0 client.15816 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 >> 172.21.15.86:0/4176033168 conn(0x55bedd6d1d30 msgr2=0x55bedd6d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 shutdown_connections 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 1 -- 172.21.15.86:0/4176033168 wait complete. 2024-04-25T19:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.020+0000 7fd804fce740 20 client.15816 trim_cache size 0 max 16384 2024-04-25T19:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd7e00a3dc0 msgr2=0x7fd7e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd7e00a3dc0 0x7fd7e00a6280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004af0 tx=0x7fd7f4005fd0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 msgr2=0x7fd7fc1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 0x7fd7fc1a0620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e9f0 tx=0x7fd7ec00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 shutdown_connections 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd7e00a3dc0 0x7fd7e00a6280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7fc10e910 0x7fd7fc19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7fc101680 0x7fd7fc1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 --2- 172.21.15.86:0/4290021003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc100d60 0x7fd7fc1a0620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 >> 172.21.15.86:0/4290021003 conn(0x7fd7fc077fd0 msgr2=0x7fd7fc10fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 shutdown_connections 2024-04-25T19:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:30.021+0000 7fd802000640 1 -- 172.21.15.86:0/4290021003 wait complete. 2024-04-25T19:08:30.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:30 smithi183 ceph-mon[32344]: pgmap v169: 97 pgs: 97 active+clean; 2.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:08:30.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:30 smithi194 ceph-mon[32740]: pgmap v169: 97 pgs: 97 active+clean; 2.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:08:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:30 smithi086 ceph-mon[27783]: pgmap v169: 97 pgs: 97 active+clean; 2.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:08:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 -- 172.21.15.86:0/2227858287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198810e3b0 msgr2=0x7f1988115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 --2- 172.21.15.86:0/2227858287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198810e3b0 0x7f1988115f80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f198000b0d0 tx=0x7f198000b3e0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 -- 172.21.15.86:0/2227858287 shutdown_connections 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 --2- 172.21.15.86:0/2227858287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198810e3b0 0x7f1988115f80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 --2- 172.21.15.86:0/2227858287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19881092d0 0x7f19881096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 --2- 172.21.15.86:0/2227858287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1988109c80 0x7f198810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 -- 172.21.15.86:0/2227858287 >> 172.21.15.86:0/2227858287 conn(0x7f1988076210 msgr2=0x7f1988107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 -- 172.21.15.86:0/2227858287 shutdown_connections 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.875+0000 7f198ca00700 1 -- 172.21.15.86:0/2227858287 wait complete. 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 Processor -- start 2024-04-25T19:08:31.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 -- start start 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 0x7f198819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1988109c80 0x7f198819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f198810e3b0 0x7f19881a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19881192a0 con 0x7f1988109c80 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1988119120 con 0x7f198810e3b0 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.876+0000 7f198ca00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1988119420 con 0x7f19881092d0 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 0x7f198819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1986800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f198810e3b0 0x7f19881a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 0x7f198819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39572/0 (socket says 172.21.15.86:39572) 2024-04-25T19:08:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 -- 172.21.15.86:0/520333495 learned_addr learned my addr 172.21.15.86:0/520333495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985400700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1988109c80 0x7f198819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 -- 172.21.15.86:0/520333495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f198810e3b0 msgr2=0x7f19881a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f198810e3b0 0x7f19881a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 -- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1988109c80 msgr2=0x7f198819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1988109c80 0x7f198819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.877+0000 7f1985e00700 1 -- 172.21.15.86:0/520333495 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881a45f0 con 0x7f19881092d0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.878+0000 7f1985e00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 0x7f198819f630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9a0 tx=0x7f197800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.878+0000 7f197ea00700 1 -- 172.21.15.86:0/520333495 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800ca10 con 0x7f19881092d0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.878+0000 7f197ea00700 1 -- 172.21.15.86:0/520333495 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197800cbb0 con 0x7f19881092d0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.878+0000 7f197ea00700 1 -- 172.21.15.86:0/520333495 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978018680 con 0x7f19881092d0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.878+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19881a48e0 con 0x7f19881092d0 2024-04-25T19:08:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.878+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881ac160 con 0x7f19881092d0 2024-04-25T19:08:31.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.882+0000 7f197ea00700 1 -- 172.21.15.86:0/520333495 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1978010430 con 0x7f19881092d0 2024-04-25T19:08:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.883+0000 7f199732db80 1 Processor -- start 2024-04-25T19:08:31.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 -- start start 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f197ea00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f196c0a3ed0 0x7f196c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d5602020 con 0x5599d56ba180 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599d55a88c0 con 0x5599d56af2d0 2024-04-25T19:08:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f199732db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599d55fdfe0 con 0x5599d56ae9b0 2024-04-25T19:08:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.884+0000 7f197ea00700 1 -- 172.21.15.86:0/520333495 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f19780d09f0 con 0x7f19881092d0 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985e00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f196c0a3ed0 0x7f196c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1986800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1986800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39588/0 (socket says 172.21.15.86:39588) 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1986800700 1 -- 172.21.15.86:0/4199069560 learned_addr learned my addr 172.21.15.86:0/4199069560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1986800700 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985400700 1 -- 172.21.15.86:0/4199069560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 msgr2=0x5599d56aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985400700 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985400700 1 -- 172.21.15.86:0/4199069560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 msgr2=0x5599d56bc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985400700 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1985400700 1 -- 172.21.15.86:0/4199069560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d559e810 con 0x5599d56af2d0 2024-04-25T19:08:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.885+0000 7f1986800700 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.886+0000 7f1985e00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f196c0a3ed0 0x7f196c0a6390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19780992b0 tx=0x7f1978009820 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.886+0000 7f1985400700 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56b9c40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f198806c260 tx=0x7f1974009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f83b99c241c019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.886+0000 7f1973400700 1 -- 172.21.15.86:0/4199069560 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974011c80 con 0x5599d56af2d0 2024-04-25T19:08:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.886+0000 7f1973400700 1 -- 172.21.15.86:0/4199069560 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974011e20 con 0x5599d56af2d0 2024-04-25T19:08:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.886+0000 7f1973400700 1 -- 172.21.15.86:0/4199069560 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974005280 con 0x5599d56af2d0 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 -- 172.21.15.86:0/4199069560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 msgr2=0x5599d56b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56b9c40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f198806c260 tx=0x7f1974009f60 comp rx=0 tx=0).stop 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 -- 172.21.15.86:0/4199069560 shutdown_connections 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 --2- 172.21.15.86:0/4199069560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56b9c40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 -- 172.21.15.86:0/4199069560 >> 172.21.15.86:0/4199069560 conn(0x5599d55dd730 msgr2=0x5599d56aae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 -- 172.21.15.86:0/4199069560 shutdown_connections 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.887+0000 7f199732db80 1 -- 172.21.15.86:0/4199069560 wait complete. 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.888+0000 7f199732db80 1 -- start start 2024-04-25T19:08:31.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.888+0000 7f199732db80 1 Processor -- start 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56af2d0 0x5599d56b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ba180 0x5599d56b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d56bcca0 con 0x5599d56af2d0 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599d55bb9f0 con 0x5599d56ba180 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599d56bce20 con 0x5599d56ae9b0 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f199732db80 10 client.0 _ll_register_callbacks cb 0x5599d559a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1985400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56af2d0 0x5599d56b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56b9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:39602/0 (socket says 172.21.15.86:39602) 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 -- 172.21.15.86:0/1532167849 learned_addr learned my addr 172.21.15.86:0/1532167849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ba180 0x5599d56b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 -- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ba180 msgr2=0x5599d56b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ba180 0x5599d56b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 -- 172.21.15.86:0/1532167849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56af2d0 msgr2=0x5599d56b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56af2d0 0x5599d56b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.889+0000 7f1986800700 1 -- 172.21.15.86:0/1532167849 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d554fa60 con 0x5599d56ae9b0 2024-04-25T19:08:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f1986800700 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56b9810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1980003f20 tx=0x7f1980024040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6648925633de0005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f1967e00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:08:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f1967e00700 1 -- 172.21.15.86:0/1532167849 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980007b20 con 0x5599d56ae9b0 2024-04-25T19:08:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f1967e00700 1 -- 172.21.15.86:0/1532167849 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1980007cc0 con 0x5599d56ae9b0 2024-04-25T19:08:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1974011e20 con 0x5599d56ae9b0 2024-04-25T19:08:31.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f1967e00700 1 -- 172.21.15.86:0/1532167849 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1980032050 con 0x5599d56ae9b0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f199732db80 10 client.25921 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599d5598c80 con 0x5599d56ae9b0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.890+0000 7f1967e00700 1 -- 172.21.15.86:0/1532167849 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974011e20 con 0x5599d56ae9b0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f1967e00700 1 -- 172.21.15.86:0/1532167849 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1980011070 con 0x5599d56ae9b0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f199732db80 10 client.25921 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f199732db80 10 client.25921 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f199732db80 20 client.25921 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f199732db80 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d56c99a0 0x5599d56cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f199732db80 4 client.25921 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599d560f810 con 0x5599d56c99a0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.891+0000 7f1985400700 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d56c99a0 0x5599d56cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.892+0000 7f1985400700 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d56c99a0 0x5599d56cbd90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197b96e21befe0c4 server_cookie=1b5656aa954eec20 in_seq=0 out_seq=0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.892+0000 7f1967e00700 10 client.25921 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.892+0000 7f1967e00700 1 -- 172.21.15.86:0/1532167849 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599d560f810 con 0x5599d56c99a0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f1967e00700 10 client.25921 handle_command_reply: tid=0 2024-04-25T19:08:31.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 client.25921 shutdown 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 msgr2=0x5599d56b9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56b9810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1980003f20 tx=0x7f1980024040 comp rx=0 tx=0).stop 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 shutdown_connections 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d56c99a0 0x5599d56cbd90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ba180 0x5599d56b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f1967e00700 0 client.25921 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ae9b0 0x5599d56b9810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 --2- 172.21.15.86:0/1532167849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56af2d0 0x5599d56b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 >> 172.21.15.86:0/1532167849 conn(0x5599d55dd730 msgr2=0x5599d56aaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 shutdown_connections 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 1 -- 172.21.15.86:0/1532167849 wait complete. 2024-04-25T19:08:31.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.893+0000 7f199732db80 20 client.25921 trim_cache size 0 max 16384 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 Processor -- start 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 -- start start 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 0x5599d56b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ba180 0x5599d56d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b3930 con 0x5599d56ae9b0 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b5470 con 0x5599d56af2d0 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.915+0000 7f199732db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b4000 con 0x5599d56ba180 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 0x5599d56b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 0x5599d56b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37798/0 (socket says 172.21.15.86:37798) 2024-04-25T19:08:31.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 -- 172.21.15.86:0/210327067 learned_addr learned my addr 172.21.15.86:0/210327067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1985400700 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ba180 0x5599d56d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 -- 172.21.15.86:0/210327067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ba180 msgr2=0x5599d56d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ba180 0x5599d56d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 -- 172.21.15.86:0/210327067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 msgr2=0x5599d56aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 -- 172.21.15.86:0/210327067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d56b5110 con 0x5599d56ae9b0 2024-04-25T19:08:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.916+0000 7f1986800700 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 0x5599d56b71b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1980012770 tx=0x7f1980008fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ca0c5e64bda186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f1972a00700 1 -- 172.21.15.86:0/210327067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198000cd70 con 0x5599d56ae9b0 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f1972a00700 1 -- 172.21.15.86:0/210327067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1980012890 con 0x5599d56ae9b0 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f1972a00700 1 -- 172.21.15.86:0/210327067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198001d6b0 con 0x5599d56ae9b0 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 -- 172.21.15.86:0/210327067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 msgr2=0x5599d56b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 0x5599d56b71b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1980012770 tx=0x7f1980008fb0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 -- 172.21.15.86:0/210327067 shutdown_connections 2024-04-25T19:08:31.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56ba180 0x5599d56d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ae9b0 0x5599d56b71b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 --2- 172.21.15.86:0/210327067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56af2d0 0x5599d56aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.917+0000 7f199732db80 1 -- 172.21.15.86:0/210327067 >> 172.21.15.86:0/210327067 conn(0x5599d55dd730 msgr2=0x5599d56aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.918+0000 7f199732db80 1 -- 172.21.15.86:0/210327067 shutdown_connections 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.918+0000 7f199732db80 1 -- 172.21.15.86:0/210327067 wait complete. 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.918+0000 7f199732db80 1 -- start start 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.918+0000 7f199732db80 1 Processor -- start 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ae9b0 0x5599d56c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56af2d0 0x5599d56c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b4da0 con 0x5599d56ba180 2024-04-25T19:08:31.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b46d0 con 0x5599d56ae9b0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b18d0 con 0x5599d56af2d0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f199732db80 10 client.0 _ll_register_callbacks cb 0x5599d559a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f1986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f1985400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56af2d0 0x5599d56c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f1986800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37812/0 (socket says 172.21.15.86:37812) 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f1986800700 1 -- 172.21.15.86:0/1392421591 learned_addr learned my addr 172.21.15.86:0/1392421591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.919+0000 7f1986800700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ae9b0 0x5599d56c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1985400700 1 -- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ae9b0 msgr2=0x5599d56c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1985400700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ae9b0 0x5599d56c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1985400700 1 -- 172.21.15.86:0/1392421591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 msgr2=0x5599d56bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1985400700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1985400700 1 -- 172.21.15.86:0/1392421591 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d56bf930 con 0x5599d56af2d0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1986800700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1985400700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56af2d0 0x5599d56c25f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1974009230 tx=0x7f1974005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d49dcb9a388049 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:31.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1967e00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1967e00700 1 -- 172.21.15.86:0/1392421591 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974004990 con 0x5599d56af2d0 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1967e00700 1 -- 172.21.15.86:0/1392421591 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f1974011b80 con 0x5599d56af2d0 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1980012890 con 0x5599d56af2d0 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f1967e00700 1 -- 172.21.15.86:0/1392421591 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1974035050 con 0x5599d56af2d0 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.920+0000 7f199732db80 10 client.25927 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599d5730960 con 0x5599d56af2d0 2024-04-25T19:08:31.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f1967e00700 1 -- 172.21.15.86:0/1392421591 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1980012890 con 0x5599d56af2d0 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f1967e00700 1 -- 172.21.15.86:0/1392421591 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1974018070 con 0x5599d56af2d0 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 10 client.25927 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 10 client.25927 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 20 client.25927 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d5730f70 0x5599d5733390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 4 client.25927 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.6, "await_for": 42.0}] 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.921+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.6, "await_for": 42.0}) -- 0x5599d56098b0 con 0x5599d5730f70 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.922+0000 7f1986800700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d5730f70 0x5599d5733390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.922+0000 7f1986800700 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d5730f70 0x5599d5733390 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20749cc98df11356 server_cookie=ec8efd9e0d20ee75 in_seq=0 out_seq=0 2024-04-25T19:08:31.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.922+0000 7f1967e00700 10 client.25927 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f1967e00700 1 -- 172.21.15.86:0/1392421591 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5599d56098b0 con 0x5599d5730f70 2024-04-25T19:08:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f1967e00700 10 client.25927 handle_command_reply: tid=0 2024-04-25T19:08:31.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 client.25927 shutdown 2024-04-25T19:08:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56af2d0 msgr2=0x5599d56c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56af2d0 0x5599d56c25f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1974009230 tx=0x7f1974005ce0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 shutdown_connections 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599d5730f70 0x5599d5733390 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d56ba180 0x5599d56bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599d56ae9b0 0x5599d56c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 --2- 172.21.15.86:0/1392421591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599d56af2d0 0x5599d56c25f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f1967e00700 0 client.25927 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.941+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 >> 172.21.15.86:0/1392421591 conn(0x5599d55dd730 msgr2=0x5599d56b83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.942+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 shutdown_connections 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.942+0000 7f199732db80 1 -- 172.21.15.86:0/1392421591 wait complete. 2024-04-25T19:08:31.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.942+0000 7f199732db80 20 client.25927 trim_cache size 0 max 16384 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 14, 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "d42824d3": { 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 14, 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:08:31.945 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 78.6, 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:31.946 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f196c0a3ed0 msgr2=0x7f196c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f196c0a3ed0 0x7f196c0a6390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19780992b0 tx=0x7f1978009820 comp rx=0 tx=0).stop 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 msgr2=0x7f198819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 0x7f198819f630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9a0 tx=0x7f197800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 shutdown_connections 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f198810e3b0 0x7f19881a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f196c0a3ed0 0x7f196c0a6390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19881092d0 0x7f198819f630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 --2- 172.21.15.86:0/520333495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1988109c80 0x7f198819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 >> 172.21.15.86:0/520333495 conn(0x7f1988076210 msgr2=0x7f1988076690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 shutdown_connections 2024-04-25T19:08:31.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:31.943+0000 7f198ca00700 1 -- 172.21.15.86:0/520333495 wait complete. 2024-04-25T19:08:32.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:32 smithi086 ceph-mon[27783]: pgmap v170: 97 pgs: 97 active+clean; 2.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T19:08:32.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:32 smithi183 ceph-mon[32344]: pgmap v170: 97 pgs: 97 active+clean; 2.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T19:08:32.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:32 smithi194 ceph-mon[32740]: pgmap v170: 97 pgs: 97 active+clean; 2.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-04-25T19:08:32.845 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd42824d3', quiesce took 7.6 seconds. Will release after: 18.6 2024-04-25T19:08:34.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:34 smithi183 ceph-mon[32344]: pgmap v171: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-25T19:08:34.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:34 smithi194 ceph-mon[32740]: pgmap v171: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-25T19:08:34.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:34 smithi086 ceph-mon[27783]: pgmap v171: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-25T19:08:36.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:36 smithi183 ceph-mon[32344]: pgmap v172: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:36.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:36 smithi194 ceph-mon[32740]: pgmap v172: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:36.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:36 smithi086 ceph-mon[27783]: pgmap v172: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:38.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:38 smithi183 ceph-mon[32344]: pgmap v173: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:38.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:38 smithi194 ceph-mon[32740]: pgmap v173: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:38.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:38 smithi086 ceph-mon[27783]: pgmap v173: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T19:08:40.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:40 smithi086 ceph-mon[27783]: pgmap v174: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-04-25T19:08:40.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:40 smithi183 ceph-mon[32344]: pgmap v174: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-04-25T19:08:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:40 smithi194 ceph-mon[32740]: pgmap v174: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-04-25T19:08:42.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:42 smithi183 ceph-mon[32344]: pgmap v175: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T19:08:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:42 smithi194 ceph-mon[32740]: pgmap v175: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T19:08:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:42 smithi086 ceph-mon[27783]: pgmap v175: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T19:08:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 -- 172.21.15.86:0/1564889896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 msgr2=0x7fc81c10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 --2- 172.21.15.86:0/1564889896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c10e800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009930 tx=0x7fc80c02a440 comp rx=0 tx=0).stop 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 -- 172.21.15.86:0/1564889896 shutdown_connections 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 --2- 172.21.15.86:0/1564889896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 0x7fc81c115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 --2- 172.21.15.86:0/1564889896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c108490 0x7fc81c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 --2- 172.21.15.86:0/1564889896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c10e800 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.160+0000 7fc81be00640 1 -- 172.21.15.86:0/1564889896 >> 172.21.15.86:0/1564889896 conn(0x7fc81c077d30 msgr2=0x7fc81c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.161+0000 7fc81be00640 1 -- 172.21.15.86:0/1564889896 shutdown_connections 2024-04-25T19:08:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.161+0000 7fc81be00640 1 -- 172.21.15.86:0/1564889896 wait complete. 2024-04-25T19:08:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.161+0000 7fc81be00640 1 Processor -- start 2024-04-25T19:08:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 -- start start 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c108490 0x7fc81c1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 0x7fc81c1a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1185d0 con 0x7fc81c10e420 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c118450 con 0x7fc81c108490 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c118750 con 0x7fc81c108e10 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc813e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 0x7fc81c1a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc813e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 0x7fc81c1a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:38036/0 (socket says 172.21.15.86:38036) 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc813e00640 1 -- 172.21.15.86:0/1096889165 learned_addr learned my addr 172.21.15.86:0/1096889165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c19e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46550/0 (socket says 172.21.15.86:46550) 2024-04-25T19:08:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.162+0000 7fc81aa00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c108490 0x7fc81c1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81b400640 1 -- 172.21.15.86:0/1096889165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 msgr2=0x7fc81c1a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81b400640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 0x7fc81c1a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81b400640 1 -- 172.21.15.86:0/1096889165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c108490 msgr2=0x7fc81c1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81b400640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c108490 0x7fc81c1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81b400640 1 -- 172.21.15.86:0/1096889165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c19f130 con 0x7fc81c10e420 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81b400640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c19e9b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc81400da10 tx=0x7fc81400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc813400640 1 -- 172.21.15.86:0/1096889165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400bd70 con 0x7fc81c10e420 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc813400640 1 -- 172.21.15.86:0/1096889165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc814003e20 con 0x7fc81c10e420 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c19f420 con 0x7fc81c10e420 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc813400640 1 -- 172.21.15.86:0/1096889165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8140106c0 con 0x7fc81c10e420 2024-04-25T19:08:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.163+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1ab6c0 con 0x7fc81c10e420 2024-04-25T19:08:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 Processor -- start 2024-04-25T19:08:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc813400640 1 -- 172.21.15.86:0/1096889165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc814004270 con 0x7fc81c10e420 2024-04-25T19:08:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 -- start start 2024-04-25T19:08:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaaca6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacb3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aaaac0c900 con 0x55aaaaca69b0 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aaaabb2ef0 con 0x55aaaacb1b90 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.167+0000 7fc824563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55aaaac088c0 con 0x55aaaaca72d0 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaaca6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc81aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacb1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaaca6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46560/0 (socket says 172.21.15.86:46560) 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc813e00640 1 -- 172.21.15.86:0/2156658306 learned_addr learned my addr 172.21.15.86:0/2156658306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc813e00640 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacb3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc81aa00640 1 -- 172.21.15.86:0/2156658306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 msgr2=0x55aaaacb3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc81aa00640 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacb3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc81aa00640 1 -- 172.21.15.86:0/2156658306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 msgr2=0x55aaaaca6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc81aa00640 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaaca6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc81aa00640 1 -- 172.21.15.86:0/2156658306 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaaa91f210 con 0x55aaaaca72d0 2024-04-25T19:08:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.168+0000 7fc813400640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc8000a3d10 0x7fc8000a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.169+0000 7fc81aa00640 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacb1650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc80c031440 tx=0x7fc80c031bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e5be4f72cc0477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.169+0000 7fc813400640 1 -- 172.21.15.86:0/1096889165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fc8140d0ba0 con 0x7fc81c10e420 2024-04-25T19:08:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.169+0000 7fc813e00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc8000a3d10 0x7fc8000a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.169+0000 7fc80be00640 1 -- 172.21.15.86:0/2156658306 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c040070 con 0x55aaaaca72d0 2024-04-25T19:08:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.169+0000 7fc80be00640 1 -- 172.21.15.86:0/2156658306 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80c0316a0 con 0x55aaaaca72d0 2024-04-25T19:08:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.169+0000 7fc80be00640 1 -- 172.21.15.86:0/2156658306 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c035040 con 0x55aaaaca72d0 2024-04-25T19:08:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 -- 172.21.15.86:0/2156658306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 msgr2=0x55aaaacb1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacb1650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc80c031440 tx=0x7fc80c031bc0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc813e00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc8000a3d10 0x7fc8000a61d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55aaaaca42e0 tx=0x7fc804015040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 -- 172.21.15.86:0/2156658306 shutdown_connections 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacb3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacb1650 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 --2- 172.21.15.86:0/2156658306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaaca6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 -- 172.21.15.86:0/2156658306 >> 172.21.15.86:0/2156658306 conn(0x55aaaaca2ec0 msgr2=0x55aaaaca32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 -- 172.21.15.86:0/2156658306 shutdown_connections 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 -- 172.21.15.86:0/2156658306 wait complete. 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.170+0000 7fc824563740 1 -- start start 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 Processor -- start 2024-04-25T19:08:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca69b0 0x55aaaacaeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 0x55aaaacaf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacaf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aaaacb4760 con 0x55aaaaca72d0 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aaaabc6310 con 0x55aaaaca69b0 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55aaaacb48e0 con 0x55aaaacb1b90 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc824563740 10 client.0 _ll_register_callbacks cb 0x55aaaab94cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 0x55aaaacaf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 0x55aaaacaf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46562/0 (socket says 172.21.15.86:46562) 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc813e00640 1 -- 172.21.15.86:0/3292146711 learned_addr learned my addr 172.21.15.86:0/3292146711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.171+0000 7fc81aa00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca69b0 0x55aaaacaeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc81b400640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacaf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc813e00640 1 -- 172.21.15.86:0/3292146711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 msgr2=0x55aaaacaf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc813e00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacaf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc813e00640 1 -- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca69b0 msgr2=0x55aaaacaeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc813e00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca69b0 0x55aaaacaeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc813e00640 1 -- 172.21.15.86:0/3292146711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaaa9eb620 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc81aa00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca69b0 0x55aaaacaeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc813e00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 0x55aaaacaf000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8040038d0 tx=0x7fc804005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87f117f5ab217cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc808c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.172+0000 7fc808c00640 1 -- 172.21.15.86:0/3292146711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804010900 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc808c00640 1 -- 172.21.15.86:0/3292146711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fc804010aa0 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc80c0316a0 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc808c00640 1 -- 172.21.15.86:0/3292146711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc804010ca0 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc824563740 10 client.15846 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaaacc0a00 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc808c00640 1 -- 172.21.15.86:0/3292146711 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80c0316a0 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.173+0000 7fc808c00640 1 -- 172.21.15.86:0/3292146711 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc804021410 con 0x55aaaaca72d0 2024-04-25T19:08:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc824563740 10 client.15846 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc824563740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc824563740 20 client.15846 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc824563740 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaacc1ec0 0x55aaaacc42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc824563740 4 client.15846 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaaac208b0 con 0x55aaaacc1ec0 2024-04-25T19:08:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.174+0000 7fc81aa00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaacc1ec0 0x55aaaacc42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.175+0000 7fc81aa00640 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaacc1ec0 0x55aaaacc42b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111ff7c74f090963 server_cookie=50400625f5a891e6 in_seq=0 out_seq=0 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.175+0000 7fc808c00640 10 client.15846 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc808c00640 1 -- 172.21.15.86:0/3292146711 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aaaac208b0 con 0x55aaaacc1ec0 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc808c00640 10 client.15846 handle_command_reply: tid=0 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 client.15846 shutdown 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 msgr2=0x55aaaacaf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 0x55aaaacaf000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8040038d0 tx=0x7fc804005f90 comp rx=0 tx=0).stop 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 shutdown_connections 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaacc1ec0 0x55aaaacc42b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacaf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca72d0 0x55aaaacaf000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc808c00640 0 client.15846 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 --2- 172.21.15.86:0/3292146711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca69b0 0x55aaaacaeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 >> 172.21.15.86:0/3292146711 conn(0x55aaaaca2ec0 msgr2=0x55aaaaca8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 shutdown_connections 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 1 -- 172.21.15.86:0/3292146711 wait complete. 2024-04-25T19:08:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.176+0000 7fc824563740 20 client.15846 trim_cache size 0 max 16384 2024-04-25T19:08:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 Processor -- start 2024-04-25T19:08:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 -- start start 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacbd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1b4150 con 0x55aaaaca69b0 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1b5c90 con 0x55aaaacb1b90 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.192+0000 7fc824563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1b4820 con 0x55aaaaca72d0 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacbd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacbd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46572/0 (socket says 172.21.15.86:46572) 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81aa00640 1 -- 172.21.15.86:0/453570184 learned_addr learned my addr 172.21.15.86:0/453570184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc813e00640 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 -- 172.21.15.86:0/453570184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 msgr2=0x55aaaacc9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacc9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 -- 172.21.15.86:0/453570184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 msgr2=0x55aaaacbd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacbd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 -- 172.21.15.86:0/453570184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaaacc4620 con 0x55aaaacb1b90 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81aa00640 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacbd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.193+0000 7fc81b400640 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacca100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc8140ce240 tx=0x7fc8140bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19914302b94843bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc80b400640 1 -- 172.21.15.86:0/453570184 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81409ca90 con 0x55aaaacb1b90 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc80b400640 1 -- 172.21.15.86:0/453570184 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8140a0e40 con 0x55aaaacb1b90 2024-04-25T19:08:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc80b400640 1 -- 172.21.15.86:0/453570184 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814099100 con 0x55aaaacb1b90 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 -- 172.21.15.86:0/453570184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 msgr2=0x55aaaacca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacca100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc8140ce240 tx=0x7fc8140bd040 comp rx=0 tx=0).stop 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 -- 172.21.15.86:0/453570184 shutdown_connections 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaacb1b90 0x55aaaacca100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaaca72d0 0x55aaaacc9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 --2- 172.21.15.86:0/453570184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacbd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.194+0000 7fc824563740 1 -- 172.21.15.86:0/453570184 >> 172.21.15.86:0/453570184 conn(0x55aaaaca2ec0 msgr2=0x55aaaaca9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.195+0000 7fc824563740 1 -- 172.21.15.86:0/453570184 shutdown_connections 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.195+0000 7fc824563740 1 -- 172.21.15.86:0/453570184 wait complete. 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.195+0000 7fc824563740 1 -- start start 2024-04-25T19:08:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.195+0000 7fc824563740 1 Processor -- start 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.195+0000 7fc824563740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.195+0000 7fc824563740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 0x55aaaacb9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc824563740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacb6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc824563740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1b55c0 con 0x55aaaaca69b0 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc824563740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1b4ef0 con 0x55aaaaca72d0 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc824563740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1b2130 con 0x55aaaacb1b90 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc824563740 10 client.0 _ll_register_callbacks cb 0x7fc8040044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc81aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 0x55aaaacb9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 0x55aaaacb9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:34338/0 (socket says 172.21.15.86:34338) 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 -- 172.21.15.86:0/3190620769 learned_addr learned my addr 172.21.15.86:0/3190620769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc81aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacb98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46576/0 (socket says 172.21.15.86:46576) 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc81b400640 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacb6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 -- 172.21.15.86:0/3190620769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 msgr2=0x55aaaacb6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacb6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 -- 172.21.15.86:0/3190620769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 msgr2=0x55aaaacb98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacb98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 -- 172.21.15.86:0/3190620769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaaacb7500 con 0x55aaaaca72d0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc813e00640 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 0x55aaaacb9e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc804021a20 tx=0x7fc804021cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a79f1f0560e13f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.196+0000 7fc808c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.197+0000 7fc808c00640 1 -- 172.21.15.86:0/3190620769 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804010e40 con 0x55aaaaca72d0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.197+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8140a0e40 con 0x55aaaaca72d0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.197+0000 7fc808c00640 1 -- 172.21.15.86:0/3190620769 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fc80403a430 con 0x55aaaaca72d0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.197+0000 7fc808c00640 1 -- 172.21.15.86:0/3190620769 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc80403a630 con 0x55aaaaca72d0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.197+0000 7fc824563740 10 client.25949 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.197+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaaad27f80 con 0x55aaaaca72d0 2024-04-25T19:08:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc808c00640 1 -- 172.21.15.86:0/3190620769 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8140a0e40 con 0x55aaaaca72d0 2024-04-25T19:08:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc808c00640 1 -- 172.21.15.86:0/3190620769 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc804029410 con 0x55aaaaca72d0 2024-04-25T19:08:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc824563740 10 client.25949 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc824563740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc824563740 20 client.25949 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc824563740 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaad28730 0x55aaaad2ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc824563740 4 client.25949 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaaac1a320 con 0x55aaaad28730 2024-04-25T19:08:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.198+0000 7fc81aa00640 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaad28730 0x55aaaad2ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.199+0000 7fc808c00640 10 client.25949 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.199+0000 7fc81aa00640 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaad28730 0x55aaaad2ab50 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdb217b977dfa6d server_cookie=4bfdde15e008c8e8 in_seq=0 out_seq=0 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc808c00640 1 -- 172.21.15.86:0/3190620769 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aaaac1a320 con 0x55aaaad28730 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc808c00640 10 client.25949 handle_command_reply: tid=0 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 client.25949 shutdown 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 msgr2=0x55aaaacb9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 0x55aaaacb9e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc804021a20 tx=0x7fc804021cd0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 shutdown_connections 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55aaaad28730 0x55aaaad2ab50 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaaacb1b90 0x55aaaacb6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc808c00640 0 client.25949 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaaaca72d0 0x55aaaacb9e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 --2- 172.21.15.86:0/3190620769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaaaca69b0 0x55aaaacb98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 >> 172.21.15.86:0/3190620769 conn(0x55aaaaca2ec0 msgr2=0x55aaaaca8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 shutdown_connections 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 1 -- 172.21.15.86:0/3190620769 wait complete. 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.213+0000 7fc824563740 20 client.25949 trim_cache size 0 max 16384 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:08:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:08:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc8000a3d10 msgr2=0x7fc8000a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc8000a3d10 0x7fc8000a61d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55aaaaca42e0 tx=0x7fc804015040 comp rx=0 tx=0).stop 2024-04-25T19:08:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 msgr2=0x7fc81c19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c19e9b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc81400da10 tx=0x7fc81400ded0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 shutdown_connections 2024-04-25T19:08:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc8000a3d10 0x7fc8000a61d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc81c10e420 0x7fc81c19e9b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81c108e10 0x7fc81c1a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 --2- 172.21.15.86:0/1096889165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c108490 0x7fc81c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.214+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 >> 172.21.15.86:0/1096889165 conn(0x7fc81c077d30 msgr2=0x7fc81c078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.215+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 shutdown_connections 2024-04-25T19:08:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:43.215+0000 7fc81be00640 1 -- 172.21.15.86:0/1096889165 wait complete. 2024-04-25T19:08:44.418 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:44 smithi183 ceph-mon[32344]: pgmap v176: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:08:44.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:44 smithi194 ceph-mon[32740]: pgmap v176: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:08:44.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:44 smithi086 ceph-mon[27783]: pgmap v176: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:08:46.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:46 smithi086 ceph-mon[27783]: pgmap v177: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T19:08:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:46 smithi183 ceph-mon[32344]: pgmap v177: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T19:08:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:46 smithi194 ceph-mon[32740]: pgmap v177: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T19:08:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:48 smithi183 ceph-mon[32344]: pgmap v178: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T19:08:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:48 smithi194 ceph-mon[32740]: pgmap v178: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T19:08:48.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:48 smithi086 ceph-mon[27783]: pgmap v178: 97 pgs: 97 active+clean; 2.3 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T19:08:50.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:50 smithi183 ceph-mon[32344]: pgmap v179: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-25T19:08:50.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:50 smithi194 ceph-mon[32740]: pgmap v179: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-25T19:08:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:50 smithi086 ceph-mon[27783]: pgmap v179: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-25T19:08:51.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:08:51.894 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:08:52.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:52 smithi086 ceph-mon[27783]: pgmap v180: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-25T19:08:52.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:52 smithi183 ceph-mon[32344]: pgmap v180: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-25T19:08:52.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:52 smithi194 ceph-mon[32740]: pgmap v180: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-25T19:08:54.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 -- 172.21.15.86:0/1866944407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464075590 msgr2=0x7fd4641136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:54.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 --2- 172.21.15.86:0/1866944407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464075590 0x7fd4641136d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd450009990 tx=0x7fd450009ca0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 -- 172.21.15.86:0/1866944407 shutdown_connections 2024-04-25T19:08:54.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 --2- 172.21.15.86:0/1866944407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464113c10 0x7fd464116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 --2- 172.21.15.86:0/1866944407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd464074c70 0x7fd464075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 --2- 172.21.15.86:0/1866944407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464075590 0x7fd4641136d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 -- 172.21.15.86:0/1866944407 >> 172.21.15.86:0/1866944407 conn(0x7fd464079ba0 msgr2=0x7fd464079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:54.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 -- 172.21.15.86:0/1866944407 shutdown_connections 2024-04-25T19:08:54.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.294+0000 7fd46a200700 1 -- 172.21.15.86:0/1866944407 wait complete. 2024-04-25T19:08:54.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.295+0000 7fd46a200700 1 Processor -- start 2024-04-25T19:08:54.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.295+0000 7fd46a200700 1 -- start start 2024-04-25T19:08:54.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd46a200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 0x7fd46419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:54.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd46a200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd464075590 0x7fd46419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:54.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd46a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464113c10 0x7fd4641a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd46a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd464119220 con 0x7fd464113c10 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd46a200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641190a0 con 0x7fd464075590 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd46a200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641193a0 con 0x7fd464074c70 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd463400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 0x7fd46419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd463400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 0x7fd46419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50820/0 (socket says 172.21.15.86:50820) 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd463400700 1 -- 172.21.15.86:0/1321555050 learned_addr learned my addr 172.21.15.86:0/1321555050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd462a00700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd464075590 0x7fd46419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.296+0000 7fd463e00700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464113c10 0x7fd4641a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463400700 1 -- 172.21.15.86:0/1321555050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd464075590 msgr2=0x7fd46419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463400700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd464075590 0x7fd46419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463400700 1 -- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464113c10 msgr2=0x7fd4641a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463400700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464113c10 0x7fd4641a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463400700 1 -- 172.21.15.86:0/1321555050 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641a4700 con 0x7fd464074c70 2024-04-25T19:08:54.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463e00700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464113c10 0x7fd4641a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:54.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd463400700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 0x7fd46419f6f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd45400eb30 tx=0x7fd45400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:54.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45400cbd0 con 0x7fd464074c70 2024-04-25T19:08:54.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4540044d0 con 0x7fd464074c70 2024-04-25T19:08:54.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4641a49f0 con 0x7fd464074c70 2024-04-25T19:08:54.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454003e30 con 0x7fd464074c70 2024-04-25T19:08:54.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.297+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641ac270 con 0x7fd464074c70 2024-04-25T19:08:54.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.298+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd464069730 con 0x7fd464074c70 2024-04-25T19:08:54.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.300+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd454004000 con 0x7fd464074c70 2024-04-25T19:08:54.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.301+0000 7fd45be00700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd4480a3db0 0x7fd4480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:54.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.302+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7fd4540d0c10 con 0x7fd464074c70 2024-04-25T19:08:54.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.302+0000 7fd462a00700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd4480a3db0 0x7fd4480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:54.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.302+0000 7fd462a00700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd4480a3db0 0x7fd4480a6270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4500098e0 tx=0x7fd450005960 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:54.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.306+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4540042f0 con 0x7fd464074c70 2024-04-25T19:08:54.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:54 smithi183 ceph-mon[32344]: pgmap v181: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-04-25T19:08:54.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:54 smithi194 ceph-mon[32740]: pgmap v181: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-04-25T19:08:54.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:54 smithi086 ceph-mon[27783]: pgmap v181: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-04-25T19:08:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.584+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4641a0a50 con 0x7fd464074c70 2024-04-25T19:08:54.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.585+0000 7fd45be00700 1 -- 172.21.15.86:0/1321555050 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fd45409d200 con 0x7fd464074c70 2024-04-25T19:08:54.587 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:08:54.588 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.588+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd4480a3db0 msgr2=0x7fd4480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.588+0000 7fd46a200700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd4480a3db0 0x7fd4480a6270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4500098e0 tx=0x7fd450005960 comp rx=0 tx=0).stop 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 msgr2=0x7fd46419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 0x7fd46419f6f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd45400eb30 tx=0x7fd45400ee40 comp rx=0 tx=0).stop 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 shutdown_connections 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd464113c10 0x7fd4641a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd4480a3db0 0x7fd4480a6270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd464074c70 0x7fd46419f6f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 --2- 172.21.15.86:0/1321555050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd464075590 0x7fd46419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:54.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 >> 172.21.15.86:0/1321555050 conn(0x7fd464079ba0 msgr2=0x7fd4641129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:54.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 shutdown_connections 2024-04-25T19:08:54.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:54.589+0000 7fd46a200700 1 -- 172.21.15.86:0/1321555050 wait complete. 2024-04-25T19:08:54.592 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:08:55.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:55 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1321555050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:55.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:55 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1321555050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:55.428 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id d42824d3 --release --await' 2024-04-25T19:08:55.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id d42824d3 --release --await 2024-04-25T19:08:55.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:55 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1321555050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:55.878 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:08:56.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:56 smithi086 ceph-mon[27783]: pgmap v182: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 -- 172.21.15.86:0/1899562560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34112c00 msgr2=0x7f3a34114ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 --2- 172.21.15.86:0/1899562560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34112c00 0x7f3a34114ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009cd0 tx=0x7f3a2c02a8f0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 -- 172.21.15.86:0/1899562560 shutdown_connections 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 --2- 172.21.15.86:0/1899562560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34112c00 0x7f3a34114ff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 --2- 172.21.15.86:0/1899562560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34076da0 0x7f3a341126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 --2- 172.21.15.86:0/1899562560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076480 0x7f3a34076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 -- 172.21.15.86:0/1899562560 >> 172.21.15.86:0/1899562560 conn(0x7f3a340ffa70 msgr2=0x7f3a34101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 -- 172.21.15.86:0/1899562560 shutdown_connections 2024-04-25T19:08:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.352+0000 7f3a33200640 1 -- 172.21.15.86:0/1899562560 wait complete. 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.353+0000 7f3a33200640 1 Processor -- start 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.353+0000 7f3a33200640 1 -- start start 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a33200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34076480 0x7f3a341a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a33200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076da0 0x7f3a341a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a33200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34112c00 0x7f3a3419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a33200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a34118620 con 0x7f3a34076da0 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a33200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341184a0 con 0x7f3a34076480 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a33200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341187a0 con 0x7f3a34112c00 2024-04-25T19:08:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34076480 0x7f3a341a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a31400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076da0 0x7f3a341a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a32800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34112c00 0x7f3a3419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34076480 0x7f3a341a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49492/0 (socket says 172.21.15.86:49492) 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.354+0000 7f3a31e00640 1 -- 172.21.15.86:0/1909509997 learned_addr learned my addr 172.21.15.86:0/1909509997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a31400640 1 -- 172.21.15.86:0/1909509997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34112c00 msgr2=0x7f3a3419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a31400640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34112c00 0x7f3a3419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a31400640 1 -- 172.21.15.86:0/1909509997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34076480 msgr2=0x7f3a341a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a31400640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34076480 0x7f3a341a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a31400640 1 -- 172.21.15.86:0/1909509997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3419f100 con 0x7f3a34076da0 2024-04-25T19:08:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a32800640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34112c00 0x7f3a3419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a31400640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076da0 0x7f3a341a4d10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000e9f0 tx=0x7f3a2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a2aa00640 1 -- 172.21.15.86:0/1909509997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000ccc0 con 0x7f3a34076da0 2024-04-25T19:08:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3419f3f0 con 0x7f3a34076da0 2024-04-25T19:08:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a2aa00640 1 -- 172.21.15.86:0/1909509997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a200108e0 con 0x7f3a34076da0 2024-04-25T19:08:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a2aa00640 1 -- 172.21.15.86:0/1909509997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20018880 con 0x7f3a34076da0 2024-04-25T19:08:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.355+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a341ab690 con 0x7f3a34076da0 2024-04-25T19:08:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 Processor -- start 2024-04-25T19:08:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a2aa00640 1 -- 172.21.15.86:0/1909509997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a20010430 con 0x7f3a34076da0 2024-04-25T19:08:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 -- start start 2024-04-25T19:08:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x55990613ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613f2d0 0x559906149650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x55990614bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599060a4900 con 0x559906149b90 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55990604aef0 con 0x55990613f2d0 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599060a08c0 con 0x55990613e9b0 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a32800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x55990613ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a31400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x55990614bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613f2d0 0x559906149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a32800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x55990613ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50846/0 (socket says 172.21.15.86:50846) 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.359+0000 7f3a32800640 1 -- 172.21.15.86:0/2826908250 learned_addr learned my addr 172.21.15.86:0/2826908250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a31e00640 1 -- 172.21.15.86:0/2826908250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 msgr2=0x55990613ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a31e00640 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x55990613ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a31e00640 1 -- 172.21.15.86:0/2826908250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 msgr2=0x55990614bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a31e00640 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x55990614bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a31e00640 1 -- 172.21.15.86:0/2826908250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559905db7210 con 0x55990613f2d0 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a31e00640 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613f2d0 0x559906149650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009cf0 tx=0x7f3a24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f35a7ec7aaf83c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a2aa00640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a180a3d10 0x7f3a180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.360+0000 7f3a1f000640 1 -- 172.21.15.86:0/2826908250 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2401e070 con 0x55990613f2d0 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.361+0000 7f3a1f000640 1 -- 172.21.15.86:0/2826908250 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a240198c0 con 0x55990613f2d0 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.361+0000 7f3a1f000640 1 -- 172.21.15.86:0/2826908250 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240186f0 con 0x55990613f2d0 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.361+0000 7f3a2aa00640 1 -- 172.21.15.86:0/1909509997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f3a200d0b30 con 0x7f3a34076da0 2024-04-25T19:08:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.361+0000 7f3a32800640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a180a3d10 0x7f3a180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.361+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2826908250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613f2d0 msgr2=0x559906149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.361+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613f2d0 0x559906149650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009cf0 tx=0x7f3a24011810 comp rx=0 tx=0).stop 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a32800640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a180a3d10 0x7f3a180a61d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c031040 tx=0x7f3a2c005d80 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2826908250 shutdown_connections 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x55990614bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613f2d0 0x559906149650 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2826908250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x55990613ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2826908250 >> 172.21.15.86:0/2826908250 conn(0x55990613aec0 msgr2=0x55990613b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2826908250 shutdown_connections 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2826908250 wait complete. 2024-04-25T19:08:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 -- start start 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.362+0000 7f3a3b7e5740 1 Processor -- start 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x559906146b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x559906147040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 0x559906147580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55990614c7c0 con 0x55990613f2d0 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55990605e310 con 0x559906149b90 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55990614c940 con 0x55990613e9b0 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a3b7e5740 10 client.0 _ll_register_callbacks cb 0x55990602ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a32800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 0x559906147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a31400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x559906147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x559906146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a32800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 0x559906147580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49496/0 (socket says 172.21.15.86:49496) 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.363+0000 7f3a32800640 1 -- 172.21.15.86:0/1552561329 learned_addr learned my addr 172.21.15.86:0/1552561329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a32800640 1 -- 172.21.15.86:0/1552561329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 msgr2=0x559906146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a32800640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x559906146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a32800640 1 -- 172.21.15.86:0/1552561329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 msgr2=0x559906147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a32800640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x559906147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a32800640 1 -- 172.21.15.86:0/1552561329 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559905e83620 con 0x559906149b90 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a31400640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x559906147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a31e00640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x559906146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a32800640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 0x559906147580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c02ae10 tx=0x7f3a2c02a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6483d6c488a41e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a13400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a13400640 1 -- 172.21.15.86:0/1552561329 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c053440 con 0x559906149b90 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a240198c0 con 0x559906149b90 2024-04-25T19:08:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.364+0000 7f3a13400640 1 -- 172.21.15.86:0/1552561329 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f3a2c049420 con 0x559906149b90 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a13400640 1 -- 172.21.15.86:0/1552561329 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a2c049620 con 0x559906149b90 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 10 client.25967 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559906158a00 con 0x559906149b90 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a13400640 1 -- 172.21.15.86:0/1552561329 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a240198c0 con 0x559906149b90 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a13400640 1 -- 172.21.15.86:0/1552561329 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a2c051020 con 0x559906149b90 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 10 client.25967 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 20 client.25967 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559906159f60 0x55990615c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 4 client.25967 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.365+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599060b88b0 con 0x559906159f60 2024-04-25T19:08:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.366+0000 7f3a31e00640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559906159f60 0x55990615c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.366+0000 7f3a31e00640 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559906159f60 0x55990615c350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5092aebc5a0973b server_cookie=37fcd47a8fd0d7a3 in_seq=0 out_seq=0 2024-04-25T19:08:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.366+0000 7f3a13400640 10 client.25967 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.367+0000 7f3a13400640 1 -- 172.21.15.86:0/1552561329 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599060b88b0 con 0x559906159f60 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.367+0000 7f3a13400640 10 client.25967 handle_command_reply: tid=0 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.367+0000 7f3a3b7e5740 1 client.25967 shutdown 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 msgr2=0x559906147580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 0x559906147580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c02ae10 tx=0x7f3a2c02a8f0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 shutdown_connections 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559906159f60 0x55990615c350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559906149b90 0x559906147580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x559906147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/1552561329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613e9b0 0x559906146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a13400640 0 client.25967 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 >> 172.21.15.86:0/1552561329 conn(0x55990613aec0 msgr2=0x559906140bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 shutdown_connections 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/1552561329 wait complete. 2024-04-25T19:08:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.368+0000 7f3a3b7e5740 20 client.25967 trim_cache size 0 max 16384 2024-04-25T19:08:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.384+0000 7f3a3b7e5740 1 Processor -- start 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 -- start start 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990613a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613f2d0 0x559906155680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x559906161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341b4230 con 0x559906149b90 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341b5d70 con 0x55990613e9b0 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341b4900 con 0x55990613f2d0 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x559906161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990613a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a31400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613f2d0 0x559906155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990613a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49502/0 (socket says 172.21.15.86:49502) 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a31e00640 1 -- 172.21.15.86:0/2061107612 learned_addr learned my addr 172.21.15.86:0/2061107612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.385+0000 7f3a32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x559906161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34872/0 (socket says 172.21.15.86:34872) 2024-04-25T19:08:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31e00640 1 -- 172.21.15.86:0/2061107612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613f2d0 msgr2=0x559906155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31e00640 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613f2d0 0x559906155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31e00640 1 -- 172.21.15.86:0/2061107612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 msgr2=0x559906161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31e00640 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x559906161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31e00640 1 -- 172.21.15.86:0/2061107612 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990613f100 con 0x55990613e9b0 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31400640 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613f2d0 0x559906155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a32800640 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x559906161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a31e00640 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990613a930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009250 tx=0x7f3a24014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca06303510786e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a1e600640 1 -- 172.21.15.86:0/2061107612 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2401e040 con 0x55990613e9b0 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a1e600640 1 -- 172.21.15.86:0/2061107612 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2402a070 con 0x55990613e9b0 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.386+0000 7f3a1e600640 1 -- 172.21.15.86:0/2061107612 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2402b790 con 0x55990613e9b0 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2061107612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 msgr2=0x55990613a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990613a930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009250 tx=0x7f3a24014000 comp rx=0 tx=0).stop 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2061107612 shutdown_connections 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559906149b90 0x559906161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55990613f2d0 0x559906155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/2061107612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990613a930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2061107612 >> 172.21.15.86:0/2061107612 conn(0x55990613aec0 msgr2=0x5599061453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2061107612 shutdown_connections 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.387+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/2061107612 wait complete. 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 -- start start 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 Processor -- start 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990614ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x55990614f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559906149b90 0x5599061be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341b56a0 con 0x55990613f2d0 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341b4fd0 con 0x55990613e9b0 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a341b2210 con 0x559906149b90 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.388+0000 7f3a3b7e5740 10 client.0 _ll_register_callbacks cb 0x7f3a2c030c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990614ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a32800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559906149b90 0x5599061be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990614ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49508/0 (socket says 172.21.15.86:49508) 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 -- 172.21.15.86:0/3576464422 learned_addr learned my addr 172.21.15.86:0/3576464422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31400640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x55990614f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 -- 172.21.15.86:0/3576464422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559906149b90 msgr2=0x5599061be480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559906149b90 0x5599061be480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 -- 172.21.15.86:0/3576464422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 msgr2=0x55990614f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x55990614f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a31e00640 1 -- 172.21.15.86:0/3576464422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599061be9c0 con 0x55990613e9b0 2024-04-25T19:08:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.389+0000 7f3a32800640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559906149b90 0x5599061be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.390+0000 7f3a31e00640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990614ed50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402b3f0 tx=0x7f3a24009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7f3d71f73ae3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.390+0000 7f3a13400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.390+0000 7f3a13400640 1 -- 172.21.15.86:0/3576464422 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2403a060 con 0x55990613e9b0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.390+0000 7f3a13400640 1 -- 172.21.15.86:0/3576464422 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f3a24016040 con 0x55990613e9b0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.390+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a2402a070 con 0x55990613e9b0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.391+0000 7f3a13400640 1 -- 172.21.15.86:0/3576464422 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a2401e040 con 0x55990613e9b0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.391+0000 7f3a3b7e5740 10 client.25973 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.391+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599061bfc40 con 0x55990613e9b0 2024-04-25T19:08:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.391+0000 7f3a13400640 1 -- 172.21.15.86:0/3576464422 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2402a070 con 0x55990613e9b0 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.391+0000 7f3a13400640 1 -- 172.21.15.86:0/3576464422 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a24034070 con 0x55990613e9b0 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a3b7e5740 10 client.25973 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a3b7e5740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a3b7e5740 20 client.25973 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599061c0160 0x5599061c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a3b7e5740 4 client.25973 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599060b2320 con 0x5599061c0160 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.392+0000 7f3a31400640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599061c0160 0x5599061c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.393+0000 7f3a31400640 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599061c0160 0x5599061c2550 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68eae96d895c3604 server_cookie=b2a98d1968399a53 in_seq=0 out_seq=0 2024-04-25T19:08:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.393+0000 7f3a13400640 10 client.25973 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a13400640 1 -- 172.21.15.86:0/3576464422 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599060b2320 con 0x5599061c0160 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a13400640 10 client.25973 handle_command_reply: tid=0 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 client.25973 shutdown 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 msgr2=0x55990614ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990614ed50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402b3f0 tx=0x7f3a24009990 comp rx=0 tx=0).stop 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 shutdown_connections 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5599061c0160 0x5599061c2550 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559906149b90 0x5599061be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55990613f2d0 0x55990614f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a13400640 0 client.25973 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 --2- 172.21.15.86:0/3576464422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55990613e9b0 0x55990614ed50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 >> 172.21.15.86:0/3576464422 conn(0x55990613aec0 msgr2=0x559906140bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 shutdown_connections 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 1 -- 172.21.15.86:0/3576464422 wait complete. 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.404+0000 7f3a3b7e5740 20 client.25973 trim_cache size 0 max 16384 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:08:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:08:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:08:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a180a3d10 msgr2=0x7f3a180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a180a3d10 0x7f3a180a61d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c031040 tx=0x7f3a2c005d80 comp rx=0 tx=0).stop 2024-04-25T19:08:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076da0 msgr2=0x7f3a341a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076da0 0x7f3a341a4d10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000e9f0 tx=0x7f3a2000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 shutdown_connections 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a180a3d10 0x7f3a180a61d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a34112c00 0x7f3a3419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a34076da0 0x7f3a341a4d10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 --2- 172.21.15.86:0/1909509997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a34076480 0x7f3a341a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 >> 172.21.15.86:0/1909509997 conn(0x7f3a340ffa70 msgr2=0x7f3a34101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.405+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 shutdown_connections 2024-04-25T19:08:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:08:56.406+0000 7f3a33200640 1 -- 172.21.15.86:0/1909509997 wait complete. 2024-04-25T19:08:56.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:56 smithi183 ceph-mon[32344]: pgmap v182: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-04-25T19:08:56.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:56 smithi194 ceph-mon[32740]: pgmap v182: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-04-25T19:08:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 -- 172.21.15.86:0/3687774948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4528109c80 msgr2=0x7f452810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 --2- 172.21.15.86:0/3687774948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4528109c80 0x7f452810dcd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45140098f0 tx=0x7f4514009c00 comp rx=0 tx=0).stop 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 -- 172.21.15.86:0/3687774948 shutdown_connections 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 --2- 172.21.15.86:0/3687774948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810e3b0 0x7f4528115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 --2- 172.21.15.86:0/3687774948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45281092d0 0x7f45281096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 --2- 172.21.15.86:0/3687774948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4528109c80 0x7f452810dcd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 -- 172.21.15.86:0/3687774948 >> 172.21.15.86:0/3687774948 conn(0x7f4528076210 msgr2=0x7f4528107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 -- 172.21.15.86:0/3687774948 shutdown_connections 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.032+0000 7f452f400700 1 -- 172.21.15.86:0/3687774948 wait complete. 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 Processor -- start 2024-04-25T19:08:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 -- start start 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 0x7f45281a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4528109c80 0x7f45281a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810e3b0 0x7f452819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4528119270 con 0x7f452810e3b0 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45281190f0 con 0x7f45281092d0 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.033+0000 7f452f400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45281193f0 con 0x7f4528109c80 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 0x7f45281a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 0x7f45281a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49532/0 (socket says 172.21.15.86:49532) 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 -- 172.21.15.86:0/2970670812 learned_addr learned my addr 172.21.15.86:0/2970670812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452d400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810e3b0 0x7f452819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f4527e00700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4528109c80 0x7f45281a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 -- 172.21.15.86:0/2970670812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4528109c80 msgr2=0x7f45281a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4528109c80 0x7f45281a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 -- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810e3b0 msgr2=0x7f452819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810e3b0 0x7f452819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.034+0000 7f452ca00700 1 -- 172.21.15.86:0/2970670812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819fd90 con 0x7f45281092d0 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.035+0000 7f452ca00700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 0x7f45281a5420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9a0 tx=0x7f451c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.035+0000 7f4525600700 1 -- 172.21.15.86:0/2970670812 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c00ca10 con 0x7f45281092d0 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.035+0000 7f4525600700 1 -- 172.21.15.86:0/2970670812 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c00cbb0 con 0x7f45281092d0 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.035+0000 7f4525600700 1 -- 172.21.15.86:0/2970670812 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c018680 con 0x7f45281092d0 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.035+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45281a0080 con 0x7f45281092d0 2024-04-25T19:08:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.035+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281ac2f0 con 0x7f45281092d0 2024-04-25T19:08:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.039+0000 7f4525600700 1 -- 172.21.15.86:0/2970670812 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f451c010430 con 0x7f45281092d0 2024-04-25T19:08:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 Processor -- start 2024-04-25T19:08:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 -- start start 2024-04-25T19:08:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4525600700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45100a3e30 0x7f45100a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea3ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea49bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4525600700 1 -- 172.21.15.86:0/2970670812 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f451c0d0990 con 0x7f45281092d0 2024-04-25T19:08:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561ae991f90 con 0x5561aea3f240 2024-04-25T19:08:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5561ae938830 con 0x5561aea3e920 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561ae98df50 con 0x5561aea4a0f0 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.040+0000 7f452d400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45100a3e30 0x7f45100a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea3ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f4527e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea49bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea4c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50894/0 (socket says 172.21.15.86:50894) 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452d400700 1 -- 172.21.15.86:0/422272261 learned_addr learned my addr 172.21.15.86:0/422272261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452ca00700 1 -- 172.21.15.86:0/422272261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 msgr2=0x5561aea4c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452ca00700 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea4c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452ca00700 1 -- 172.21.15.86:0/422272261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 msgr2=0x5561aea49bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452ca00700 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea49bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.041+0000 7f452ca00700 1 -- 172.21.15.86:0/422272261 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ae92e780 con 0x5561aea3e920 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.042+0000 7f452ca00700 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea3ed00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f451c0ce090 tx=0x7f451c09c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cdc82f28ec24da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.042+0000 7f452d400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45100a3e30 0x7f45100a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4520007bb0 tx=0x7f45200092a0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.042+0000 7f451a000700 1 -- 172.21.15.86:0/422272261 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a5550 con 0x5561aea3e920 2024-04-25T19:08:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.042+0000 7f451a000700 1 -- 172.21.15.86:0/422272261 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c09abe0 con 0x5561aea3e920 2024-04-25T19:08:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.042+0000 7f451a000700 1 -- 172.21.15.86:0/422272261 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a5320 con 0x5561aea3e920 2024-04-25T19:08:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 -- 172.21.15.86:0/422272261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 msgr2=0x5561aea3ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea3ed00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f451c0ce090 tx=0x7f451c09c350 comp rx=0 tx=0).stop 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 -- 172.21.15.86:0/422272261 shutdown_connections 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea4c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea3ed00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/422272261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 -- 172.21.15.86:0/422272261 >> 172.21.15.86:0/422272261 conn(0x5561ae96d6a0 msgr2=0x5561aea3add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 -- 172.21.15.86:0/422272261 shutdown_connections 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 -- 172.21.15.86:0/422272261 wait complete. 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.043+0000 7f4539d1fb80 1 -- start start 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 Processor -- start 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3e920 0x5561aea49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 0x5561aea46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561aea4cbb0 con 0x5561aea3e920 2024-04-25T19:08:58.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5561ae94b960 con 0x5561aea3f240 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561aea4cd30 con 0x5561aea4a0f0 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.044+0000 7f4539d1fb80 10 client.0 _ll_register_callbacks cb 0x5561ae92a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3e920 0x5561aea49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f4527e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 0x5561aea46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f4527e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 0x5561aea46b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49542/0 (socket says 172.21.15.86:49542) 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f4527e00700 1 -- 172.21.15.86:0/2252503238 learned_addr learned my addr 172.21.15.86:0/2252503238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3e920 0x5561aea49880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34916/0 (socket says 172.21.15.86:34916) 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 -- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 msgr2=0x5561aea46b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 0x5561aea46b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 -- 172.21.15.86:0/2252503238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3e920 msgr2=0x5561aea49880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3e920 0x5561aea49880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 -- 172.21.15.86:0/2252503238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ae8df9d0 con 0x5561aea4a0f0 2024-04-25T19:08:58.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f4527e00700 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 0x5561aea46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f452d400700 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea470c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f45200095e0 tx=0x7f4520004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5081ea8dc3dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.045+0000 7f450ea00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:08:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f450ea00700 1 -- 172.21.15.86:0/2252503238 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520028020 con 0x5561aea4a0f0 2024-04-25T19:08:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451c09abe0 con 0x5561aea4a0f0 2024-04-25T19:08:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f450ea00700 1 -- 172.21.15.86:0/2252503238 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f4520003a00 con 0x5561aea4a0f0 2024-04-25T19:08:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f450ea00700 1 -- 172.21.15.86:0/2252503238 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4520021070 con 0x5561aea4a0f0 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f4539d1fb80 10 client.25981 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561ae928bf0 con 0x5561aea4a0f0 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.046+0000 7f450ea00700 1 -- 172.21.15.86:0/2252503238 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c09abe0 con 0x5561aea4a0f0 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f450ea00700 1 -- 172.21.15.86:0/2252503238 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f45200045f0 con 0x5561aea4a0f0 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f4539d1fb80 10 client.25981 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f4539d1fb80 10 client.25981 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f4539d1fb80 20 client.25981 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aea59940 0x5561aea5bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f4539d1fb80 4 client.25981 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ae99f780 con 0x5561aea59940 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.047+0000 7f452ca00700 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aea59940 0x5561aea5bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.048+0000 7f452ca00700 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aea59940 0x5561aea5bd80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d12cb92259b3fbc server_cookie=139e798243fd3d00 in_seq=0 out_seq=0 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.048+0000 7f450ea00700 10 client.25981 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f450ea00700 1 -- 172.21.15.86:0/2252503238 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561ae99f780 con 0x5561aea59940 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f450ea00700 10 client.25981 handle_command_reply: tid=0 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 client.25981 shutdown 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 msgr2=0x5561aea470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea470c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f45200095e0 tx=0x7f4520004b40 comp rx=0 tx=0).stop 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 shutdown_connections 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aea59940 0x5561aea5bd80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea4a0f0 0x5561aea470c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3e920 0x5561aea49880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f450ea00700 0 client.25981 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/2252503238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3f240 0x5561aea46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 >> 172.21.15.86:0/2252503238 conn(0x5561ae96d6a0 msgr2=0x5561aea3a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 shutdown_connections 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 1 -- 172.21.15.86:0/2252503238 wait complete. 2024-04-25T19:08:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.049+0000 7f4539d1fb80 20 client.25981 trim_cache size 0 max 16384 2024-04-25T19:08:58.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 Processor -- start 2024-04-25T19:08:58.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 -- start start 2024-04-25T19:08:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea55ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3f240 0x5561aea61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea4a0f0 0x5561aea61b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b3b60 con 0x5561aea4a0f0 2024-04-25T19:08:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b56a0 con 0x5561aea3e920 2024-04-25T19:08:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b4230 con 0x5561aea3f240 2024-04-25T19:08:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea55ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.071+0000 7f452d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea4a0f0 0x5561aea61b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea55ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:49556/0 (socket says 172.21.15.86:49556) 2024-04-25T19:08:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 -- 172.21.15.86:0/3398794603 learned_addr learned my addr 172.21.15.86:0/3398794603 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f4527e00700 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3f240 0x5561aea61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 -- 172.21.15.86:0/3398794603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3f240 msgr2=0x5561aea61650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3f240 0x5561aea61650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 -- 172.21.15.86:0/3398794603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea4a0f0 msgr2=0x5561aea61b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea4a0f0 0x5561aea61b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 -- 172.21.15.86:0/3398794603 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561aea450f0 con 0x5561aea3e920 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f452ca00700 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea55ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f451c0ce140 tx=0x7f451c010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db5abf45581f4c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f4519600700 1 -- 172.21.15.86:0/3398794603 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0bb070 con 0x5561aea3e920 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f4519600700 1 -- 172.21.15.86:0/3398794603 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c0870b0 con 0x5561aea3e920 2024-04-25T19:08:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.072+0000 7f4519600700 1 -- 172.21.15.86:0/3398794603 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a50c0 con 0x5561aea3e920 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 -- 172.21.15.86:0/3398794603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 msgr2=0x5561aea55ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea55ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f451c0ce140 tx=0x7f451c010c10 comp rx=0 tx=0).stop 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 -- 172.21.15.86:0/3398794603 shutdown_connections 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea4a0f0 0x5561aea61b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea3e920 0x5561aea55ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/3398794603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3f240 0x5561aea61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 -- 172.21.15.86:0/3398794603 >> 172.21.15.86:0/3398794603 conn(0x5561ae96d6a0 msgr2=0x5561aea410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 -- 172.21.15.86:0/3398794603 shutdown_connections 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.073+0000 7f4539d1fb80 1 -- 172.21.15.86:0/3398794603 wait complete. 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.074+0000 7f4539d1fb80 1 -- start start 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.074+0000 7f4539d1fb80 1 Processor -- start 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.074+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 0x5561aea524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea4ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4539d1fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea4a0f0 0x5561aea4f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b4fd0 con 0x5561aea3f240 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b4900 con 0x5561aea4a0f0 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4539d1fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b1b00 con 0x5561aea3e920 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4539d1fb80 10 client.0 _ll_register_callbacks cb 0x5561ae92a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 0x5561aea524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f4527e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea4ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea4a0f0 0x5561aea4f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 0x5561aea524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:50934/0 (socket says 172.21.15.86:50934) 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 -- 172.21.15.86:0/1686457120 learned_addr learned my addr 172.21.15.86:0/1686457120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 -- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea4a0f0 msgr2=0x5561aea4f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea4a0f0 0x5561aea4f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 -- 172.21.15.86:0/1686457120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 msgr2=0x5561aea4ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea4ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.075+0000 7f452ca00700 1 -- 172.21.15.86:0/1686457120 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561aeabfed0 con 0x5561aea3e920 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f452ca00700 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 0x5561aea524f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f451c09d0d0 tx=0x7f451c09c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce51646549ea59f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f450ea00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f450ea00700 1 -- 172.21.15.86:0/1686457120 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c080060 con 0x5561aea3e920 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f450ea00700 1 -- 172.21.15.86:0/1686457120 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f451c09cc50 con 0x5561aea3e920 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451c0870b0 con 0x5561aea3e920 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f450ea00700 1 -- 172.21.15.86:0/1686457120 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f451c0bb070 con 0x5561aea3e920 2024-04-25T19:08:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f4539d1fb80 10 client.25987 fetch_fsmap learned FSMap version 29 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561aeac10c0 con 0x5561aea3e920 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.076+0000 7f450ea00700 1 -- 172.21.15.86:0/1686457120 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c0870b0 con 0x5561aea3e920 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f450ea00700 1 -- 172.21.15.86:0/1686457120 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f451c098630 con 0x5561aea3e920 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4539d1fb80 10 client.25987 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4539d1fb80 10 client.25987 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4539d1fb80 20 client.25987 populate_metadata read hostname 'smithi086' 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aeac12f0 0x5561aeac36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4539d1fb80 4 client.25987 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d42824d3", "release": true, "await": true}] 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d42824d3", "release": true, "await": true}) -- 0x5561ae999820 con 0x5561aeac12f0 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.077+0000 7f4527e00700 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aeac12f0 0x5561aeac36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.078+0000 7f4527e00700 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aeac12f0 0x5561aeac36e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367b19dbbdd1c8c7 server_cookie=a6710637f2dd2c1b in_seq=0 out_seq=0 2024-04-25T19:08:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.078+0000 7f450ea00700 10 client.25987 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f450ea00700 1 -- 172.21.15.86:0/1686457120 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5561ae999820 con 0x5561aeac12f0 2024-04-25T19:08:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f450ea00700 10 client.25987 handle_command_reply: tid=0 2024-04-25T19:08:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f4539d1fb80 1 client.25987 shutdown 2024-04-25T19:08:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 msgr2=0x5561aea524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 0x5561aea524f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f451c09d0d0 tx=0x7f451c09c370 comp rx=0 tx=0).stop 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 shutdown_connections 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.081+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561aeac12f0 0x5561aeac36e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561aea4a0f0 0x5561aea4f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aea3e920 0x5561aea524f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f450ea00700 0 client.25987 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 1 --2- 172.21.15.86:0/1686457120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561aea3f240 0x5561aea4ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 >> 172.21.15.86:0/1686457120 conn(0x5561ae96d6a0 msgr2=0x5561aea483b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 shutdown_connections 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 1 -- 172.21.15.86:0/1686457120 wait complete. 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.082+0000 7f4539d1fb80 20 client.25987 trim_cache size 0 max 16384 2024-04-25T19:08:58.084 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 16, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "d42824d3": { 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 16, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 78.6, 2024-04-25T19:08:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:58.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45100a3e30 msgr2=0x7f45100a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45100a3e30 0x7f45100a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4520007bb0 tx=0x7f45200092a0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 msgr2=0x7f45281a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 0x7f45281a5420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9a0 tx=0x7f451c00ecb0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 shutdown_connections 2024-04-25T19:08:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810e3b0 0x7f452819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281092d0 0x7f45281a5420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45100a3e30 0x7f45100a62f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 --2- 172.21.15.86:0/2970670812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4528109c80 0x7f45281a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 >> 172.21.15.86:0/2970670812 conn(0x7f4528076210 msgr2=0x7f45281086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 shutdown_connections 2024-04-25T19:08:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:08:58.083+0000 7f452f400700 1 -- 172.21.15.86:0/2970670812 wait complete. 2024-04-25T19:08:58.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:08:58 smithi183 ceph-mon[32344]: pgmap v183: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-04-25T19:08:58.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:08:58 smithi194 ceph-mon[32740]: pgmap v183: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-04-25T19:08:58.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:08:58 smithi086 ceph-mon[27783]: pgmap v183: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 11 op/s 2024-04-25T19:08:59.188 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd42824d3', seconds elapsed: 33.9 2024-04-25T19:08:59.188 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.6 2024-04-25T19:08:59.188 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:08:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:00.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:00 smithi194 ceph-mon[32740]: pgmap v184: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T19:09:00.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:00 smithi086 ceph-mon[27783]: pgmap v184: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T19:09:00.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:00 smithi183 ceph-mon[32344]: pgmap v184: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.780+0000 7f892be00700 1 -- 172.21.15.86:0/1844919706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89241063f0 msgr2=0x7f892410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.780+0000 7f892be00700 1 --2- 172.21.15.86:0/1844919706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89241063f0 0x7f892410dfc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c240 tx=0x7f891c00c550 comp rx=0 tx=0).stop 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.780+0000 7f892be00700 1 -- 172.21.15.86:0/1844919706 shutdown_connections 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.781+0000 7f892be00700 1 --2- 172.21.15.86:0/1844919706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89241063f0 0x7f892410dfc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.781+0000 7f892be00700 1 --2- 172.21.15.86:0/1844919706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8924101360 0x7f8924101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.781+0000 7f892be00700 1 --2- 172.21.15.86:0/1844919706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8924101d10 0x7f8924105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.781+0000 7f892be00700 1 -- 172.21.15.86:0/1844919706 >> 172.21.15.86:0/1844919706 conn(0x7f8924077fb0 msgr2=0x7f89240ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:01.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.781+0000 7f892be00700 1 -- 172.21.15.86:0/1844919706 shutdown_connections 2024-04-25T19:09:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.781+0000 7f892be00700 1 -- 172.21.15.86:0/1844919706 wait complete. 2024-04-25T19:09:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.782+0000 7f892be00700 1 Processor -- start 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.782+0000 7f892be00700 1 -- start start 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f892be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 0x7f892419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f892be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8924101d10 0x7f892419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f892be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89241063f0 0x7f892419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f892be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8924110a70 con 0x7f8924101360 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f892be00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89241108f0 con 0x7f89241063f0 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f892be00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8924110bf0 con 0x7f8924101d10 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 0x7f892419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89241063f0 0x7f892419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 0x7f892419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48584/0 (socket says 172.21.15.86:48584) 2024-04-25T19:09:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929400700 1 -- 172.21.15.86:0/687286587 learned_addr learned my addr 172.21.15.86:0/687286587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8928a00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8924101d10 0x7f892419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929e00700 1 -- 172.21.15.86:0/687286587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8924101d10 msgr2=0x7f892419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929e00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8924101d10 0x7f892419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929e00700 1 -- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 msgr2=0x7f892419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929e00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 0x7f892419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929e00700 1 -- 172.21.15.86:0/687286587 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241a0310 con 0x7f89241063f0 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.783+0000 7f8929400700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 0x7f892419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.784+0000 7f8929e00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89241063f0 0x7f892419fbd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c8e0 tx=0x7f891c008270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.784+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c008ae0 con 0x7f89241063f0 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.784+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891c0050d0 con 0x7f89241063f0 2024-04-25T19:09:01.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.784+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241a05a0 con 0x7f89241063f0 2024-04-25T19:09:01.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.784+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c007700 con 0x7f89241063f0 2024-04-25T19:09:01.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.784+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241a7e80 con 0x7f89241063f0 2024-04-25T19:09:01.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.785+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89241027f0 con 0x7f89241063f0 2024-04-25T19:09:01.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.788+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f891c005280 con 0x7f89241063f0 2024-04-25T19:09:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.789+0000 7f8922000700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f890c0a3e30 0x7f890c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:01.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.789+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7f891c010020 con 0x7f89241063f0 2024-04-25T19:09:01.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.789+0000 7f8929400700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f890c0a3e30 0x7f890c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:01.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.790+0000 7f8929400700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f890c0a3e30 0x7f890c0a62f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8918011590 tx=0x7f8918009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:01.792+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f891c09e410 con 0x7f89241063f0 2024-04-25T19:09:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.066+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89241a8050 con 0x7f89241063f0 2024-04-25T19:09:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.068+0000 7f8922000700 1 -- 172.21.15.86:0/687286587 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f891c09dd10 con 0x7f89241063f0 2024-04-25T19:09:02.070 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:09:02.070 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f890c0a3e30 msgr2=0x7f890c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f890c0a3e30 0x7f890c0a62f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8918011590 tx=0x7f8918009260 comp rx=0 tx=0).stop 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89241063f0 msgr2=0x7f892419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89241063f0 0x7f892419fbd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c8e0 tx=0x7f891c008270 comp rx=0 tx=0).stop 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 shutdown_connections 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89241063f0 0x7f892419fbd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8924101360 0x7f892419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f890c0a3e30 0x7f890c0a62f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 --2- 172.21.15.86:0/687286587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8924101d10 0x7f892419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 >> 172.21.15.86:0/687286587 conn(0x7f8924077fb0 msgr2=0x7f8924105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 shutdown_connections 2024-04-25T19:09:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:02.071+0000 7f892be00700 1 -- 172.21.15.86:0/687286587 wait complete. 2024-04-25T19:09:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:09:02.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:02 smithi086 ceph-mon[27783]: pgmap v185: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T19:09:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:02 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/687286587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:09:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:02 smithi194 ceph-mon[32740]: pgmap v185: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T19:09:02.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:02 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/687286587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:09:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:02 smithi183 ceph-mon[32344]: pgmap v185: 97 pgs: 97 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T19:09:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:02 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/687286587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:09:02.879 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 78.6 --await-for 42.0' 2024-04-25T19:09:02.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 78.6 --await-for 42.0 2024-04-25T19:09:03.329 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:04.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:04 smithi194 ceph-mon[32740]: pgmap v186: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-25T19:09:04.501 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:04 smithi086 ceph-mon[27783]: pgmap v186: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-25T19:09:04.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:04 smithi183 ceph-mon[32344]: pgmap v186: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-04-25T19:09:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 -- 172.21.15.86:0/229650341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad410e3b0 msgr2=0x7ffad4115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 --2- 172.21.15.86:0/229650341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad410e3b0 0x7ffad4115f80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00b0d0 tx=0x7ffacc00b3e0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 -- 172.21.15.86:0/229650341 shutdown_connections 2024-04-25T19:09:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 --2- 172.21.15.86:0/229650341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad410e3b0 0x7ffad4115f80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 --2- 172.21.15.86:0/229650341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad41092d0 0x7ffad41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 --2- 172.21.15.86:0/229650341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad4109c80 0x7ffad410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 -- 172.21.15.86:0/229650341 >> 172.21.15.86:0/229650341 conn(0x7ffad4076210 msgr2=0x7ffad4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 -- 172.21.15.86:0/229650341 shutdown_connections 2024-04-25T19:09:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.716+0000 7ffada600700 1 -- 172.21.15.86:0/229650341 wait complete. 2024-04-25T19:09:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.717+0000 7ffada600700 1 Processor -- start 2024-04-25T19:09:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.717+0000 7ffada600700 1 -- start start 2024-04-25T19:09:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffada600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad41092d0 0x7ffad419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffada600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad4109c80 0x7ffad419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffada600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad410e3b0 0x7ffad41a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffada600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41192c0 con 0x7ffad41092d0 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffada600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4119140 con 0x7ffad4109c80 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffada600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4119440 con 0x7ffad410e3b0 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffad3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad41092d0 0x7ffad419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad4109c80 0x7ffad419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffad3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad41092d0 0x7ffad419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48600/0 (socket says 172.21.15.86:48600) 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.718+0000 7ffad3400700 1 -- 172.21.15.86:0/2989927759 learned_addr learned my addr 172.21.15.86:0/2989927759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad3e00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad410e3b0 0x7ffad41a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad2a00700 1 -- 172.21.15.86:0/2989927759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad410e3b0 msgr2=0x7ffad41a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad2a00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad410e3b0 0x7ffad41a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad2a00700 1 -- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad41092d0 msgr2=0x7ffad419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad2a00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad41092d0 0x7ffad419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad2a00700 1 -- 172.21.15.86:0/2989927759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad41a4620 con 0x7ffad4109c80 2024-04-25T19:09:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffad2a00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad4109c80 0x7ffad419fb50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffac000e9c0 tx=0x7ffac000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.719+0000 7ffacbe00700 1 -- 172.21.15.86:0/2989927759 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004220 con 0x7ffad4109c80 2024-04-25T19:09:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.720+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41a4910 con 0x7ffad4109c80 2024-04-25T19:09:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.720+0000 7ffacbe00700 1 -- 172.21.15.86:0/2989927759 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac00043c0 con 0x7ffad4109c80 2024-04-25T19:09:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.720+0000 7ffacbe00700 1 -- 172.21.15.86:0/2989927759 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0005020 con 0x7ffad4109c80 2024-04-25T19:09:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.720+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41ac190 con 0x7ffad4109c80 2024-04-25T19:09:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.723+0000 7ffacbe00700 1 -- 172.21.15.86:0/2989927759 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffac0004750 con 0x7ffad4109c80 2024-04-25T19:09:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.725+0000 7ffae4faeb80 1 Processor -- start 2024-04-25T19:09:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.725+0000 7ffae4faeb80 1 -- start start 2024-04-25T19:09:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.725+0000 7ffacbe00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffab80a3e50 0x7ffab80a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 0x561bcfd92020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfd94950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffacbe00700 1 -- 172.21.15.86:0/2989927759 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ffac00d07a0 con 0x7ffad4109c80 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bcfcda020 con 0x561bcfd86dd0 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bcfc808c0 con 0x561bcfd92560 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561bcfcd5fe0 con 0x561bcfd876f0 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffad3e00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffab80a3e50 0x7ffab80a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffad3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 0x561bcfd92020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffad3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48616/0 (socket says 172.21.15.86:48616) 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffad3400700 1 -- 172.21.15.86:0/3441640254 learned_addr learned my addr 172.21.15.86:0/3441640254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.726+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 0x561bcfd92020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53358/0 (socket says 172.21.15.86:53358) 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad3e00700 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfd94950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad2a00700 1 -- 172.21.15.86:0/3441640254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 msgr2=0x561bcfd94950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfd94950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad2a00700 1 -- 172.21.15.86:0/3441640254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 msgr2=0x561bcfd871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd871b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad2a00700 1 -- 172.21.15.86:0/3441640254 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bcfc76810 con 0x561bcfd876f0 2024-04-25T19:09:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad3e00700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffab80a3e50 0x7ffab80a6310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009fd0 tx=0x7ffacc009d00 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 0x561bcfd92020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffac00cde40 tx=0x7ffac000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5107ced7eb1bac27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffac8c00700 1 -- 172.21.15.86:0/3441640254 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac001c070 con 0x561bcfd876f0 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffac8c00700 1 -- 172.21.15.86:0/3441640254 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac00b3030 con 0x561bcfd876f0 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.727+0000 7ffac8c00700 1 -- 172.21.15.86:0/3441640254 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0098d00 con 0x561bcfd876f0 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3441640254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 msgr2=0x561bcfd92020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 0x561bcfd92020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffac00cde40 tx=0x7ffac000c7e0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3441640254 shutdown_connections 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfd94950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3441640254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd876f0 0x561bcfd92020 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3441640254 >> 172.21.15.86:0/3441640254 conn(0x561bcfcb60e0 msgr2=0x561bcfd832e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3441640254 shutdown_connections 2024-04-25T19:09:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.728+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3441640254 wait complete. 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.729+0000 7ffae4faeb80 1 -- start start 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.729+0000 7ffae4faeb80 1 Processor -- start 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd91d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfd8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfd8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bcfd95050 con 0x561bcfd86dd0 2024-04-25T19:09:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bcfc939f0 con 0x561bcfd876f0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561bcfd951d0 con 0x561bcfd92560 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffae4faeb80 10 client.0 _ll_register_callbacks cb 0x561bcfc728b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfd8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfd8f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58276/0 (socket says 172.21.15.86:58276) 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffad2a00700 1 -- 172.21.15.86:0/3680419200 learned_addr learned my addr 172.21.15.86:0/3680419200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffad3e00700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfd8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffad3400700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd91d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.730+0000 7ffad2a00700 1 -- 172.21.15.86:0/3680419200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 msgr2=0x561bcfd8f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfd8f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffad2a00700 1 -- 172.21.15.86:0/3680419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 msgr2=0x561bcfd91d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd91d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffad2a00700 1 -- 172.21.15.86:0/3680419200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcfc27a60 con 0x561bcfd876f0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfd8f030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffac000f010 tx=0x7ffac009c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7194a67de658311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffabd600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffabd600700 1 -- 172.21.15.86:0/3680419200 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0066060 con 0x561bcfd876f0 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffabd600700 1 -- 172.21.15.86:0/3680419200 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ffac001edb0 con 0x561bcfd876f0 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac00b3030 con 0x561bcfd876f0 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffabd600700 1 -- 172.21.15.86:0/3680419200 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffac001c070 con 0x561bcfd876f0 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffae4faeb80 10 client.26009 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.731+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bcfc70c80 con 0x561bcfd876f0 2024-04-25T19:09:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffabd600700 1 -- 172.21.15.86:0/3680419200 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac00b3030 con 0x561bcfd876f0 2024-04-25T19:09:05.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffabd600700 1 -- 172.21.15.86:0/3680419200 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffac0085c60 con 0x561bcfd876f0 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffae4faeb80 10 client.26009 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffae4faeb80 10 client.26009 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffae4faeb80 20 client.26009 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfda1c50 0x561bcfda4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffae4faeb80 4 client.26009 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bcfce7810 con 0x561bcfda1c50 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.732+0000 7ffad3400700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfda1c50 0x561bcfda4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.733+0000 7ffad3400700 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfda1c50 0x561bcfda4040 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b703b9524eeb9244 server_cookie=78a633df6c98211b in_seq=0 out_seq=0 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.733+0000 7ffabd600700 10 client.26009 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffabd600700 1 -- 172.21.15.86:0/3680419200 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bcfce7810 con 0x561bcfda1c50 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffabd600700 10 client.26009 handle_command_reply: tid=0 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 client.26009 shutdown 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 msgr2=0x561bcfd8f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfd8f030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffac000f010 tx=0x7ffac009c710 comp rx=0 tx=0).stop 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 shutdown_connections 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfda1c50 0x561bcfda4040 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfd8f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfd91d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3680419200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfd8f030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 >> 172.21.15.86:0/3680419200 conn(0x561bcfcb60e0 msgr2=0x561bcfd82ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffabd600700 0 client.26009 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 shutdown_connections 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.734+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3680419200 wait complete. 2024-04-25T19:09:05.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.735+0000 7ffae4faeb80 20 client.26009 trim_cache size 0 max 16384 2024-04-25T19:09:05.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 Processor -- start 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 -- start start 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 0x561bcfd834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd876f0 0x561bcfd91240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfda6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b3980 con 0x561bcfd876f0 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b54c0 con 0x561bcfd92560 2024-04-25T19:09:05.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b4050 con 0x561bcfd86dd0 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffad3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 0x561bcfd834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd876f0 0x561bcfd91240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffad3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 0x561bcfd834a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53374/0 (socket says 172.21.15.86:53374) 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.756+0000 7ffad3400700 1 -- 172.21.15.86:0/3956236760 learned_addr learned my addr 172.21.15.86:0/3956236760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3e00700 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfda6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3400700 1 -- 172.21.15.86:0/3956236760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 msgr2=0x561bcfda6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3400700 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfda6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3400700 1 -- 172.21.15.86:0/3956236760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd876f0 msgr2=0x561bcfd91240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3400700 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd876f0 0x561bcfd91240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3400700 1 -- 172.21.15.86:0/3956236760 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bcfd87d70 con 0x561bcfd86dd0 2024-04-25T19:09:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad2a00700 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd876f0 0x561bcfd91240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffad3400700 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 0x561bcfd834a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffac4005ce0 tx=0x7ffac4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd612156125adba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.757+0000 7ffabfe00700 1 -- 172.21.15.86:0/3956236760 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4011d90 con 0x561bcfd86dd0 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffabfe00700 1 -- 172.21.15.86:0/3956236760 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac4024030 con 0x561bcfd86dd0 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffabfe00700 1 -- 172.21.15.86:0/3956236760 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac40200d0 con 0x561bcfd86dd0 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3956236760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 msgr2=0x561bcfd834a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 0x561bcfd834a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffac4005ce0 tx=0x7ffac4009260 comp rx=0 tx=0).stop 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3956236760 shutdown_connections 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd92560 0x561bcfda6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd86dd0 0x561bcfd834a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/3956236760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd876f0 0x561bcfd91240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3956236760 >> 172.21.15.86:0/3956236760 conn(0x561bcfcb60e0 msgr2=0x561bcfd9dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.758+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3956236760 shutdown_connections 2024-04-25T19:09:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.759+0000 7ffae4faeb80 1 -- 172.21.15.86:0/3956236760 wait complete. 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.759+0000 7ffae4faeb80 1 -- start start 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.759+0000 7ffae4faeb80 1 Processor -- start 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfda5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfda6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfe07e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b4df0 con 0x561bcfd86dd0 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b4720 con 0x561bcfd876f0 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffad41b1920 con 0x561bcfd92560 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffae4faeb80 10 client.0 _ll_register_callbacks cb 0x561bcfc728b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfda5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfda6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfda6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58298/0 (socket says 172.21.15.86:58298) 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad2a00700 1 -- 172.21.15.86:0/1497253634 learned_addr learned my addr 172.21.15.86:0/1497253634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad3e00700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfe07e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad3400700 1 -- 172.21.15.86:0/1497253634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 msgr2=0x561bcfe07e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad3400700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfe07e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.760+0000 7ffad3400700 1 -- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 msgr2=0x561bcfda6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffad3400700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfda6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffad3400700 1 -- 172.21.15.86:0/1497253634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcfe08380 con 0x561bcfd86dd0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffad2a00700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfda6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffad3400700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfda5dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffac4005310 tx=0x7ffac4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a25d23d69abd70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffabd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffabd600700 1 -- 172.21.15.86:0/1497253634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac402c070 con 0x561bcfd86dd0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffabd600700 1 -- 172.21.15.86:0/1497253634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8593+0+0 (secure 0 0 0) 0x7ffac4005410 con 0x561bcfd86dd0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.761+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac4024030 con 0x561bcfd86dd0 2024-04-25T19:09:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffabd600700 1 -- 172.21.15.86:0/1497253634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffac4038050 con 0x561bcfd86dd0 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 10 client.15918 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bcfe09630 con 0x561bcfd86dd0 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffabd600700 1 -- 172.21.15.86:0/1497253634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac4024030 con 0x561bcfd86dd0 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffabd600700 1 -- 172.21.15.86:0/1497253634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffac4020310 con 0x561bcfd86dd0 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 10 client.15918 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 20 client.15918 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfe09c70 0x561bcfe0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 4 client.15918 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.6, "await_for": 42.0}] 2024-04-25T19:09:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.762+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.6, "await_for": 42.0}) -- 0x561bcfce18b0 con 0x561bcfe09c70 2024-04-25T19:09:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.763+0000 7ffad2a00700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfe09c70 0x561bcfe0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.763+0000 7ffad2a00700 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfe09c70 0x561bcfe0c090 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92d29de237dc0f4 server_cookie=21759a0559cfba10 in_seq=0 out_seq=0 2024-04-25T19:09:05.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.763+0000 7ffabd600700 10 client.15918 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:05.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.776+0000 7ffabd600700 1 -- 172.21.15.86:0/1497253634 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561bcfce18b0 con 0x561bcfe09c70 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.776+0000 7ffabd600700 10 client.15918 handle_command_reply: tid=0 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.776+0000 7ffae4faeb80 1 client.15918 shutdown 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 msgr2=0x561bcfda5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfda5dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffac4005310 tx=0x7ffac4009f00 comp rx=0 tx=0).stop 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 shutdown_connections 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561bcfe09c70 0x561bcfe0c090 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcfd92560 0x561bcfe07e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bcfd86dd0 0x561bcfda5dd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 --2- 172.21.15.86:0/1497253634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bcfd876f0 0x561bcfda6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 >> 172.21.15.86:0/1497253634 conn(0x561bcfcb60e0 msgr2=0x561bcfd82490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffabd600700 0 client.15918 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:05.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 shutdown_connections 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 1 -- 172.21.15.86:0/1497253634 wait complete. 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.777+0000 7ffae4faeb80 20 client.15918 trim_cache size 0 max 16384 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 18, 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout: "780813fa": { 2024-04-25T19:09:05.781 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 18, 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 78.6, 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:09:05.782 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffab80a3e50 msgr2=0x7ffab80a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffab80a3e50 0x7ffab80a6310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009fd0 tx=0x7ffacc009d00 comp rx=0 tx=0).stop 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad4109c80 msgr2=0x7ffad419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad4109c80 0x7ffad419fb50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffac000e9c0 tx=0x7ffac000ecd0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 shutdown_connections 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffab80a3e50 0x7ffab80a6310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad410e3b0 0x7ffad41a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffad41092d0 0x7ffad419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 --2- 172.21.15.86:0/2989927759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffad4109c80 0x7ffad419fb50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 >> 172.21.15.86:0/2989927759 conn(0x7ffad4076210 msgr2=0x7ffad4076650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 shutdown_connections 2024-04-25T19:09:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:05.778+0000 7ffada600700 1 -- 172.21.15.86:0/2989927759 wait complete. 2024-04-25T19:09:06.469 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:06 smithi183 ceph-mon[32344]: pgmap v187: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-04-25T19:09:06.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:06 smithi194 ceph-mon[32740]: pgmap v187: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-04-25T19:09:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:06 smithi086 ceph-mon[27783]: pgmap v187: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-04-25T19:09:06.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '780813fa', quiesce took 7.6 seconds. Will release after: 18.6 2024-04-25T19:09:07.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:07 smithi194 ceph-mon[32740]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T19:09:07.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:07 smithi086 ceph-mon[27783]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T19:09:07.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:07 smithi183 ceph-mon[32344]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T19:09:08.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:08 smithi194 ceph-mon[32740]: pgmap v189: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 35 op/s 2024-04-25T19:09:08.573 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:08 smithi086 ceph-mon[27783]: pgmap v189: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 35 op/s 2024-04-25T19:09:08.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:08 smithi183 ceph-mon[32344]: pgmap v189: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 35 op/s 2024-04-25T19:09:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 -- 172.21.15.86:0/4088959670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4112c30 msgr2=0x7fd6b4115020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 --2- 172.21.15.86:0/4088959670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4112c30 0x7fd6b4115020 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009ca0 tx=0x7fd6ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 -- 172.21.15.86:0/4088959670 shutdown_connections 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 --2- 172.21.15.86:0/4088959670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4112c30 0x7fd6b4115020 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 --2- 172.21.15.86:0/4088959670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b4076da0 0x7fd6b41126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 --2- 172.21.15.86:0/4088959670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b4076480 0x7fd6b4076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.547+0000 7fd6b9600640 1 -- 172.21.15.86:0/4088959670 >> 172.21.15.86:0/4088959670 conn(0x7fd6b40ffa60 msgr2=0x7fd6b4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.548+0000 7fd6b9600640 1 -- 172.21.15.86:0/4088959670 shutdown_connections 2024-04-25T19:09:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.548+0000 7fd6b9600640 1 -- 172.21.15.86:0/4088959670 wait complete. 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.548+0000 7fd6b9600640 1 Processor -- start 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 -- start start 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b4076480 0x7fd6b41a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 0x7fd6b41a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b4112c30 0x7fd6b419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4118670 con 0x7fd6b4076da0 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41184f0 con 0x7fd6b4076480 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b9600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41187f0 con 0x7fd6b4112c30 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 0x7fd6b41a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 0x7fd6b41a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48668/0 (socket says 172.21.15.86:48668) 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b3400640 1 -- 172.21.15.86:0/3395575809 learned_addr learned my addr 172.21.15.86:0/3395575809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b4112c30 0x7fd6b419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.549+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b4076480 0x7fd6b41a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b3400640 1 -- 172.21.15.86:0/3395575809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b4112c30 msgr2=0x7fd6b419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b3400640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b4112c30 0x7fd6b419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b3400640 1 -- 172.21.15.86:0/3395575809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b4076480 msgr2=0x7fd6b41a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b3400640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b4076480 0x7fd6b41a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b3400640 1 -- 172.21.15.86:0/3395575809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b419f130 con 0x7fd6b4076da0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b3400640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 0x7fd6b41a4cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000e9f0 tx=0x7fd6a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b0c00640 1 -- 172.21.15.86:0/3395575809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000cd80 con 0x7fd6b4076da0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b0c00640 1 -- 172.21.15.86:0/3395575809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a00054d0 con 0x7fd6b4076da0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b419f420 con 0x7fd6b4076da0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b0c00640 1 -- 172.21.15.86:0/3395575809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0010690 con 0x7fd6b4076da0 2024-04-25T19:09:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.550+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b41ab670 con 0x7fd6b4076da0 2024-04-25T19:09:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 Processor -- start 2024-04-25T19:09:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 -- start start 2024-04-25T19:09:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7b9b0 0x563595b7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6b0c00640 1 -- 172.21.15.86:0/3395575809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6a0004e90 con 0x7fd6b4076da0 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b86b90 0x563595b88f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563595ae1900 con 0x563595b7c2d0 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563595a87ef0 con 0x563595b7b9b0 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.554+0000 7fd6bda57740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563595add8c0 con 0x563595b86b90 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b86650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48680/0 (socket says 172.21.15.86:48680) 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 -- 172.21.15.86:0/105893747 learned_addr learned my addr 172.21.15.86:0/105893747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3400640 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b86b90 0x563595b88f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7b9b0 0x563595b7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 -- 172.21.15.86:0/105893747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b86b90 msgr2=0x563595b88f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b86b90 0x563595b88f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 -- 172.21.15.86:0/105893747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7b9b0 msgr2=0x563595b7bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7b9b0 0x563595b7bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 -- 172.21.15.86:0/105893747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635957f4210 con 0x563595b7c2d0 2024-04-25T19:09:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3400640 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b86b90 0x563595b88f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:09:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.555+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b86650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009c90 tx=0x7fd6a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e699296d0b462b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6a9200640 1 -- 172.21.15.86:0/105893747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004230 con 0x563595b7c2d0 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6a9200640 1 -- 172.21.15.86:0/105893747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a40043d0 con 0x563595b7c2d0 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6a9200640 1 -- 172.21.15.86:0/105893747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4019620 con 0x563595b7c2d0 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6b0c00640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6980a3d10 0x7fd6980a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6b0c00640 1 -- 172.21.15.86:0/3395575809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8681+0+0 (secure 0 0 0) 0x7fd6a00d0c80 con 0x7fd6b4076da0 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6bda57740 1 -- 172.21.15.86:0/105893747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 msgr2=0x563595b86650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6bda57740 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b86650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009c90 tx=0x7fd6a4011810 comp rx=0 tx=0).stop 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.556+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6980a3d10 0x7fd6980a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 -- 172.21.15.86:0/105893747 shutdown_connections 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b86b90 0x563595b88f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b86650 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 --2- 172.21.15.86:0/105893747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7b9b0 0x563595b7bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 -- 172.21.15.86:0/105893747 >> 172.21.15.86:0/105893747 conn(0x563595b77ec0 msgr2=0x563595b782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 -- 172.21.15.86:0/105893747 shutdown_connections 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 -- 172.21.15.86:0/105893747 wait complete. 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6980a3d10 0x7fd6980a61d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac004af0 tx=0x7fd6ac005d80 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.557+0000 7fd6bda57740 1 -- start start 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 Processor -- start 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b83a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563595b897c0 con 0x563595b86b90 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563595a9b310 con 0x563595b7c2d0 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563595b89940 con 0x563595b7b9b0 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6bda57740 10 client.0 _ll_register_callbacks cb 0x563595a69cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b844d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48684/0 (socket says 172.21.15.86:48684) 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6b8c00640 1 -- 172.21.15.86:0/139090406 learned_addr learned my addr 172.21.15.86:0/139090406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.558+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b83a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b3400640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b8c00640 1 -- 172.21.15.86:0/139090406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 msgr2=0x563595b83a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b83a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b8c00640 1 -- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 msgr2=0x563595b83f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b83f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b8c00640 1 -- 172.21.15.86:0/139090406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635958c0620 con 0x563595b86b90 2024-04-25T19:09:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b844d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac02f470 tx=0x7fd6ac02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa1395672f295f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd69d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd69d600640 1 -- 172.21.15.86:0/139090406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac02f610 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd69d600640 1 -- 172.21.15.86:0/139090406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8681+0+0 (secure 0 0 0) 0x7fd6ac02f7b0 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.559+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a40043d0 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd69d600640 1 -- 172.21.15.86:0/139090406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6ac02f9b0 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563595b95950 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd69d600640 1 -- 172.21.15.86:0/139090406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a40043d0 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd69d600640 1 -- 172.21.15.86:0/139090406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd6ac051020 con 0x563595b86b90 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 20 client.15930 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595b96e00 0x563595b99250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 4 client.15930 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:09:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.560+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563595af58b0 con 0x563595b96e00 2024-04-25T19:09:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.561+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595b96e00 0x563595b99250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.561+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595b96e00 0x563595b99250 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a757fc84685b99 server_cookie=13bdcdead53d1cd5 in_seq=0 out_seq=0 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.561+0000 7fd69d600640 10 client.15930 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd69d600640 1 -- 172.21.15.86:0/139090406 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563595af58b0 con 0x563595b96e00 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd69d600640 10 client.15930 handle_command_reply: tid=0 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 client.15930 shutdown 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 msgr2=0x563595b844d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b844d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac02f470 tx=0x7fd6ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 shutdown_connections 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595b96e00 0x563595b99250 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b844d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd69d600640 0 client.15930 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b83f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 --2- 172.21.15.86:0/139090406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b83a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 >> 172.21.15.86:0/139090406 conn(0x563595b77ec0 msgr2=0x563595b7dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.562+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 shutdown_connections 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.563+0000 7fd6bda57740 1 -- 172.21.15.86:0/139090406 wait complete. 2024-04-25T19:09:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.563+0000 7fd6bda57740 20 client.15930 trim_cache size 0 max 16384 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 Processor -- start 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 -- start start 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b92680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b7e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b86b90 0x563595b84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41b42e0 con 0x563595b7c2d0 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41b5e20 con 0x563595b86b90 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6bda57740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41b49b0 con 0x563595b7b9b0 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b7e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b7e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48696/0 (socket says 172.21.15.86:48696) 2024-04-25T19:09:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.579+0000 7fd6b3400640 1 -- 172.21.15.86:0/990834951 learned_addr learned my addr 172.21.15.86:0/990834951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b86b90 0x563595b84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3400640 1 -- 172.21.15.86:0/990834951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 msgr2=0x563595b92680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3400640 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b92680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3400640 1 -- 172.21.15.86:0/990834951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b86b90 msgr2=0x563595b84120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3400640 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b86b90 0x563595b84120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3400640 1 -- 172.21.15.86:0/990834951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563595b7e8b0 con 0x563595b7c2d0 2024-04-25T19:09:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b86b90 0x563595b84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd6b3400640 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b7e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00994f0 tx=0x7fd6a0004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2519a009d64dc899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd69fe00640 1 -- 172.21.15.86:0/990834951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a009c480 con 0x563595b7c2d0 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd69fe00640 1 -- 172.21.15.86:0/990834951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a009de50 con 0x563595b7c2d0 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.580+0000 7fd69fe00640 1 -- 172.21.15.86:0/990834951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0005180 con 0x563595b7c2d0 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 -- 172.21.15.86:0/990834951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 msgr2=0x563595b7e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b7e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00994f0 tx=0x7fd6a0004a90 comp rx=0 tx=0).stop 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 -- 172.21.15.86:0/990834951 shutdown_connections 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b86b90 0x563595b84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b7c2d0 0x563595b7e370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 --2- 172.21.15.86:0/990834951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b92680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 -- 172.21.15.86:0/990834951 >> 172.21.15.86:0/990834951 conn(0x563595b77ec0 msgr2=0x563595b83a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 -- 172.21.15.86:0/990834951 shutdown_connections 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.581+0000 7fd6bda57740 1 -- 172.21.15.86:0/990834951 wait complete. 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 -- start start 2024-04-25T19:09:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 Processor -- start 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b8ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b8c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41b5750 con 0x563595b86b90 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41b5080 con 0x563595b7c2d0 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41b22e0 con 0x563595b7b9b0 2024-04-25T19:09:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.582+0000 7fd6bda57740 10 client.0 _ll_register_callbacks cb 0x7fd6ac031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b8c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b8ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b8ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53412/0 (socket says 172.21.15.86:53412) 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 -- 172.21.15.86:0/2783192331 learned_addr learned my addr 172.21.15.86:0/2783192331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3400640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 -- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 msgr2=0x563595b8c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b8c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 -- 172.21.15.86:0/2783192331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 msgr2=0x563595b8c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b8c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 -- 172.21.15.86:0/2783192331 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563595b8cb70 con 0x563595b7b9b0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b8c00640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b8c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd6b3e00640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b8ece0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40097e0 tx=0x7fd6a4003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f4c79a6ee78573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd69d600640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:09:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.583+0000 7fd69d600640 1 -- 172.21.15.86:0/2783192331 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a401f040 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a009de50 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd69d600640 1 -- 172.21.15.86:0/2783192331 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8681+0+0 (secure 0 0 0) 0x7fd6a40037f0 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd69d600640 1 -- 172.21.15.86:0/2783192331 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6a403a050 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 10 client.26026 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563595bfd170 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd69d600640 1 -- 172.21.15.86:0/2783192331 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a009de50 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd69d600640 1 -- 172.21.15.86:0/2783192331 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd6a4018070 con 0x563595b7b9b0 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 10 client.26026 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 10 client.26026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 20 client.26026 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595bfd720 0x563595bffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 4 client.26026 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.584+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563595aef320 con 0x563595bfd720 2024-04-25T19:09:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.585+0000 7fd6b3400640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595bfd720 0x563595bffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.585+0000 7fd6b3400640 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595bfd720 0x563595bffb40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc9aba3cf23fd65 server_cookie=b72dea6cf23e5398 in_seq=0 out_seq=0 2024-04-25T19:09:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.585+0000 7fd69d600640 10 client.26026 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd69d600640 1 -- 172.21.15.86:0/2783192331 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563595aef320 con 0x563595bfd720 2024-04-25T19:09:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd69d600640 10 client.26026 handle_command_reply: tid=0 2024-04-25T19:09:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 client.26026 shutdown 2024-04-25T19:09:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 msgr2=0x563595b8ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b8ece0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40097e0 tx=0x7fd6a4003e10 comp rx=0 tx=0).stop 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 shutdown_connections 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563595bfd720 0x563595bffb40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563595b86b90 0x563595b8c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563595b7c2d0 0x563595b8c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd69d600640 0 client.26026 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 --2- 172.21.15.86:0/2783192331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563595b7b9b0 0x563595b8ece0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.597+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 >> 172.21.15.86:0/2783192331 conn(0x563595b77ec0 msgr2=0x563595b842a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.598+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 shutdown_connections 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.598+0000 7fd6bda57740 1 -- 172.21.15.86:0/2783192331 wait complete. 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.598+0000 7fd6bda57740 20 client.26026 trim_cache size 0 max 16384 2024-04-25T19:09:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:09:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:09:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:09:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:09:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6980a3d10 msgr2=0x7fd6980a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6980a3d10 0x7fd6980a61d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac004af0 tx=0x7fd6ac005d80 comp rx=0 tx=0).stop 2024-04-25T19:09:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 msgr2=0x7fd6b41a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 0x7fd6b41a4cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000e9f0 tx=0x7fd6a000edf0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 shutdown_connections 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6980a3d10 0x7fd6980a61d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b4112c30 0x7fd6b419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6b4076da0 0x7fd6b41a4cf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 --2- 172.21.15.86:0/3395575809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b4076480 0x7fd6b41a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 >> 172.21.15.86:0/3395575809 conn(0x7fd6b40ffa60 msgr2=0x7fd6b4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 shutdown_connections 2024-04-25T19:09:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:09.599+0000 7fd6b9600640 1 -- 172.21.15.86:0/3395575809 wait complete. 2024-04-25T19:09:10.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:10 smithi194 ceph-mon[32740]: pgmap v190: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-04-25T19:09:10.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:10 smithi086 ceph-mon[27783]: pgmap v190: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-04-25T19:09:10.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:10 smithi183 ceph-mon[32344]: pgmap v190: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-04-25T19:09:12.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:12 smithi194 ceph-mon[32740]: pgmap v191: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-04-25T19:09:12.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:12 smithi086 ceph-mon[27783]: pgmap v191: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-04-25T19:09:12.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:12 smithi183 ceph-mon[32344]: pgmap v191: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 87 KiB/s wr, 35 op/s 2024-04-25T19:09:13.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:09:13.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:09:13.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:09:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:14 smithi183 ceph-mon[32344]: pgmap v192: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T19:09:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:14 smithi183 ceph-mon[32344]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T19:09:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:14 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:09:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:14 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:09:14.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:14 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:09:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:14 smithi194 ceph-mon[32740]: pgmap v192: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T19:09:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:14 smithi194 ceph-mon[32740]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T19:09:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:14 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:09:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:14 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:09:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:14 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:09:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:14 smithi086 ceph-mon[27783]: pgmap v192: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T19:09:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:14 smithi086 ceph-mon[27783]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T19:09:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:14 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:09:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:14 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:09:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:14 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:09:16.660 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:16 smithi086 ceph-mon[27783]: pgmap v194: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:09:16.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:16 smithi183 ceph-mon[32344]: pgmap v194: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:09:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:16 smithi194 ceph-mon[32740]: pgmap v194: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:09:18.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:18 smithi086 ceph-mon[27783]: pgmap v195: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T19:09:18.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:18 smithi183 ceph-mon[32344]: pgmap v195: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T19:09:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:18 smithi194 ceph-mon[32740]: pgmap v195: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T19:09:20.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:20 smithi183 ceph-mon[32344]: pgmap v196: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T19:09:20.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:20 smithi194 ceph-mon[32740]: pgmap v196: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T19:09:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:20 smithi086 ceph-mon[27783]: pgmap v196: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T19:09:22.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:22 smithi183 ceph-mon[32344]: pgmap v197: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T19:09:22.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:22 smithi194 ceph-mon[32740]: pgmap v197: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T19:09:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 -- 172.21.15.86:0/2622992578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4020108560 msgr2=0x7f40201089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 --2- 172.21.15.86:0/2622992578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4020108560 0x7f40201089e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f400c009930 tx=0x7f400c02a440 comp rx=0 tx=0).stop 2024-04-25T19:09:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 -- 172.21.15.86:0/2622992578 shutdown_connections 2024-04-25T19:09:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 --2- 172.21.15.86:0/2622992578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 0x7f40201153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 --2- 172.21.15.86:0/2622992578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4020108560 0x7f40201089e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 --2- 172.21.15.86:0/2622992578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f402010e620 0x7f402010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 -- 172.21.15.86:0/2622992578 >> 172.21.15.86:0/2622992578 conn(0x7f40200764e0 msgr2=0x7f4020078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 -- 172.21.15.86:0/2622992578 shutdown_connections 2024-04-25T19:09:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.742+0000 7f4025400640 1 -- 172.21.15.86:0/2622992578 wait complete. 2024-04-25T19:09:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.743+0000 7f4025400640 1 Processor -- start 2024-04-25T19:09:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 -- start start 2024-04-25T19:09:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 0x7f40201a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 0x7f40201a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f402010e620 0x7f402019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4020118790 con 0x7f4020108f20 2024-04-25T19:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4020118610 con 0x7f402010e620 2024-04-25T19:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f4025400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4020118910 con 0x7f4020108560 2024-04-25T19:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f401de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 0x7f40201a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f401e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 0x7f40201a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f401f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f402010e620 0x7f402019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.744+0000 7f401e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 0x7f40201a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36312/0 (socket says 172.21.15.86:36312) 2024-04-25T19:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 -- 172.21.15.86:0/2083119809 learned_addr learned my addr 172.21.15.86:0/2083119809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 0x7f40201a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47232/0 (socket says 172.21.15.86:47232) 2024-04-25T19:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 -- 172.21.15.86:0/2083119809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f402010e620 msgr2=0x7f402019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f402010e620 0x7f402019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 -- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 msgr2=0x7f40201a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 0x7f40201a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 -- 172.21.15.86:0/2083119809 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402019f230 con 0x7f4020108560 2024-04-25T19:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.745+0000 7f401e800640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 0x7f40201a4900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f401000e9f0 tx=0x7f401000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.746+0000 7f4017400640 1 -- 172.21.15.86:0/2083119809 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401000ccc0 con 0x7f4020108560 2024-04-25T19:09:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.746+0000 7f4017400640 1 -- 172.21.15.86:0/2083119809 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40100108e0 con 0x7f4020108560 2024-04-25T19:09:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.746+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402019f520 con 0x7f4020108560 2024-04-25T19:09:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.746+0000 7f4017400640 1 -- 172.21.15.86:0/2083119809 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010018880 con 0x7f4020108560 2024-04-25T19:09:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.746+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40201ab7c0 con 0x7f4020108560 2024-04-25T19:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 Processor -- start 2024-04-25T19:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 -- start start 2024-04-25T19:09:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fc8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4017400640 1 -- 172.21.15.86:0/2083119809 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4010010430 con 0x7f4020108560 2024-04-25T19:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8f140 0x55743fc994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc9bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55743fbf4900 con 0x55743fc8f140 2024-04-25T19:09:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55743fb9aef0 con 0x55743fc99a00 2024-04-25T19:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.749+0000 7f4028206740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55743fbf08c0 con 0x55743fc8e820 2024-04-25T19:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fc8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8f140 0x55743fc994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fc8ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36316/0 (socket says 172.21.15.86:36316) 2024-04-25T19:09:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 -- 172.21.15.86:0/658907383 learned_addr learned my addr 172.21.15.86:0/658907383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401de00640 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc9bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 -- 172.21.15.86:0/658907383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 msgr2=0x55743fc9bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc9bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 -- 172.21.15.86:0/658907383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8f140 msgr2=0x55743fc994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8f140 0x55743fc994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 -- 172.21.15.86:0/658907383 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55743f907210 con 0x55743fc8e820 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.750+0000 7f401f200640 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fc8ec00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4018011440 tx=0x7f4018011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99709b947a2c5f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f400b400640 1 -- 172.21.15.86:0/658907383 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018003c30 con 0x55743fc8e820 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f400b400640 1 -- 172.21.15.86:0/658907383 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4018003dd0 con 0x55743fc8e820 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f400b400640 1 -- 172.21.15.86:0/658907383 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401800faa0 con 0x55743fc8e820 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f4017400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f40040a3dc0 0x7f40040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f4017400640 1 -- 172.21.15.86:0/2083119809 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7f40100d0c80 con 0x7f4020108560 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f401e800640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f40040a3dc0 0x7f40040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f4028206740 1 -- 172.21.15.86:0/658907383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 msgr2=0x55743fc8ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.751+0000 7f4028206740 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fc8ec00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4018011440 tx=0x7f4018011900 comp rx=0 tx=0).stop 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 -- 172.21.15.86:0/658907383 shutdown_connections 2024-04-25T19:09:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc9bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8f140 0x55743fc994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 --2- 172.21.15.86:0/658907383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fc8ec00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 -- 172.21.15.86:0/658907383 >> 172.21.15.86:0/658907383 conn(0x55743fc8ad30 msgr2=0x55743fc8b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 -- 172.21.15.86:0/658907383 shutdown_connections 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 -- 172.21.15.86:0/658907383 wait complete. 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f401e800640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f40040a3dc0 0x7f40040a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55743fc8fdc0 tx=0x7f40100d4730 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 -- start start 2024-04-25T19:09:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.752+0000 7f4028206740 1 Processor -- start 2024-04-25T19:09:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 0x55743fc96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc97490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55743fc9c630 con 0x55743fc8e820 2024-04-25T19:09:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55743fbae310 con 0x55743fc99a00 2024-04-25T19:09:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55743fc9c7b0 con 0x55743fc8f140 2024-04-25T19:09:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f4028206740 10 client.0 _ll_register_callbacks cb 0x55743fb7ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f401f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 0x55743fc96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f401f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 0x55743fc96a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47248/0 (socket says 172.21.15.86:47248) 2024-04-25T19:09:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f401f200640 1 -- 172.21.15.86:0/1825203593 learned_addr learned my addr 172.21.15.86:0/1825203593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f401de00640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.753+0000 7f401f200640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc97490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f401f200640 1 -- 172.21.15.86:0/1825203593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 msgr2=0x55743fc96f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f401f200640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc96f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f401f200640 1 -- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 msgr2=0x55743fc97490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f401f200640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc97490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f401f200640 1 -- 172.21.15.86:0/1825203593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743f9d3620 con 0x55743fc8e820 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f401f200640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 0x55743fc96a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f401801b040 tx=0x7f401800f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba1d84eee7bb114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f3fffe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f3fffe00640 1 -- 172.21.15.86:0/1825203593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401802c020 con 0x55743fc8e820 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f3fffe00640 1 -- 172.21.15.86:0/1825203593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7f401800fa20 con 0x55743fc8e820 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4018003dd0 con 0x55743fc8e820 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f3fffe00640 1 -- 172.21.15.86:0/1825203593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f401800fc20 con 0x55743fc8e820 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f4028206740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.754+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55743fca8920 con 0x55743fc8e820 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f3fffe00640 1 -- 172.21.15.86:0/1825203593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4018003dd0 con 0x55743fc8e820 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f3fffe00640 1 -- 172.21.15.86:0/1825203593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4018005040 con 0x55743fc8e820 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f4028206740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f4028206740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f4028206740 20 client.15948 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f4028206740 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fca9d80 0x55743fcac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f4028206740 4 client.15948 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55743fc088b0 con 0x55743fca9d80 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.755+0000 7f401de00640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fca9d80 0x55743fcac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.756+0000 7f401de00640 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fca9d80 0x55743fcac170 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f173be0b05b859fc server_cookie=9d205f1b7c08fd2a in_seq=0 out_seq=0 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.756+0000 7f3fffe00640 10 client.15948 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.757+0000 7f3fffe00640 1 -- 172.21.15.86:0/1825203593 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55743fc088b0 con 0x55743fca9d80 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.757+0000 7f3fffe00640 10 client.15948 handle_command_reply: tid=0 2024-04-25T19:09:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 client.15948 shutdown 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 msgr2=0x55743fc96a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 0x55743fc96a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f401801b040 tx=0x7f401800f680 comp rx=0 tx=0).stop 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 shutdown_connections 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fca9d80 0x55743fcac170 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc99a00 0x55743fc97490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 --2- 172.21.15.86:0/1825203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc8e820 0x55743fc96a10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f3fffe00640 0 client.15948 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 >> 172.21.15.86:0/1825203593 conn(0x55743fc8ad30 msgr2=0x55743fc90ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 shutdown_connections 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 1 -- 172.21.15.86:0/1825203593 wait complete. 2024-04-25T19:09:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.758+0000 7f4028206740 20 client.15948 trim_cache size 0 max 16384 2024-04-25T19:09:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.774+0000 7f4028206740 1 Processor -- start 2024-04-25T19:09:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.774+0000 7f4028206740 1 -- start start 2024-04-25T19:09:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f4028206740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fca54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f4028206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 0x55743fc8a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f4028206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fc908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f4028206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40201b4340 con 0x55743fc99a00 2024-04-25T19:09:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f4028206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40201b5e80 con 0x55743fc8f140 2024-04-25T19:09:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f4028206740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f40201b4a10 con 0x55743fc8e820 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f401f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fc908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f401de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 0x55743fc8a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f401de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 0x55743fc8a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37428/0 (socket says 172.21.15.86:37428) 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f401f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fc908e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47262/0 (socket says 172.21.15.86:47262) 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f401de00640 1 -- 172.21.15.86:0/2967441842 learned_addr learned my addr 172.21.15.86:0/2967441842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.775+0000 7f401f200640 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fca54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401de00640 1 -- 172.21.15.86:0/2967441842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 msgr2=0x55743fca54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401de00640 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fca54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401de00640 1 -- 172.21.15.86:0/2967441842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 msgr2=0x55743fc908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401de00640 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fc908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401de00640 1 -- 172.21.15.86:0/2967441842 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55743fc8ab50 con 0x55743fc8f140 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401f200640 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fca54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f401de00640 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 0x55743fc8a610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f400c0096c0 tx=0x7f400c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a987c991fd68b54a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f400aa00640 1 -- 172.21.15.86:0/2967441842 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c04a030 con 0x55743fc8f140 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f400aa00640 1 -- 172.21.15.86:0/2967441842 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400c031950 con 0x55743fc8f140 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.776+0000 7f400aa00640 1 -- 172.21.15.86:0/2967441842 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c035040 con 0x55743fc8f140 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 -- 172.21.15.86:0/2967441842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 msgr2=0x55743fc8a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 0x55743fc8a610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f400c0096c0 tx=0x7f400c005fd0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 -- 172.21.15.86:0/2967441842 shutdown_connections 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fc908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8f140 0x55743fc8a610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 --2- 172.21.15.86:0/2967441842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8e820 0x55743fca54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 -- 172.21.15.86:0/2967441842 >> 172.21.15.86:0/2967441842 conn(0x55743fc8ad30 msgr2=0x55743fc901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 -- 172.21.15.86:0/2967441842 shutdown_connections 2024-04-25T19:09:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.777+0000 7f4028206740 1 -- 172.21.15.86:0/2967441842 wait complete. 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 -- start start 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 Processor -- start 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8e820 0x55743fc9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fd07000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40201b57b0 con 0x55743fc99a00 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40201b50e0 con 0x55743fc8e820 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f40201b2340 con 0x55743fc8f140 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.778+0000 7f4028206740 10 client.0 _ll_register_callbacks cb 0x7f4018009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fd07000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fd07000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47264/0 (socket says 172.21.15.86:47264) 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401f200640 1 -- 172.21.15.86:0/1161699263 learned_addr learned my addr 172.21.15.86:0/1161699263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401f200640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8e820 0x55743fc9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 -- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8e820 msgr2=0x55743fc9ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8e820 0x55743fc9ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 -- 172.21.15.86:0/1161699263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 msgr2=0x55743fd07000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fd07000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 -- 172.21.15.86:0/1161699263 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743fd07640 con 0x55743fc8f140 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401f200640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fd07000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f401de00640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc9f4d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55743fc91a50 tx=0x7f400c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da620163c70e0367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.779+0000 7f3fffe00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f3fffe00640 1 -- 172.21.15.86:0/1161699263 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c035070 con 0x55743fc8f140 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f3fffe00640 1 -- 172.21.15.86:0/1161699263 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7f400c005030 con 0x55743fc8f140 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55743fc02320 con 0x55743fc8f140 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f3fffe00640 1 -- 172.21.15.86:0/1161699263 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f400c05e050 con 0x55743fc8f140 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f4028206740 10 client.26044 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55743fd0fc00 con 0x55743fc8f140 2024-04-25T19:09:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.780+0000 7f3fffe00640 1 -- 172.21.15.86:0/1161699263 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55743fc02320 con 0x55743fc8f140 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f3fffe00640 1 -- 172.21.15.86:0/1161699263 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f400c004ba0 con 0x55743fc8f140 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f4028206740 10 client.26044 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f4028206740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f4028206740 20 client.26044 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f4028206740 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fd102f0 0x55743fd12710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f4028206740 4 client.26044 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55743fbfc190 con 0x55743fd102f0 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.781+0000 7f401f200640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fd102f0 0x55743fd12710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.782+0000 7f401f200640 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fd102f0 0x55743fd12710 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3006cacd5f4c2 server_cookie=63c35149c911d79 in_seq=0 out_seq=0 2024-04-25T19:09:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.782+0000 7f3fffe00640 10 client.26044 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f3fffe00640 1 -- 172.21.15.86:0/1161699263 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55743fbfc190 con 0x55743fd102f0 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f3fffe00640 10 client.26044 handle_command_reply: tid=0 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 client.26044 shutdown 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 msgr2=0x55743fc9f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc9f4d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55743fc91a50 tx=0x7f400c004040 comp rx=0 tx=0).stop 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 shutdown_connections 2024-04-25T19:09:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55743fd102f0 0x55743fd12710 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743fc99a00 0x55743fd07000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55743fc8f140 0x55743fc9f4d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 --2- 172.21.15.86:0/1161699263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55743fc8e820 0x55743fc9ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f3fffe00640 0 client.26044 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 >> 172.21.15.86:0/1161699263 conn(0x55743fc8ad30 msgr2=0x55743fc97260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 shutdown_connections 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 1 -- 172.21.15.86:0/1161699263 wait complete. 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.791+0000 7f4028206740 20 client.26044 trim_cache size 0 max 16384 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:09:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f40040a3dc0 msgr2=0x7f40040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f40040a3dc0 0x7f40040a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55743fc8fdc0 tx=0x7f40100d4730 comp rx=0 tx=0).stop 2024-04-25T19:09:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 msgr2=0x7f40201a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 0x7f40201a4900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f401000e9f0 tx=0x7f401000edf0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 shutdown_connections 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f40040a3dc0 0x7f40040a6280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f402010e620 0x7f402019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4020108f20 0x7f40201a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 --2- 172.21.15.86:0/2083119809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4020108560 0x7f40201a4900 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.792+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 >> 172.21.15.86:0/2083119809 conn(0x7f40200764e0 msgr2=0x7f4020077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.793+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 shutdown_connections 2024-04-25T19:09:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:22.793+0000 7f4025400640 1 -- 172.21.15.86:0/2083119809 wait complete. 2024-04-25T19:09:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:22 smithi086 ceph-mon[27783]: pgmap v197: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-04-25T19:09:24.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:24 smithi183 ceph-mon[32344]: pgmap v198: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-04-25T19:09:24.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:24 smithi194 ceph-mon[32740]: pgmap v198: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-04-25T19:09:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:24 smithi086 ceph-mon[27783]: pgmap v198: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-04-25T19:09:25.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:09:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:26 smithi194 ceph-mon[32740]: pgmap v199: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-25T19:09:26.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:26 smithi086 ceph-mon[27783]: pgmap v199: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-25T19:09:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:26 smithi183 ceph-mon[32344]: pgmap v199: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-25T19:09:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 -- 172.21.15.86:0/1000974781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 msgr2=0x7fafb81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 --2- 172.21.15.86:0/1000974781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 0x7fafb81098b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099c0 tx=0x7fafa0009cd0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 -- 172.21.15.86:0/1000974781 shutdown_connections 2024-04-25T19:09:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 --2- 172.21.15.86:0/1000974781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 0x7fafb8111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 --2- 172.21.15.86:0/1000974781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 --2- 172.21.15.86:0/1000974781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 0x7fafb81098b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 -- 172.21.15.86:0/1000974781 >> 172.21.15.86:0/1000974781 conn(0x7fafb8100740 msgr2=0x7fafb8102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 -- 172.21.15.86:0/1000974781 shutdown_connections 2024-04-25T19:09:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.145+0000 7fafbca00700 1 -- 172.21.15.86:0/1000974781 wait complete. 2024-04-25T19:09:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 Processor -- start 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 -- start start 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 0x7fafb819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 0x7fafb819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114e50 con 0x7fafb8105860 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114cd0 con 0x7fafb8109f90 2024-04-25T19:09:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafbca00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114fd0 con 0x7fafb8104eb0 2024-04-25T19:09:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafad400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 0x7fafb819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.147+0000 7fafb5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36362/0 (socket says 172.21.15.86:36362) 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 -- 172.21.15.86:0/3427421856 learned_addr learned my addr 172.21.15.86:0/3427421856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb6800700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 0x7fafb819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 -- 172.21.15.86:0/3427421856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 msgr2=0x7fafb819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 0x7fafb819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 -- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 msgr2=0x7fafb819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 0x7fafb819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 -- 172.21.15.86:0/3427421856 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a03d0 con 0x7fafb8104eb0 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb6800700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 0x7fafb819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.148+0000 7fafb5e00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb819b3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fafa800ef10 tx=0x7fafa800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.149+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800c9c0 con 0x7fafb8104eb0 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.149+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a0660 con 0x7fafb8104eb0 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.149+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa8019070 con 0x7fafb8104eb0 2024-04-25T19:09:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.149+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8014480 con 0x7fafb8104eb0 2024-04-25T19:09:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.149+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a7f40 con 0x7fafb8104eb0 2024-04-25T19:09:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.150+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb8069730 con 0x7fafb8104eb0 2024-04-25T19:09:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.152+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fafa8012050 con 0x7fafb8104eb0 2024-04-25T19:09:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.153+0000 7fafaf400700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7faf9c0a3db0 0x7faf9c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.154+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fafa80d0a90 con 0x7fafb8104eb0 2024-04-25T19:09:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.154+0000 7fafad400700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7faf9c0a3db0 0x7faf9c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.154+0000 7fafad400700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7faf9c0a3db0 0x7faf9c0a6270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fafa0000c00 tx=0x7fafa0005fb0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.156+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafa8010040 con 0x7fafb8104eb0 2024-04-25T19:09:28.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.433+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafb802df80 con 0x7fafb8104eb0 2024-04-25T19:09:28.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.434+0000 7fafaf400700 1 -- 172.21.15.86:0/3427421856 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fafa809d060 con 0x7fafb8104eb0 2024-04-25T19:09:28.437 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:09:28.437 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:09:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7faf9c0a3db0 msgr2=0x7faf9c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7faf9c0a3db0 0x7faf9c0a6270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fafa0000c00 tx=0x7fafa0005fb0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 msgr2=0x7fafb819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb819b3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fafa800ef10 tx=0x7fafa800c3a0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 shutdown_connections 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8109f90 0x7fafb819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7faf9c0a3db0 0x7faf9c0a6270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb8104eb0 0x7fafb819b3c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 --2- 172.21.15.86:0/3427421856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafb8105860 0x7fafb819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 >> 172.21.15.86:0/3427421856 conn(0x7fafb8100740 msgr2=0x7fafb8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 shutdown_connections 2024-04-25T19:09:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:28.437+0000 7fafbca00700 1 -- 172.21.15.86:0/3427421856 wait complete. 2024-04-25T19:09:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:09:28.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:28 smithi086 ceph-mon[27783]: pgmap v200: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-04-25T19:09:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:28 smithi194 ceph-mon[32740]: pgmap v200: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-04-25T19:09:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:28 smithi183 ceph-mon[32344]: pgmap v200: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-04-25T19:09:29.268 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 780813fa --release --await' 2024-04-25T19:09:29.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 780813fa --release --await 2024-04-25T19:09:29.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:29 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3427421856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:09:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:29.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:29 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3427421856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:09:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:29 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3427421856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:09:30.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:30 smithi086 ceph-mon[27783]: pgmap v201: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-25T19:09:30.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:30 smithi194 ceph-mon[32740]: pgmap v201: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-25T19:09:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:30 smithi183 ceph-mon[32344]: pgmap v201: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-25T19:09:32.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 -- 172.21.15.86:0/3023670787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c104ee0 msgr2=0x7fce0c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 --2- 172.21.15.86:0/3023670787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c104ee0 0x7fce0c1052c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009920 tx=0x7fcdfc009c30 comp rx=0 tx=0).stop 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 -- 172.21.15.86:0/3023670787 shutdown_connections 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 --2- 172.21.15.86:0/3023670787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c109fc0 0x7fce0c111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 --2- 172.21.15.86:0/3023670787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c104ee0 0x7fce0c1052c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 --2- 172.21.15.86:0/3023670787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 -- 172.21.15.86:0/3023670787 >> 172.21.15.86:0/3023670787 conn(0x7fce0c100770 msgr2=0x7fce0c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 -- 172.21.15.86:0/3023670787 shutdown_connections 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.171+0000 7fce13000700 1 -- 172.21.15.86:0/3023670787 wait complete. 2024-04-25T19:09:32.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.172+0000 7fce13000700 1 Processor -- start 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.172+0000 7fce13000700 1 -- start start 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce13000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 0x7fce0c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce13000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce13000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c109fc0 0x7fce0c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce13000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c114eb0 con 0x7fce0c105890 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce13000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c114d30 con 0x7fce0c104ee0 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce13000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c115030 con 0x7fce0c109fc0 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce0be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 0x7fce0c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60998/0 (socket says 172.21.15.86:60998) 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce0be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 0x7fce0c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43766/0 (socket says 172.21.15.86:43766) 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce0b400700 1 -- 172.21.15.86:0/1431923778 learned_addr learned my addr 172.21.15.86:0/1431923778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.173+0000 7fce11000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c109fc0 0x7fce0c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0be00700 1 -- 172.21.15.86:0/1431923778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c109fc0 msgr2=0x7fce0c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0be00700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c109fc0 0x7fce0c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0be00700 1 -- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 msgr2=0x7fce0c19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0be00700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0be00700 1 -- 172.21.15.86:0/1431923778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c1a0400 con 0x7fce0c104ee0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0b400700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce0be00700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 0x7fce0c19b440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000c00 tx=0x7fcdfc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce08c00700 1 -- 172.21.15.86:0/1431923778 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc027070 con 0x7fce0c104ee0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c1a0690 con 0x7fce0c104ee0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce08c00700 1 -- 172.21.15.86:0/1431923778 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdfc005570 con 0x7fce0c104ee0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.174+0000 7fce08c00700 1 -- 172.21.15.86:0/1431923778 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004930 con 0x7fce0c104ee0 2024-04-25T19:09:32.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.175+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c1a7f70 con 0x7fce0c104ee0 2024-04-25T19:09:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.178+0000 7fce08c00700 1 -- 172.21.15.86:0/1431923778 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcdfc02a070 con 0x7fce0c104ee0 2024-04-25T19:09:32.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.179+0000 7fce1da71b80 1 Processor -- start 2024-04-25T19:09:32.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 -- start start 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883df8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883df8660 0x55f883e02f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883e034d0 0x55f883e058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce08c00700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdf00a3db0 0x7fcdf00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f883d4af90 con 0x55f883df7d40 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f883cf1830 con 0x55f883e034d0 2024-04-25T19:09:32.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce1da71b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f883d46f50 con 0x55f883df8660 2024-04-25T19:09:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.180+0000 7fce11000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883df8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883df8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32774/0 (socket says 172.21.15.86:32774) 2024-04-25T19:09:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 -- 172.21.15.86:0/3863038884 learned_addr learned my addr 172.21.15.86:0/3863038884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce08c00700 1 -- 172.21.15.86:0/1431923778 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fcdfc0d1270 con 0x7fce0c104ee0 2024-04-25T19:09:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce0b400700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdf00a3db0 0x7fcdf00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce0be00700 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883df8660 0x55f883e02f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883e034d0 0x55f883e058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 -- 172.21.15.86:0/3863038884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883df8660 msgr2=0x55f883e02f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883df8660 0x55f883e02f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 -- 172.21.15.86:0/3863038884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883e034d0 msgr2=0x55f883e058c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883e034d0 0x55f883e058c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce11000700 1 -- 172.21.15.86:0/3863038884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f883ce7780 con 0x55f883df7d40 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.181+0000 7fce0be00700 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883df8660 0x55f883e02f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.182+0000 7fce0b400700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdf00a3db0 0x7fcdf00a6270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009750 tx=0x7fcdf8009310 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.182+0000 7fce11000700 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883df8120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fce04009ec0 tx=0x7fce04003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc24dbf07f6aa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.182+0000 7fce01600700 1 -- 172.21.15.86:0/3863038884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04004db0 con 0x55f883df7d40 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.182+0000 7fce01600700 1 -- 172.21.15.86:0/3863038884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce040079b0 con 0x55f883df7d40 2024-04-25T19:09:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.182+0000 7fce01600700 1 -- 172.21.15.86:0/3863038884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0401b800 con 0x55f883df7d40 2024-04-25T19:09:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 -- 172.21.15.86:0/3863038884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 msgr2=0x55f883df8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883df8120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fce04009ec0 tx=0x7fce04003a40 comp rx=0 tx=0).stop 2024-04-25T19:09:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 -- 172.21.15.86:0/3863038884 shutdown_connections 2024-04-25T19:09:32.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883e034d0 0x55f883e058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883df8120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3863038884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883df8660 0x55f883e02f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 -- 172.21.15.86:0/3863038884 >> 172.21.15.86:0/3863038884 conn(0x55f883d27050 msgr2=0x55f883df4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 -- 172.21.15.86:0/3863038884 shutdown_connections 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.183+0000 7fce1da71b80 1 -- 172.21.15.86:0/3863038884 wait complete. 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 -- start start 2024-04-25T19:09:32.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 Processor -- start 2024-04-25T19:09:32.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883e02b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df8660 0x55f883dffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f883e05f60 con 0x55f883df7d40 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f883d04960 con 0x55f883df8660 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f883e060e0 con 0x55f883e034d0 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.184+0000 7fce1da71b80 10 client.0 _ll_register_callbacks cb 0x55f883ce3820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883e02b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883e02b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32790/0 (socket says 172.21.15.86:32790) 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 -- 172.21.15.86:0/39271209 learned_addr learned my addr 172.21.15.86:0/39271209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0b400700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df8660 0x55f883dffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce11000700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 -- 172.21.15.86:0/39271209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 msgr2=0x55f883e003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 -- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df8660 msgr2=0x55f883dffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df8660 0x55f883dffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0be00700 1 -- 172.21.15.86:0/39271209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f883c989d0 con 0x55f883df7d40 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.185+0000 7fce0b400700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df8660 0x55f883dffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fce0be00700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883e02b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0057d0 tx=0x7fcdfc09d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba37c2eedce1724a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fcdf6000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fcdf6000700 1 -- 172.21.15.86:0/39271209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0b5020 con 0x55f883df7d40 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fcdf6000700 1 -- 172.21.15.86:0/39271209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fcdfc07c960 con 0x55f883df7d40 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce040079b0 con 0x55f883df7d40 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fcdf6000700 1 -- 172.21.15.86:0/39271209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc097e60 con 0x55f883df7d40 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fce1da71b80 10 client.15972 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f883ce1bf0 con 0x55f883df7d40 2024-04-25T19:09:32.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.186+0000 7fcdf6000700 1 -- 172.21.15.86:0/39271209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce040079b0 con 0x55f883df7d40 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fcdf6000700 1 -- 172.21.15.86:0/39271209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdfc09d550 con 0x55f883df7d40 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce1da71b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce1da71b80 10 client.15972 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce1da71b80 20 client.15972 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce1da71b80 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e12b60 0x55f883e14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce1da71b80 4 client.15972 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f883d58780 con 0x55f883e12b60 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.187+0000 7fce0b400700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e12b60 0x55f883e14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.188+0000 7fce0b400700 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e12b60 0x55f883e14fe0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57100ece6239118 server_cookie=d36c35157c74c212 in_seq=0 out_seq=0 2024-04-25T19:09:32.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.188+0000 7fcdf6000700 10 client.15972 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fcdf6000700 1 -- 172.21.15.86:0/39271209 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f883d58780 con 0x55f883e12b60 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fcdf6000700 10 client.15972 handle_command_reply: tid=0 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 client.15972 shutdown 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 msgr2=0x55f883e02b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883e02b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0057d0 tx=0x7fcdfc09d9f0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 shutdown_connections 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e12b60 0x55f883e14fe0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df7d40 0x55f883e02b70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 --2- 172.21.15.86:0/39271209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df8660 0x55f883dffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fcdf6000700 0 client.15972 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 >> 172.21.15.86:0/39271209 conn(0x55f883d27050 msgr2=0x55f883df3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 shutdown_connections 2024-04-25T19:09:32.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 1 -- 172.21.15.86:0/39271209 wait complete. 2024-04-25T19:09:32.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.189+0000 7fce1da71b80 20 client.15972 trim_cache size 0 max 16384 2024-04-25T19:09:32.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.210+0000 7fce1da71b80 1 Processor -- start 2024-04-25T19:09:32.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 -- start start 2024-04-25T19:09:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883df43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e0ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e0b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1af640 con 0x55f883df8660 2024-04-25T19:09:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1b1180 con 0x55f883df7d40 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce1da71b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1afd10 con 0x55f883e034d0 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e0ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce11000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e0b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e0ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32800/0 (socket says 172.21.15.86:32800) 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce0b400700 1 -- 172.21.15.86:0/3723437674 learned_addr learned my addr 172.21.15.86:0/3723437674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.211+0000 7fce0be00700 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883df43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce11000700 1 -- 172.21.15.86:0/3723437674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 msgr2=0x55f883df43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce11000700 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883df43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce11000700 1 -- 172.21.15.86:0/3723437674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 msgr2=0x55f883e0ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce11000700 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e0ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce11000700 1 -- 172.21.15.86:0/3723437674 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f883e15660 con 0x55f883e034d0 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce0be00700 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883df43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:32.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce0b400700 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce11000700 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e0b390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce04004330 tx=0x7fce04005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d133489bb9b425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.212+0000 7fce00c00700 1 -- 172.21.15.86:0/3723437674 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0402b070 con 0x55f883e034d0 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce00c00700 1 -- 172.21.15.86:0/3723437674 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce04007bb0 con 0x55f883e034d0 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce00c00700 1 -- 172.21.15.86:0/3723437674 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0401b6e0 con 0x55f883e034d0 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 -- 172.21.15.86:0/3723437674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 msgr2=0x55f883e0b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e0b390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce04004330 tx=0x7fce04005ac0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 -- 172.21.15.86:0/3723437674 shutdown_connections 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e0b390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883df43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 --2- 172.21.15.86:0/3723437674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 -- 172.21.15.86:0/3723437674 >> 172.21.15.86:0/3723437674 conn(0x55f883d27050 msgr2=0x55f883df3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 -- 172.21.15.86:0/3723437674 shutdown_connections 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.213+0000 7fce1da71b80 1 -- 172.21.15.86:0/3723437674 wait complete. 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.214+0000 7fce1da71b80 1 -- start start 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.214+0000 7fce1da71b80 1 Processor -- start 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883e06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e06de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1b0ab0 con 0x55f883df8660 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1b03e0 con 0x55f883df7d40 2024-04-25T19:09:32.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1ad600 con 0x55f883e034d0 2024-04-25T19:09:32.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce1da71b80 10 client.0 _ll_register_callbacks cb 0x55f883ce3820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:32.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce0be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883e06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e06de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e06de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35146/0 (socket says 172.21.15.86:35146) 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 -- 172.21.15.86:0/148346305 learned_addr learned my addr 172.21.15.86:0/148346305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce0be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883e06360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43808/0 (socket says 172.21.15.86:43808) 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 -- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 msgr2=0x55f883e06360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883e06360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 -- 172.21.15.86:0/148346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 msgr2=0x55f883e068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.215+0000 7fce11000700 1 -- 172.21.15.86:0/148346305 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f883e79410 con 0x55f883e034d0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fce11000700 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e06de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce040093f0 tx=0x7fce04007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab9b88a6320f3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fcdf6000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fcdf6000700 1 -- 172.21.15.86:0/148346305 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0402b070 con 0x55f883e034d0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce04007bb0 con 0x55f883e034d0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fcdf6000700 1 -- 172.21.15.86:0/148346305 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fce04007910 con 0x55f883e034d0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fcdf6000700 1 -- 172.21.15.86:0/148346305 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce0401b480 con 0x55f883e034d0 2024-04-25T19:09:32.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fce1da71b80 10 client.26068 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.216+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f883e7a710 con 0x55f883e034d0 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fcdf6000700 1 -- 172.21.15.86:0/148346305 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce04007bb0 con 0x55f883e034d0 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fcdf6000700 1 -- 172.21.15.86:0/148346305 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fce04015050 con 0x55f883e034d0 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fce1da71b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fce1da71b80 10 client.26068 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fce1da71b80 20 client.26068 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fce1da71b80 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e7ab10 0x55f883e7cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fce1da71b80 4 client.26068 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "780813fa", "release": true, "await": true}] 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.217+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "780813fa", "release": true, "await": true}) -- 0x55f883d52820 con 0x55f883e7ab10 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.218+0000 7fce0be00700 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e7ab10 0x55f883e7cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.218+0000 7fcdf6000700 10 client.26068 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:32.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.218+0000 7fce0be00700 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e7ab10 0x55f883e7cf00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8284076d4a35815 server_cookie=db9f03acdadc08c6 in_seq=0 out_seq=0 2024-04-25T19:09:32.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fcdf6000700 1 -- 172.21.15.86:0/148346305 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f883d52820 con 0x55f883e7ab10 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fcdf6000700 10 client.26068 handle_command_reply: tid=0 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 client.26068 shutdown 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 msgr2=0x55f883e06de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e06de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce040093f0 tx=0x7fce04007f90 comp rx=0 tx=0).stop 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 shutdown_connections 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f883e7ab10 0x55f883e7cf00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f883e034d0 0x55f883e06de0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f883df7d40 0x55f883e06360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 --2- 172.21.15.86:0/148346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f883df8660 0x55f883e068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fcdf6000700 0 client.26068 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 >> 172.21.15.86:0/148346305 conn(0x55f883d27050 msgr2=0x55f883e0de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 shutdown_connections 2024-04-25T19:09:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 1 -- 172.21.15.86:0/148346305 wait complete. 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.222+0000 7fce1da71b80 20 client.26068 trim_cache size 0 max 16384 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 20, 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "780813fa": { 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 20, 2024-04-25T19:09:32.225 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 78.6, 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:32.226 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:32.227 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:09:32.227 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:09:32.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdf00a3db0 msgr2=0x7fcdf00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdf00a3db0 0x7fcdf00a6270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009750 tx=0x7fcdf8009310 comp rx=0 tx=0).stop 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 msgr2=0x7fce0c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 0x7fce0c19b440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000c00 tx=0x7fcdfc00fa00 comp rx=0 tx=0).stop 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 shutdown_connections 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcdf00a3db0 0x7fcdf00a6270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce0c109fc0 0x7fce0c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0c104ee0 0x7fce0c19b440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 --2- 172.21.15.86:0/1431923778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce0c105890 0x7fce0c19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 >> 172.21.15.86:0/1431923778 conn(0x7fce0c100770 msgr2=0x7fce0c101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 shutdown_connections 2024-04-25T19:09:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:09:32.223+0000 7fce13000700 1 -- 172.21.15.86:0/1431923778 wait complete. 2024-04-25T19:09:32.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:32 smithi086 ceph-mon[27783]: pgmap v202: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-25T19:09:32.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:32 smithi194 ceph-mon[32740]: pgmap v202: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-25T19:09:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:32 smithi183 ceph-mon[32344]: pgmap v202: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-25T19:09:33.264 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '780813fa', seconds elapsed: 34.1 2024-04-25T19:09:34.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:34 smithi194 ceph-mon[32740]: pgmap v203: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-04-25T19:09:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:34 smithi086 ceph-mon[27783]: pgmap v203: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-04-25T19:09:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:34 smithi183 ceph-mon[32344]: pgmap v203: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-04-25T19:09:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.937+0000 7f6f89c00640 1 -- 172.21.15.86:0/3179316796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f84106170 msgr2=0x7f6f8410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.937+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3179316796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f84106170 0x7f6f8410dd40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009ca0 tx=0x7f6f7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 -- 172.21.15.86:0/3179316796 shutdown_connections 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3179316796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f84106170 0x7f6f8410dd40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3179316796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f84101aa0 0x7f6f84105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3179316796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f841010f0 0x7f6f841014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 -- 172.21.15.86:0/3179316796 >> 172.21.15.86:0/3179316796 conn(0x7f6f84077fd0 msgr2=0x7f6f840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 -- 172.21.15.86:0/3179316796 shutdown_connections 2024-04-25T19:09:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.938+0000 7f6f89c00640 1 -- 172.21.15.86:0/3179316796 wait complete. 2024-04-25T19:09:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 Processor -- start 2024-04-25T19:09:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 -- start start 2024-04-25T19:09:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 0x7f6f84102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f84106170 0x7f6f84103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f841035b0 0x7f6f841a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84111130 con 0x7f6f84106170 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84110fb0 con 0x7f6f841010f0 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.939+0000 7f6f89c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841112b0 con 0x7f6f841035b0 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 0x7f6f84102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 0x7f6f84102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43826/0 (socket says 172.21.15.86:43826) 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 -- 172.21.15.86:0/3152567839 learned_addr learned my addr 172.21.15.86:0/3152567839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82000640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f84106170 0x7f6f84103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f83400640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f841035b0 0x7f6f841a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 -- 172.21.15.86:0/3152567839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f841035b0 msgr2=0x7f6f841a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f841035b0 0x7f6f841a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 -- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f84106170 msgr2=0x7f6f84103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f84106170 0x7f6f84103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 -- 172.21.15.86:0/3152567839 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f841a6f90 con 0x7f6f841010f0 2024-04-25T19:09:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.940+0000 7f6f82a00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 0x7f6f84102b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400e9f0 tx=0x7f6f7400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.941+0000 7f6f7b400640 1 -- 172.21.15.86:0/3152567839 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7400cd80 con 0x7f6f841010f0 2024-04-25T19:09:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.941+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f841a7280 con 0x7f6f841010f0 2024-04-25T19:09:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.941+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f841a77d0 con 0x7f6f841010f0 2024-04-25T19:09:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.941+0000 7f6f7b400640 1 -- 172.21.15.86:0/3152567839 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f740054d0 con 0x7f6f841010f0 2024-04-25T19:09:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.941+0000 7f6f7b400640 1 -- 172.21.15.86:0/3152567839 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74010430 con 0x7f6f841010f0 2024-04-25T19:09:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.944+0000 7f6f8c988740 1 Processor -- start 2024-04-25T19:09:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.944+0000 7f6f8c988740 1 -- start start 2024-04-25T19:09:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.944+0000 7f6f7b400640 1 -- 172.21.15.86:0/3152567839 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f740046c0 con 0x7f6f841010f0 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 0x55dea0275c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0276140 0x55dea02804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0282df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f8c988740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dea01db900 con 0x55dea0275820 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f8c988740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dea0181ef0 con 0x55dea0280a00 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f8c988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dea01d78c0 con 0x55dea0276140 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 0x55dea0275c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 0x55dea0275c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32838/0 (socket says 172.21.15.86:32838) 2024-04-25T19:09:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 -- 172.21.15.86:0/2150171470 learned_addr learned my addr 172.21.15.86:0/2150171470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f82a00640 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0276140 0x55dea02804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f82000640 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0282df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 -- 172.21.15.86:0/2150171470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0276140 msgr2=0x55dea02804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0276140 0x55dea02804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 -- 172.21.15.86:0/2150171470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 msgr2=0x55dea0282df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0282df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.945+0000 7f6f83400640 1 -- 172.21.15.86:0/2150171470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9feee210 con 0x55dea0275820 2024-04-25T19:09:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f83400640 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 0x55dea0275c00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009ca0 tx=0x7f6f7c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcae7dbc54ca9cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f6fe00640 1 -- 172.21.15.86:0/2150171470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c031cf0 con 0x55dea0275820 2024-04-25T19:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f6fe00640 1 -- 172.21.15.86:0/2150171470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f7c036030 con 0x55dea0275820 2024-04-25T19:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f6fe00640 1 -- 172.21.15.86:0/2150171470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c004240 con 0x55dea0275820 2024-04-25T19:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f7b400640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6f680a3e40 0x7f6f680a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f7b400640 1 -- 172.21.15.86:0/3152567839 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7f6f740d0b90 con 0x7f6f841010f0 2024-04-25T19:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.946+0000 7f6f82a00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6f680a3e40 0x7f6f680a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 -- 172.21.15.86:0/2150171470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 msgr2=0x55dea0275c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 0x55dea0275c00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009ca0 tx=0x7f6f7c0093e0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 -- 172.21.15.86:0/2150171470 shutdown_connections 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0282df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0276140 0x55dea02804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 --2- 172.21.15.86:0/2150171470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0275820 0x55dea0275c00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 -- 172.21.15.86:0/2150171470 >> 172.21.15.86:0/2150171470 conn(0x55dea0271d30 msgr2=0x55dea0272110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 -- 172.21.15.86:0/2150171470 shutdown_connections 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f8c988740 1 -- 172.21.15.86:0/2150171470 wait complete. 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.947+0000 7f6f82a00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6f680a3e40 0x7f6f680a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55dea0276dc0 tx=0x7f6f740d4640 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 -- start start 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 Processor -- start 2024-04-25T19:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0275820 0x55dea027d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea027dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0280a00 0x55dea027e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dea0283450 con 0x55dea0276140 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dea0195310 con 0x55dea0275820 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dea02835d0 con 0x55dea0280a00 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f8c988740 10 client.0 _ll_register_callbacks cb 0x55dea0163cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.948+0000 7f6f82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea027dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f83400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0275820 0x55dea027d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea027dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32840/0 (socket says 172.21.15.86:32840) 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 -- 172.21.15.86:0/1686727529 learned_addr learned my addr 172.21.15.86:0/1686727529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f83400640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0280a00 0x55dea027e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 -- 172.21.15.86:0/1686727529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0280a00 msgr2=0x55dea027e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0280a00 0x55dea027e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 -- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0275820 msgr2=0x55dea027d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0275820 0x55dea027d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 -- 172.21.15.86:0/1686727529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9ffba620 con 0x55dea0276140 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f83400640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0275820 0x55dea027d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f82000640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea027dea0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6f70012700 tx=0x7f6f70012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4526a4fac3c1f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f6cc00640 1 -- 172.21.15.86:0/1686727529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70004030 con 0x55dea0276140 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f6cc00640 1 -- 172.21.15.86:0/1686727529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7f6f700041d0 con 0x55dea0276140 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.949+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7c036030 con 0x55dea0276140 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f6cc00640 1 -- 172.21.15.86:0/1686727529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f700043d0 con 0x55dea0276140 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 10 client.15990 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dea028f760 con 0x55dea0276140 2024-04-25T19:09:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f6cc00640 1 -- 172.21.15.86:0/1686727529 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f7c036030 con 0x55dea0276140 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f6cc00640 1 -- 172.21.15.86:0/1686727529 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6f70003a00 con 0x55dea0276140 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 20 client.15990 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea0290c20 0x55dea0293030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 4 client.15990 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.950+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea01ef8b0 con 0x55dea0290c20 2024-04-25T19:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.951+0000 7f6f83400640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea0290c20 0x55dea0293030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.951+0000 7f6f83400640 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea0290c20 0x55dea0293030 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf930cb9c007ff87 server_cookie=8a76228d79d6bef9 in_seq=0 out_seq=0 2024-04-25T19:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.951+0000 7f6f6cc00640 10 client.15990 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.952+0000 7f6f6cc00640 1 -- 172.21.15.86:0/1686727529 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dea01ef8b0 con 0x55dea0290c20 2024-04-25T19:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.952+0000 7f6f6cc00640 10 client.15990 handle_command_reply: tid=0 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.952+0000 7f6f8c988740 1 client.15990 shutdown 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 msgr2=0x55dea027dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea027dea0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6f70012700 tx=0x7f6f70012bc0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 shutdown_connections 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea0290c20 0x55dea0293030 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0280a00 0x55dea027e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea027dea0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f6cc00640 0 client.15990 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1686727529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0275820 0x55dea027d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 >> 172.21.15.86:0/1686727529 conn(0x55dea0271d30 msgr2=0x55dea0277a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 shutdown_connections 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 1 -- 172.21.15.86:0/1686727529 wait complete. 2024-04-25T19:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.953+0000 7f6f8c988740 20 client.15990 trim_cache size 0 max 16384 2024-04-25T19:09:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 Processor -- start 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 -- start start 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea028cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea028c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0270d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841afe00 con 0x55dea0276140 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b1940 con 0x55dea0280a00 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f8c988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b04d0 con 0x55dea0275820 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea028c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea028c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32842/0 (socket says 172.21.15.86:32842) 2024-04-25T19:09:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f82000640 1 -- 172.21.15.86:0/1012850613 learned_addr learned my addr 172.21.15.86:0/1012850613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.969+0000 7f6f83400640 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea028cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f83400640 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0270d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f82000640 1 -- 172.21.15.86:0/1012850613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 msgr2=0x55dea028cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f82000640 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea028cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f82000640 1 -- 172.21.15.86:0/1012850613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 msgr2=0x55dea0270d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f82000640 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0270d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f82000640 1 -- 172.21.15.86:0/1012850613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea0275f70 con 0x55dea0276140 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f83400640 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea028cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f82000640 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea028c310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f70000c00 tx=0x7f6f70005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c1560976ed19a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f6f400640 1 -- 172.21.15.86:0/1012850613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70028040 con 0x55dea0276140 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f6f400640 1 -- 172.21.15.86:0/1012850613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f7000fdd0 con 0x55dea0276140 2024-04-25T19:09:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.970+0000 7f6f6f400640 1 -- 172.21.15.86:0/1012850613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70003ba0 con 0x55dea0276140 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 -- 172.21.15.86:0/1012850613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 msgr2=0x55dea028c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea028c310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f70000c00 tx=0x7f6f70005600 comp rx=0 tx=0).stop 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 -- 172.21.15.86:0/1012850613 shutdown_connections 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0270d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea028c310 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 --2- 172.21.15.86:0/1012850613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea028cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 -- 172.21.15.86:0/1012850613 >> 172.21.15.86:0/1012850613 conn(0x55dea0271d30 msgr2=0x55dea0270600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 -- 172.21.15.86:0/1012850613 shutdown_connections 2024-04-25T19:09:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 -- 172.21.15.86:0/1012850613 wait complete. 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.971+0000 7f6f8c988740 1 -- start start 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 Processor -- start 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea0286430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea0286970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0286eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b1270 con 0x55dea0276140 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b0ba0 con 0x55dea0280a00 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841ade00 con 0x55dea0275820 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f8c988740 10 client.0 _ll_register_callbacks cb 0x7f6f70009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f83400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea0286430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f83400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea0286430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35224/0 (socket says 172.21.15.86:35224) 2024-04-25T19:09:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f83400640 1 -- 172.21.15.86:0/720257301 learned_addr learned my addr 172.21.15.86:0/720257301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f83400640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0286eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.972+0000 7f6f82000640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea0286970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f83400640 1 -- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 msgr2=0x55dea0286eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f83400640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0286eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f83400640 1 -- 172.21.15.86:0/720257301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 msgr2=0x55dea0286970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f83400640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea0286970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f83400640 1 -- 172.21.15.86:0/720257301 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea02f63c0 con 0x55dea0275820 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f83400640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea0286430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02ae80 tx=0x7f6f70009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07611b2cd332d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f6cc00640 1 -- 172.21.15.86:0/720257301 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c031720 con 0x55dea0275820 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.973+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7000fdd0 con 0x55dea0275820 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f6cc00640 1 -- 172.21.15.86:0/720257301 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7f6f7c0318c0 con 0x55dea0275820 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f6cc00640 1 -- 172.21.15.86:0/720257301 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f7c004040 con 0x55dea0275820 2024-04-25T19:09:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 10 client.26086 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dea02f76f0 con 0x55dea0275820 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f6cc00640 1 -- 172.21.15.86:0/720257301 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f7000fdd0 con 0x55dea0275820 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f6cc00640 1 -- 172.21.15.86:0/720257301 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6f7c043070 con 0x55dea0275820 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 10 client.26086 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 20 client.26086 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea02f7af0 0x55dea02f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 4 client.26086 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.974+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dea01e9320 con 0x55dea02f7af0 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.975+0000 7f6f82000640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea02f7af0 0x55dea02f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.975+0000 7f6f82000640 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea02f7af0 0x55dea02f9ee0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ee18d06587ecf1 server_cookie=bba302afa781a489 in_seq=0 out_seq=0 2024-04-25T19:09:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.975+0000 7f6f6cc00640 10 client.26086 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.985+0000 7f6f6cc00640 1 -- 172.21.15.86:0/720257301 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dea01e9320 con 0x55dea02f7af0 2024-04-25T19:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.985+0000 7f6f6cc00640 10 client.26086 handle_command_reply: tid=0 2024-04-25T19:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.985+0000 7f6f8c988740 1 client.26086 shutdown 2024-04-25T19:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 msgr2=0x55dea0286430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea0286430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02ae80 tx=0x7f6f70009d50 comp rx=0 tx=0).stop 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 shutdown_connections 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55dea02f7af0 0x55dea02f9ee0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dea0280a00 0x55dea0286eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea0276140 0x55dea0286970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 --2- 172.21.15.86:0/720257301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dea0275820 0x55dea0286430 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 >> 172.21.15.86:0/720257301 conn(0x55dea0271d30 msgr2=0x55dea0270e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f6cc00640 0 client.26086 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 shutdown_connections 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 1 -- 172.21.15.86:0/720257301 wait complete. 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.986+0000 7f6f8c988740 20 client.26086 trim_cache size 0 max 16384 2024-04-25T19:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:09:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:09:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:09:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6f680a3e40 msgr2=0x7f6f680a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6f680a3e40 0x7f6f680a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55dea0276dc0 tx=0x7f6f740d4640 comp rx=0 tx=0).stop 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 msgr2=0x7f6f84102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 0x7f6f84102b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400e9f0 tx=0x7f6f7400edf0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 shutdown_connections 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6f680a3e40 0x7f6f680a6300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f841035b0 0x7f6f841a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f84106170 0x7f6f84103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 --2- 172.21.15.86:0/3152567839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f841010f0 0x7f6f84102b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 >> 172.21.15.86:0/3152567839 conn(0x7f6f84077fd0 msgr2=0x7f6f84069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 shutdown_connections 2024-04-25T19:09:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:35.987+0000 7f6f89c00640 1 -- 172.21.15.86:0/3152567839 wait complete. 2024-04-25T19:09:36.719 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:36 smithi183 ceph-mon[32344]: pgmap v204: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T19:09:36.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:36 smithi194 ceph-mon[32740]: pgmap v204: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T19:09:36.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:36 smithi086 ceph-mon[27783]: pgmap v204: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T19:09:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:38 smithi194 ceph-mon[32740]: pgmap v205: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T19:09:38.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:38 smithi086 ceph-mon[27783]: pgmap v205: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T19:09:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:38 smithi183 ceph-mon[32344]: pgmap v205: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T19:09:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:40 smithi194 ceph-mon[32740]: pgmap v206: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:09:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:40 smithi086 ceph-mon[27783]: pgmap v206: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:09:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:40 smithi183 ceph-mon[32344]: pgmap v206: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T19:09:42.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:42 smithi194 ceph-mon[32740]: pgmap v207: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T19:09:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:42 smithi086 ceph-mon[27783]: pgmap v207: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T19:09:42.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:42 smithi183 ceph-mon[32344]: pgmap v207: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T19:09:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:44 smithi194 ceph-mon[32740]: pgmap v208: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-25T19:09:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:44 smithi086 ceph-mon[27783]: pgmap v208: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-25T19:09:44.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:44 smithi183 ceph-mon[32344]: pgmap v208: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-25T19:09:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:46 smithi194 ceph-mon[32740]: pgmap v209: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:09:46.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:46 smithi086 ceph-mon[27783]: pgmap v209: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:09:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:46 smithi183 ceph-mon[32344]: pgmap v209: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:09:48.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:48 smithi086 ceph-mon[27783]: pgmap v210: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:09:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:48 smithi183 ceph-mon[32344]: pgmap v210: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:09:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:48 smithi194 ceph-mon[32740]: pgmap v210: 97 pgs: 97 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 -- 172.21.15.86:0/282849475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae981043b0 msgr2=0x7fae98104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 --2- 172.21.15.86:0/282849475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae981043b0 0x7fae98104790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c02a410 comp rx=0 tx=0).stop 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 -- 172.21.15.86:0/282849475 shutdown_connections 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 --2- 172.21.15.86:0/282849475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae98109480 0x7fae98111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 --2- 172.21.15.86:0/282849475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae98104d60 0x7fae98108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 --2- 172.21.15.86:0/282849475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae981043b0 0x7fae98104790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 -- 172.21.15.86:0/282849475 >> 172.21.15.86:0/282849475 conn(0x7fae980ffc50 msgr2=0x7fae98102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 -- 172.21.15.86:0/282849475 shutdown_connections 2024-04-25T19:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.129+0000 7faea0600640 1 -- 172.21.15.86:0/282849475 wait complete. 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.130+0000 7faea0600640 1 Processor -- start 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.130+0000 7faea0600640 1 -- start start 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7faea0600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae981043b0 0x7fae981a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7faea0600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae98109480 0x7fae981a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7faea0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9819a710 0x7fae9819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7faea0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae98114410 con 0x7fae9819a710 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7faea0600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae98114290 con 0x7fae981043b0 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7faea0600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae98114590 con 0x7fae98109480 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae981043b0 0x7fae981a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae98109480 0x7fae981a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae981043b0 0x7fae981a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59292/0 (socket says 172.21.15.86:59292) 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9dc00640 1 -- 172.21.15.86:0/4009785351 learned_addr learned my addr 172.21.15.86:0/4009785351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9e600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9819a710 0x7fae9819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9e600640 1 -- 172.21.15.86:0/4009785351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae98109480 msgr2=0x7fae981a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9e600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae98109480 0x7fae981a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9e600640 1 -- 172.21.15.86:0/4009785351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae981043b0 msgr2=0x7fae981a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9e600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae981043b0 0x7fae981a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.131+0000 7fae9e600640 1 -- 172.21.15.86:0/4009785351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9819b310 con 0x7fae9819a710 2024-04-25T19:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.132+0000 7fae9e600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9819a710 0x7fae9819ab90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fae90009fd0 tx=0x7fae9000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.132+0000 7fae96000640 1 -- 172.21.15.86:0/4009785351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000ce20 con 0x7fae9819a710 2024-04-25T19:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.132+0000 7fae96000640 1 -- 172.21.15.86:0/4009785351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae90005020 con 0x7fae9819a710 2024-04-25T19:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.132+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae981a73f0 con 0x7fae9819a710 2024-04-25T19:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.132+0000 7fae96000640 1 -- 172.21.15.86:0/4009785351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90010690 con 0x7fae9819a710 2024-04-25T19:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.132+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a77f0 con 0x7fae9819a710 2024-04-25T19:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.135+0000 7faea3565740 1 Processor -- start 2024-04-25T19:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.135+0000 7fae96000640 1 -- 172.21.15.86:0/4009785351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae900054d0 con 0x7fae9819a710 2024-04-25T19:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 -- start start 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f6a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f6cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570a6ec5900 con 0x5570a6f60140 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5570a6e6bef0 con 0x5570a6f5f820 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7faea3565740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5570a6ec18c0 con 0x5570a6f6aa00 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f6a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7fae9d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f6a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44538/0 (socket says 172.21.15.86:44538) 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7fae9dc00640 1 -- 172.21.15.86:0/2285492190 learned_addr learned my addr 172.21.15.86:0/2285492190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.136+0000 7fae9d200640 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f6cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9dc00640 1 -- 172.21.15.86:0/2285492190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 msgr2=0x5570a6f6cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f6cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9dc00640 1 -- 172.21.15.86:0/2285492190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 msgr2=0x5570a6f5fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f5fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9dc00640 1 -- 172.21.15.86:0/2285492190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a6bd8210 con 0x5570a6f60140 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae96000640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae800a3e40 0x7fae800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f6a4c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fae8c006010 tx=0x7fae8c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef86c27732adeb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae96000640 1 -- 172.21.15.86:0/4009785351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fae900d0e90 con 0x7fae9819a710 2024-04-25T19:09:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.137+0000 7fae9d200640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae800a3e40 0x7fae800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.138+0000 7fae8aa00640 1 -- 172.21.15.86:0/2285492190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c040070 con 0x5570a6f60140 2024-04-25T19:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.138+0000 7fae8aa00640 1 -- 172.21.15.86:0/2285492190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0054d0 con 0x5570a6f60140 2024-04-25T19:09:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.138+0000 7fae8aa00640 1 -- 172.21.15.86:0/2285492190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c035040 con 0x5570a6f60140 2024-04-25T19:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.138+0000 7faea3565740 1 -- 172.21.15.86:0/2285492190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 msgr2=0x5570a6f6a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.138+0000 7fae9d200640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae800a3e40 0x7fae800a6300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5570a6f5d150 tx=0x7fae84004730 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.138+0000 7faea3565740 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f6a4c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fae8c006010 tx=0x7fae8c005c00 comp rx=0 tx=0).stop 2024-04-25T19:09:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 -- 172.21.15.86:0/2285492190 shutdown_connections 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f6cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f6a4c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 --2- 172.21.15.86:0/2285492190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 -- 172.21.15.86:0/2285492190 >> 172.21.15.86:0/2285492190 conn(0x5570a6f5bd30 msgr2=0x5570a6f5c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 -- 172.21.15.86:0/2285492190 shutdown_connections 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 -- 172.21.15.86:0/2285492190 wait complete. 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.139+0000 7faea3565740 1 -- start start 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 Processor -- start 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570a6f6d5d0 con 0x5570a6f60140 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5570a6e7f310 con 0x5570a6f5f820 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5570a6f6d750 con 0x5570a6f6aa00 2024-04-25T19:09:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7faea3565740 10 client.0 _ll_register_callbacks cb 0x5570a6e4dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7fae9d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f67920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59308/0 (socket says 172.21.15.86:59308) 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7fae9d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f67e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44552/0 (socket says 172.21.15.86:44552) 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7fae9dc00640 1 -- 172.21.15.86:0/2716786429 learned_addr learned my addr 172.21.15.86:0/2716786429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.140+0000 7fae9e600640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9dc00640 1 -- 172.21.15.86:0/2716786429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 msgr2=0x5570a6f683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9dc00640 1 -- 172.21.15.86:0/2716786429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 msgr2=0x5570a6f67e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f67e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9dc00640 1 -- 172.21.15.86:0/2716786429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a6ca4620 con 0x5570a6f5f820 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9e600640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f67920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005ab0 tx=0x7fae8c0045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928ed4d0f432d94a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.141+0000 7fae7f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7fae7f400640 1 -- 172.21.15.86:0/2716786429 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c035070 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7fae7f400640 1 -- 172.21.15.86:0/2716786429 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fae8c004f20 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8c0054d0 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7fae7f400640 1 -- 172.21.15.86:0/2716786429 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae8c040040 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7faea3565740 10 client.26105 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570a6f79810 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.142+0000 7fae7f400640 1 -- 172.21.15.86:0/2716786429 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0054d0 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7fae7f400640 1 -- 172.21.15.86:0/2716786429 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fae8c003cd0 con 0x5570a6f5f820 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7faea3565740 10 client.26105 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7faea3565740 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7faea3565740 20 client.26105 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7faea3565740 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6f7ab60 0x5570a6f7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7faea3565740 4 client.26105 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570a6ed98b0 con 0x5570a6f7ab60 2024-04-25T19:09:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.143+0000 7fae9d200640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6f7ab60 0x5570a6f7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.144+0000 7fae9d200640 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6f7ab60 0x5570a6f7cf50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a028969f1169ec server_cookie=cf71bd04aa9d5613 in_seq=0 out_seq=0 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.144+0000 7fae7f400640 10 client.26105 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.144+0000 7fae7f400640 1 -- 172.21.15.86:0/2716786429 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570a6ed98b0 con 0x5570a6f7ab60 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.144+0000 7fae7f400640 10 client.26105 handle_command_reply: tid=0 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 client.26105 shutdown 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 msgr2=0x5570a6f67920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f67920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005ab0 tx=0x7fae8c0045a0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 shutdown_connections 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6f7ab60 0x5570a6f7cf50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f6aa00 0x5570a6f683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f60140 0x5570a6f67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7fae7f400640 0 client.26105 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 --2- 172.21.15.86:0/2716786429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f67920 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 >> 172.21.15.86:0/2716786429 conn(0x5570a6f5bd30 msgr2=0x5570a6f61ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 shutdown_connections 2024-04-25T19:09:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 1 -- 172.21.15.86:0/2716786429 wait complete. 2024-04-25T19:09:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.145+0000 7faea3565740 20 client.26105 trim_cache size 0 max 16384 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.161+0000 7faea3565740 1 Processor -- start 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.161+0000 7faea3565740 1 -- start start 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.161+0000 7faea3565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f7ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.161+0000 7faea3565740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 0x5570a6f804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.161+0000 7faea3565740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6f5a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.161+0000 7faea3565740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae981afec0 con 0x5570a6f6aa00 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7faea3565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b1a00 con 0x5570a6f5f820 2024-04-25T19:09:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7faea3565740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b0590 con 0x5570a6f60140 2024-04-25T19:09:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6f5a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 0x5570a6f804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6f5a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44568/0 (socket says 172.21.15.86:44568) 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9e600640 1 -- 172.21.15.86:0/4288991880 learned_addr learned my addr 172.21.15.86:0/4288991880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 0x5570a6f804b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51226/0 (socket says 172.21.15.86:51226) 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9dc00640 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f7ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 -- 172.21.15.86:0/4288991880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 msgr2=0x5570a6f7ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f7ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 -- 172.21.15.86:0/4288991880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 msgr2=0x5570a6f5a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6f5a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9d200640 1 -- 172.21.15.86:0/4288991880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a6f5b860 con 0x5570a6f60140 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.162+0000 7fae9e600640 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6f5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.163+0000 7fae9d200640 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 0x5570a6f804b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae84010fd0 tx=0x7fae84012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb2d3bd180ea12d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.163+0000 7fae8a000640 1 -- 172.21.15.86:0/4288991880 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84019040 con 0x5570a6f60140 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.163+0000 7fae8a000640 1 -- 172.21.15.86:0/4288991880 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae840047a0 con 0x5570a6f60140 2024-04-25T19:09:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.163+0000 7fae8a000640 1 -- 172.21.15.86:0/4288991880 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84030350 con 0x5570a6f60140 2024-04-25T19:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.163+0000 7faea3565740 1 -- 172.21.15.86:0/4288991880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 msgr2=0x5570a6f804b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.163+0000 7faea3565740 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 0x5570a6f804b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae84010fd0 tx=0x7fae84012040 comp rx=0 tx=0).stop 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 -- 172.21.15.86:0/4288991880 shutdown_connections 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6f5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f60140 0x5570a6f804b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 --2- 172.21.15.86:0/4288991880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f5f820 0x5570a6f7ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 -- 172.21.15.86:0/4288991880 >> 172.21.15.86:0/4288991880 conn(0x5570a6f5bd30 msgr2=0x5570a6f62c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 -- 172.21.15.86:0/4288991880 shutdown_connections 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 -- 172.21.15.86:0/4288991880 wait complete. 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 -- start start 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.164+0000 7faea3565740 1 Processor -- start 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f5f820 0x5570a6f7f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 0x5570a6f7f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6fe7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b1330 con 0x5570a6f6aa00 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b0c60 con 0x5570a6f60140 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae981adec0 con 0x5570a6f5f820 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7faea3565740 10 client.0 _ll_register_callbacks cb 0x7fae8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6fe7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 0x5570a6f7f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f5f820 0x5570a6f7f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6fe7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44584/0 (socket says 172.21.15.86:44584) 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 0x5570a6f7f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59312/0 (socket says 172.21.15.86:59312) 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9e600640 1 -- 172.21.15.86:0/2244553034 learned_addr learned my addr 172.21.15.86:0/2244553034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 -- 172.21.15.86:0/2244553034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f5f820 msgr2=0x5570a6f7f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f5f820 0x5570a6f7f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 -- 172.21.15.86:0/2244553034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 msgr2=0x5570a6fe7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6fe7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9d200640 1 -- 172.21.15.86:0/2244553034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a6fe7bc0 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.165+0000 7fae9e600640 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6fe7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.166+0000 7fae9d200640 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 0x5570a6f7f6a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae840099e0 tx=0x7fae84004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69230e94d499e883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.166+0000 7fae7f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.166+0000 7fae7f400640 1 -- 172.21.15.86:0/2244553034 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8403d060 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.166+0000 7fae7f400640 1 -- 172.21.15.86:0/2244553034 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8637+0+0 (secure 0 0 0) 0x7fae84005000 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.166+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae840047a0 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.166+0000 7fae7f400640 1 -- 172.21.15.86:0/2244553034 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae84005200 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 10 client.26111 fetch_fsmap learned FSMap version 29 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570a6fe8ed0 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7fae7f400640 1 -- 172.21.15.86:0/2244553034 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae840047a0 con 0x5570a6f60140 2024-04-25T19:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7fae7f400640 1 -- 172.21.15.86:0/2244553034 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fae84020420 con 0x5570a6f60140 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 10 client.26111 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 10 client.26111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 20 client.26111 populate_metadata read hostname 'smithi086' 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6fe95a0 0x5570a6feb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 4 client.26111 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.167+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570a6ed3320 con 0x5570a6fe95a0 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.168+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6fe95a0 0x5570a6feb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.168+0000 7fae9dc00640 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6fe95a0 0x5570a6feb9c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be552c329a4b7c0 server_cookie=10f51991d2ef6d40 in_seq=0 out_seq=0 2024-04-25T19:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.168+0000 7fae7f400640 10 client.26111 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7fae7f400640 1 -- 172.21.15.86:0/2244553034 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570a6ed3320 con 0x5570a6fe95a0 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7fae7f400640 10 client.26111 handle_command_reply: tid=0 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 client.26111 shutdown 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 msgr2=0x5570a6f7f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 0x5570a6f7f6a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae840099e0 tx=0x7fae84004950 comp rx=0 tx=0).stop 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 shutdown_connections 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5570a6fe95a0 0x5570a6feb9c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570a6f6aa00 0x5570a6fe7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570a6f60140 0x5570a6f7f6a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7fae7f400640 0 client.26111 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 --2- 172.21.15.86:0/2244553034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5570a6f5f820 0x5570a6f7f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.182+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 >> 172.21.15.86:0/2244553034 conn(0x5570a6f5bd30 msgr2=0x5570a6f759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 shutdown_connections 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea3565740 1 -- 172.21.15.86:0/2244553034 wait complete. 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea3565740 20 client.26111 trim_cache size 0 max 16384 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:09:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:09:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae800a3e40 msgr2=0x7fae800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea0600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae800a3e40 0x7fae800a6300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5570a6f5d150 tx=0x7fae84004730 comp rx=0 tx=0).stop 2024-04-25T19:09:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9819a710 msgr2=0x7fae9819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.183+0000 7faea0600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9819a710 0x7fae9819ab90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fae90009fd0 tx=0x7fae9000ef10 comp rx=0 tx=0).stop 2024-04-25T19:09:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 shutdown_connections 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae800a3e40 0x7fae800a6300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9819a710 0x7fae9819ab90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae98109480 0x7fae981a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 --2- 172.21.15.86:0/4009785351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae981043b0 0x7fae981a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 >> 172.21.15.86:0/4009785351 conn(0x7fae980ffc50 msgr2=0x7fae98101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 shutdown_connections 2024-04-25T19:09:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:09:49.184+0000 7faea0600640 1 -- 172.21.15.86:0/4009785351 wait complete. 2024-04-25T19:09:50.784 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:50 smithi194 ceph-mon[32740]: pgmap v211: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-25T19:09:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:50 smithi086 ceph-mon[27783]: pgmap v211: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-25T19:09:50.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:50 smithi183 ceph-mon[32344]: pgmap v211: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:127/600: open fd = 3 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.394 INFO:tasks.workunit.client.0.smithi086.stdout:128/600: open fd = 3 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:129/600: open fd = 3 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:130/600: open fd = 3 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:131/600: open fd = 3 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.395 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:132/600: open fd = 3 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:133/600: open fd = 3 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:134/600: open fd = 3 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:135/600: open fd = 3 2024-04-25T19:09:51.396 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:136/600: open fd = 3 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:137/600: open fd = 3 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:138/600: open fd = 3 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.397 INFO:tasks.workunit.client.0.smithi086.stdout:139/600: open fd = 3 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:140/600: open fd = 3 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:141/600: open fd = 3 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:142/600: open fd = 3 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.398 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:143/600: open fd = 3 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:144/600: open fd = 3 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:145/600: open fd = 3 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.399 INFO:tasks.workunit.client.0.smithi086.stdout:146/600: open fd = 3 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:147/600: open fd = 3 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:148/600: open fd = 3 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:149/600: open fd = 3 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.400 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:150/600: open fd = 3 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:151/600: open fd = 3 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:152/600: open fd = 3 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:153/600: open fd = 3 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.401 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:154/600: open fd = 3 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:155/600: open fd = 3 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:156/600: open fd = 3 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.402 INFO:tasks.workunit.client.0.smithi086.stdout:157/600: open fd = 3 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:158/600: open fd = 3 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:159/600: open fd = 3 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:160/600: open fd = 3 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:161/600: open fd = 3 2024-04-25T19:09:51.403 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:162/600: open fd = 3 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:163/600: open fd = 3 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:164/600: open fd = 3 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:165/600: open fd = 3 2024-04-25T19:09:51.404 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:166/600: open fd = 3 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:167/600: open fd = 3 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:168/600: open fd = 3 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.405 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:169/600: open fd = 3 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:170/600: open fd = 3 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:171/600: open fd = 3 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:172/600: open fd = 3 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.406 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:173/600: open fd = 3 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:174/600: open fd = 3 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:175/600: open fd = 3 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:176/600: open fd = 3 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.407 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:177/600: open fd = 3 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:178/600: open fd = 3 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:179/600: open fd = 3 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:180/600: open fd = 3 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.408 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:181/600: open fd = 3 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:182/600: open fd = 3 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:183/600: open fd = 3 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:184/600: open fd = 3 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.409 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:185/600: open fd = 3 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:186/600: open fd = 3 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:187/600: open fd = 3 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:09:51.410 INFO:tasks.workunit.client.0.smithi086.stdout:188/600: open fd = 3 2024-04-25T19:09:51.411 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:51.411 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:09:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:52 smithi086 ceph-mon[27783]: pgmap v212: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T19:09:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:52 smithi183 ceph-mon[32344]: pgmap v212: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T19:09:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:52 smithi194 ceph-mon[32740]: pgmap v212: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T19:09:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:54 smithi086 ceph-mon[27783]: pgmap v213: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:09:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:54 smithi183 ceph-mon[32344]: pgmap v213: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:09:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:54 smithi194 ceph-mon[32740]: pgmap v213: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:09:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:56 smithi086 ceph-mon[27783]: pgmap v214: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:09:56.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:56 smithi183 ceph-mon[32344]: pgmap v214: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:09:56.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:56 smithi194 ceph-mon[32740]: pgmap v214: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:09:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:09:58 smithi086 ceph-mon[27783]: pgmap v215: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:09:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:09:58 smithi183 ceph-mon[32344]: pgmap v215: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:09:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:09:58 smithi194 ceph-mon[32740]: pgmap v215: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:10:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:00 smithi086 ceph-mon[27783]: pgmap v216: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T19:10:00.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:00 smithi086 ceph-mon[27783]: overall HEALTH_OK 2024-04-25T19:10:00.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:00 smithi183 ceph-mon[32344]: pgmap v216: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T19:10:00.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:00 smithi183 ceph-mon[32344]: overall HEALTH_OK 2024-04-25T19:10:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:00 smithi194 ceph-mon[32740]: pgmap v216: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T19:10:00.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:00 smithi194 ceph-mon[32740]: overall HEALTH_OK 2024-04-25T19:10:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 -- 172.21.15.86:0/1824985118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 msgr2=0x7f346c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 --2- 172.21.15.86:0/1824985118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c10bfe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3454009900 tx=0x7f345402a480 comp rx=0 tx=0).stop 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 -- 172.21.15.86:0/1824985118 shutdown_connections 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 --2- 172.21.15.86:0/1824985118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c10c520 0x7f346c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 --2- 172.21.15.86:0/1824985118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c10bfe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 --2- 172.21.15.86:0/1824985118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f346c1031e0 0x7f346c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 -- 172.21.15.86:0/1824985118 >> 172.21.15.86:0/1824985118 conn(0x7f346c078000 msgr2=0x7f346c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 -- 172.21.15.86:0/1824985118 shutdown_connections 2024-04-25T19:10:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.324+0000 7f3473200640 1 -- 172.21.15.86:0/1824985118 wait complete. 2024-04-25T19:10:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.325+0000 7f3473200640 1 Processor -- start 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.325+0000 7f3473200640 1 -- start start 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3473200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c1031e0 0x7f346c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3473200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3473200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f346c10c520 0x7f346c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3473200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1121e0 con 0x7f346c103b00 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3473200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f346c112060 con 0x7f346c1031e0 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3473200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f346c112360 con 0x7f346c10c520 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3463400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3463400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48632/0 (socket says 172.21.15.86:48632) 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3463400640 1 -- 172.21.15.86:0/2860288053 learned_addr learned my addr 172.21.15.86:0/2860288053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f3471200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f346c10c520 0x7f346c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.326+0000 7f346be00640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c1031e0 0x7f346c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3463400640 1 -- 172.21.15.86:0/2860288053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f346c10c520 msgr2=0x7f346c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3463400640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f346c10c520 0x7f346c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3463400640 1 -- 172.21.15.86:0/2860288053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c1031e0 msgr2=0x7f346c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3463400640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c1031e0 0x7f346c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3463400640 1 -- 172.21.15.86:0/2860288053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c19d690 con 0x7f346c103b00 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f346be00640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c1031e0 0x7f346c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3463400640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c198be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3454005ab0 tx=0x7f3454003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3469600640 1 -- 172.21.15.86:0/2860288053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345403f070 con 0x7f346c103b00 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3469600640 1 -- 172.21.15.86:0/2860288053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3454031b60 con 0x7f346c103b00 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c19d920 con 0x7f346c103b00 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.327+0000 7f3469600640 1 -- 172.21.15.86:0/2860288053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454035030 con 0x7f346c103b00 2024-04-25T19:10:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.328+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a5210 con 0x7f346c103b00 2024-04-25T19:10:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.332+0000 7f3476003740 1 Processor -- start 2024-04-25T19:10:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.332+0000 7f3476003740 1 -- start start 2024-04-25T19:10:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.332+0000 7f3469600640 1 -- 172.21.15.86:0/2860288053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3454042070 con 0x7f346c103b00 2024-04-25T19:10:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.332+0000 7f3476003740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.332+0000 7f3476003740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.332+0000 7f3476003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3476003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ca7129900 con 0x559ca71d2e10 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3476003740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ca70cfef0 con 0x559ca71c36e0 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3476003740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559ca71258c0 con 0x559ca71c4000 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71d5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48634/0 (socket says 172.21.15.86:48634) 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 -- 172.21.15.86:0/1494263941 learned_addr learned my addr 172.21.15.86:0/1494263941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3471200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f346be00640 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 -- 172.21.15.86:0/1494263941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 msgr2=0x559ca71d28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71d28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 -- 172.21.15.86:0/1494263941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 msgr2=0x559ca71c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3463400640 1 -- 172.21.15.86:0/1494263941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca6e3c210 con 0x559ca71d2e10 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f346be00640 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71d28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.333+0000 7f3471200640 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3463400640 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71d5200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34540ee5b0 tx=0x7f34540f4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b075bf0442cb46ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3461600640 1 -- 172.21.15.86:0/1494263941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34540f9cb0 con 0x559ca71d2e10 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3469600640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f34500a3dc0 0x7f34500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3461600640 1 -- 172.21.15.86:0/1494263941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34540fe030 con 0x559ca71d2e10 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3461600640 1 -- 172.21.15.86:0/1494263941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34540bc7f0 con 0x559ca71d2e10 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3469600640 1 -- 172.21.15.86:0/2860288053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8681+0+0 (secure 0 0 0) 0x7f34540f0f90 con 0x7f346c103b00 2024-04-25T19:10:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.334+0000 7f3471200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f34500a3dc0 0x7f34500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 -- 172.21.15.86:0/1494263941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 msgr2=0x559ca71d5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71d5200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34540ee5b0 tx=0x7f34540f4f90 comp rx=0 tx=0).stop 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3471200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f34500a3dc0 0x7f34500a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3464009fd0 tx=0x7f3464009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 -- 172.21.15.86:0/1494263941 shutdown_connections 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71d5200 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71d28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 --2- 172.21.15.86:0/1494263941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 -- 172.21.15.86:0/1494263941 >> 172.21.15.86:0/1494263941 conn(0x559ca71bfd30 msgr2=0x559ca71c0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 -- 172.21.15.86:0/1494263941 shutdown_connections 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.335+0000 7f3476003740 1 -- 172.21.15.86:0/1494263941 wait complete. 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 -- start start 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 Processor -- start 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c36e0 0x559ca71cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 0x559ca71d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71d2e10 0x559ca71d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ca71d57a0 con 0x559ca71c4000 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ca70e3310 con 0x559ca71d2e10 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.336+0000 7f3476003740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559ca71d5920 con 0x559ca71c36e0 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3476003740 10 client.0 _ll_register_callbacks cb 0x559ca70b1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 0x559ca71d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 0x559ca71d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48642/0 (socket says 172.21.15.86:48642) 2024-04-25T19:10:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 -- 172.21.15.86:0/457175452 learned_addr learned my addr 172.21.15.86:0/457175452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3471200640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71d2e10 0x559ca71d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f346be00640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c36e0 0x559ca71cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 -- 172.21.15.86:0/457175452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c36e0 msgr2=0x559ca71cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c36e0 0x559ca71cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 -- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71d2e10 msgr2=0x559ca71d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71d2e10 0x559ca71d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 -- 172.21.15.86:0/457175452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca6f08620 con 0x559ca71c4000 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f3463400640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 0x559ca71d0270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f34540bcb40 tx=0x7f3454000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf033b0b1e608b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.337+0000 7f345a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f345a000640 1 -- 172.21.15.86:0/457175452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34540e74c0 con 0x559ca71c4000 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f345a000640 1 -- 172.21.15.86:0/457175452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8681+0+0 (secure 0 0 0) 0x7f34540f4350 con 0x559ca71c4000 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34540fe030 con 0x559ca71c4000 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f345a000640 1 -- 172.21.15.86:0/457175452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3454049020 con 0x559ca71c4000 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f3476003740 10 client.16026 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ca71d57a0 con 0x559ca71c4000 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f345a000640 1 -- 172.21.15.86:0/457175452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34540fe030 con 0x559ca71c4000 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f345a000640 1 -- 172.21.15.86:0/457175452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f34540ee8c0 con 0x559ca71c4000 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f3476003740 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.338+0000 7f3476003740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.339+0000 7f3476003740 20 client.16026 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.339+0000 7f3476003740 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca71e2cd0 0x559ca71e5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.339+0000 7f3476003740 4 client.16026 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.339+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca713d8b0 con 0x559ca71e2cd0 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.339+0000 7f346be00640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca71e2cd0 0x559ca71e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.340+0000 7f346be00640 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca71e2cd0 0x559ca71e5100 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e03ceb2dfe1223f server_cookie=117ac3ac5f9e28f2 in_seq=0 out_seq=0 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.340+0000 7f345a000640 10 client.16026 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.340+0000 7f345a000640 1 -- 172.21.15.86:0/457175452 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ca713d8b0 con 0x559ca71e2cd0 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.340+0000 7f345a000640 10 client.16026 handle_command_reply: tid=0 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.340+0000 7f3476003740 1 client.16026 shutdown 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 msgr2=0x559ca71d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 0x559ca71d0270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f34540bcb40 tx=0x7f3454000bc0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 shutdown_connections 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca71e2cd0 0x559ca71e5100 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71d2e10 0x559ca71d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c4000 0x559ca71d0270 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 --2- 172.21.15.86:0/457175452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c36e0 0x559ca71cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f345a000640 0 client.16026 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 >> 172.21.15.86:0/457175452 conn(0x559ca71bfd30 msgr2=0x559ca71c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 shutdown_connections 2024-04-25T19:10:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 1 -- 172.21.15.86:0/457175452 wait complete. 2024-04-25T19:10:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.341+0000 7f3476003740 20 client.16026 trim_cache size 0 max 16384 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.357+0000 7f3476003740 1 Processor -- start 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.357+0000 7f3476003740 1 -- start start 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3476003740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3476003740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3476003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3476003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1abbb0 con 0x559ca71d2e10 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3476003740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ad6f0 con 0x559ca71c36e0 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3476003740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac280 con 0x559ca71c4000 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71eaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48658/0 (socket says 172.21.15.86:48658) 2024-04-25T19:10:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 -- 172.21.15.86:0/3240907811 learned_addr learned my addr 172.21.15.86:0/3240907811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f346be00640 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3463400640 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 -- 172.21.15.86:0/3240907811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 msgr2=0x559ca71bed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71bed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 -- 172.21.15.86:0/3240907811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 msgr2=0x559ca71de660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71de660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3471200640 1 -- 172.21.15.86:0/3240907811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca71e5780 con 0x559ca71d2e10 2024-04-25T19:10:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.358+0000 7f3463400640 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71bed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:10:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.359+0000 7f3471200640 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71eaa10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34640114b0 tx=0x7f3464019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95e11c4d84c0f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.359+0000 7f3460c00640 1 -- 172.21.15.86:0/3240907811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464003970 con 0x559ca71d2e10 2024-04-25T19:10:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.359+0000 7f3460c00640 1 -- 172.21.15.86:0/3240907811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3464004020 con 0x559ca71d2e10 2024-04-25T19:10:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.359+0000 7f3460c00640 1 -- 172.21.15.86:0/3240907811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464010820 con 0x559ca71d2e10 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 -- 172.21.15.86:0/3240907811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 msgr2=0x559ca71eaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71eaa10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34640114b0 tx=0x7f3464019000 comp rx=0 tx=0).stop 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 -- 172.21.15.86:0/3240907811 shutdown_connections 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71d2e10 0x559ca71eaa10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71c4000 0x559ca71bed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 --2- 172.21.15.86:0/3240907811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c36e0 0x559ca71de660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 -- 172.21.15.86:0/3240907811 >> 172.21.15.86:0/3240907811 conn(0x559ca71bfd30 msgr2=0x559ca71be600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 -- 172.21.15.86:0/3240907811 shutdown_connections 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 -- 172.21.15.86:0/3240907811 wait complete. 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 -- start start 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.360+0000 7f3476003740 1 Processor -- start 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 0x559ca71d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c4000 0x559ca71d93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71d2e10 0x559ca72422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ad020 con 0x559ca71c36e0 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac950 con 0x559ca71c4000 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1a9bd0 con 0x559ca71d2e10 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3476003740 10 client.0 _ll_register_callbacks cb 0x7f34540f90e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f346be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 0x559ca71d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f346be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 0x559ca71d8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48674/0 (socket says 172.21.15.86:48674) 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f346be00640 1 -- 172.21.15.86:0/3056720309 learned_addr learned my addr 172.21.15.86:0/3056720309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3463400640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c4000 0x559ca71d93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f3471200640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71d2e10 0x559ca72422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f346be00640 1 -- 172.21.15.86:0/3056720309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71d2e10 msgr2=0x559ca72422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.361+0000 7f346be00640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71d2e10 0x559ca72422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f346be00640 1 -- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c4000 msgr2=0x559ca71d93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f346be00640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c4000 0x559ca71d93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f346be00640 1 -- 172.21.15.86:0/3056720309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca72428c0 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f346be00640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 0x559ca71d8e90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f345c004700 tx=0x7f345c01f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d4e49fa28a793b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f345a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f345a000640 1 -- 172.21.15.86:0/3056720309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c004050 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f345a000640 1 -- 172.21.15.86:0/3056720309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8681+0+0 (secure 0 0 0) 0x7f345c0041f0 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3464004020 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f345a000640 1 -- 172.21.15.86:0/3056720309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f345c0043f0 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f3476003740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.362+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ca7249cd0 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f345a000640 1 -- 172.21.15.86:0/3056720309 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3464004020 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f345a000640 1 -- 172.21.15.86:0/3056720309 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f345c026020 con 0x559ca71c36e0 2024-04-25T19:10:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3476003740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3476003740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3476003740 20 client.16032 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3476003740 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca724a400 0x559ca724c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3476003740 4 client.16032 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ca7137320 con 0x559ca724a400 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.363+0000 7f3463400640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca724a400 0x559ca724c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.364+0000 7f345a000640 10 client.16032 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.364+0000 7f3463400640 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca724a400 0x559ca724c820 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20eab7319f9ac01 server_cookie=da68417e7baa1a96 in_seq=0 out_seq=0 2024-04-25T19:10:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.377+0000 7f345a000640 1 -- 172.21.15.86:0/3056720309 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ca7137320 con 0x559ca724a400 2024-04-25T19:10:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.377+0000 7f345a000640 10 client.16032 handle_command_reply: tid=0 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.377+0000 7f3476003740 1 client.16032 shutdown 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 msgr2=0x559ca71d8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 0x559ca71d8e90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f345c004700 tx=0x7f345c01f920 comp rx=0 tx=0).stop 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 shutdown_connections 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559ca724a400 0x559ca724c820 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ca71d2e10 0x559ca72422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ca71c4000 0x559ca71d93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f345a000640 0 client.16032 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 --2- 172.21.15.86:0/3056720309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ca71c36e0 0x559ca71d8e90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 >> 172.21.15.86:0/3056720309 conn(0x559ca71bfd30 msgr2=0x559ca71bee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 shutdown_connections 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 1 -- 172.21.15.86:0/3056720309 wait complete. 2024-04-25T19:10:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.378+0000 7f3476003740 20 client.16032 trim_cache size 0 max 16384 2024-04-25T19:10:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:10:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:10:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:10:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f34500a3dc0 msgr2=0x7f34500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f34500a3dc0 0x7f34500a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3464009fd0 tx=0x7f3464009500 comp rx=0 tx=0).stop 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 msgr2=0x7f346c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c198be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3454005ab0 tx=0x7f3454003c10 comp rx=0 tx=0).stop 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 shutdown_connections 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f34500a3dc0 0x7f34500a6280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f346c10c520 0x7f346c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f346c103b00 0x7f346c198be0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 --2- 172.21.15.86:0/2860288053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f346c1031e0 0x7f346c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 >> 172.21.15.86:0/2860288053 conn(0x7f346c078000 msgr2=0x7f346c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 shutdown_connections 2024-04-25T19:10:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:02.379+0000 7f3473200640 1 -- 172.21.15.86:0/2860288053 wait complete. 2024-04-25T19:10:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:02 smithi086 ceph-mon[27783]: pgmap v217: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:10:02.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:02 smithi086 ceph-mon[27783]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T19:10:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:02 smithi183 ceph-mon[32344]: pgmap v217: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:10:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:02 smithi183 ceph-mon[32344]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T19:10:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:02 smithi194 ceph-mon[32740]: pgmap v217: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:10:02.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:02 smithi194 ceph-mon[32740]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T19:10:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:04 smithi086 ceph-mon[27783]: pgmap v219: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:04.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:04 smithi183 ceph-mon[32344]: pgmap v219: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:04.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:04 smithi194 ceph-mon[32740]: pgmap v219: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:06 smithi086 ceph-mon[27783]: pgmap v220: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:06.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:06 smithi183 ceph-mon[32344]: pgmap v220: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:06 smithi194 ceph-mon[32740]: pgmap v220: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:08 smithi086 ceph-mon[27783]: pgmap v221: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:08.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:08 smithi086 ceph-mon[27783]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T19:10:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:08 smithi183 ceph-mon[32344]: pgmap v221: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:08 smithi183 ceph-mon[32344]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T19:10:08.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:08 smithi194 ceph-mon[32740]: pgmap v221: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-25T19:10:08.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:08 smithi194 ceph-mon[32740]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T19:10:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:10 smithi086 ceph-mon[27783]: pgmap v223: 97 pgs: 97 active+clean; 4.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 129 KiB/s wr, 48 op/s 2024-04-25T19:10:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:10 smithi183 ceph-mon[32344]: pgmap v223: 97 pgs: 97 active+clean; 4.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 129 KiB/s wr, 48 op/s 2024-04-25T19:10:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:10 smithi194 ceph-mon[32740]: pgmap v223: 97 pgs: 97 active+clean; 4.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 129 KiB/s wr, 48 op/s 2024-04-25T19:10:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:12 smithi086 ceph-mon[27783]: pgmap v224: 97 pgs: 97 active+clean; 4.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 109 KiB/s wr, 41 op/s 2024-04-25T19:10:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:12 smithi183 ceph-mon[32344]: pgmap v224: 97 pgs: 97 active+clean; 4.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 109 KiB/s wr, 41 op/s 2024-04-25T19:10:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:12 smithi194 ceph-mon[32740]: pgmap v224: 97 pgs: 97 active+clean; 4.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 109 KiB/s wr, 41 op/s 2024-04-25T19:10:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:14 smithi086 ceph-mon[27783]: pgmap v225: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:14 smithi183 ceph-mon[32344]: pgmap v225: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:14 smithi194 ceph-mon[32740]: pgmap v225: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 -- 172.21.15.86:0/2910291079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 msgr2=0x7f87a4076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 --2- 172.21.15.86:0/2910291079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a4076860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8794009890 tx=0x7f879402a360 comp rx=0 tx=0).stop 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 -- 172.21.15.86:0/2910291079 shutdown_connections 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 --2- 172.21.15.86:0/2910291079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 --2- 172.21.15.86:0/2910291079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a4076da0 0x7f87a41128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 --2- 172.21.15.86:0/2910291079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a4076860 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 -- 172.21.15.86:0/2910291079 >> 172.21.15.86:0/2910291079 conn(0x7f87a40ffc50 msgr2=0x7f87a4102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 -- 172.21.15.86:0/2910291079 shutdown_connections 2024-04-25T19:10:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.527+0000 7f87ab200640 1 -- 172.21.15.86:0/2910291079 wait complete. 2024-04-25T19:10:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 Processor -- start 2024-04-25T19:10:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 -- start start 2024-04-25T19:10:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a4076da0 0x7f87a419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87a4118800 con 0x7f87a4112de0 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87a4118680 con 0x7f87a4076480 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.528+0000 7f87ab200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a4118980 con 0x7f87a4076da0 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:57762/0 (socket says 172.21.15.86:57762) 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40086/0 (socket says 172.21.15.86:40086) 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 -- 172.21.15.86:0/348557937 learned_addr learned my addr 172.21.15.86:0/348557937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3400640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a4076da0 0x7f87a419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 -- 172.21.15.86:0/348557937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a4076da0 msgr2=0x7f87a419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a4076da0 0x7f87a419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 -- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 msgr2=0x7f87a41a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 -- 172.21.15.86:0/348557937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a41a3d20 con 0x7f87a4076480 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a9200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.529+0000 7f87a3e00640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a419ed80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8794009860 tx=0x7f8794032d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.530+0000 7f87a0c00640 1 -- 172.21.15.86:0/348557937 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794040020 con 0x7f87a4076480 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.530+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a41a3fb0 con 0x7f87a4076480 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.530+0000 7f87a0c00640 1 -- 172.21.15.86:0/348557937 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8794030d20 con 0x7f87a4076480 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.530+0000 7f87a0c00640 1 -- 172.21.15.86:0/348557937 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794037040 con 0x7f87a4076480 2024-04-25T19:10:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.530+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a41ab8a0 con 0x7f87a4076480 2024-04-25T19:10:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.533+0000 7f87ae1d7740 1 Processor -- start 2024-04-25T19:10:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.533+0000 7f87a0c00640 1 -- 172.21.15.86:0/348557937 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8794043020 con 0x7f87a4076480 2024-04-25T19:10:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.533+0000 7f87ae1d7740 1 -- start start 2024-04-25T19:10:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.533+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 0x55f115b7dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.533+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 0x55f115b884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.533+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f115ae3900 con 0x55f115b88a00 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f115a89ef0 con 0x55f115b7d820 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f115adf8c0 con 0x55f115b7e140 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 0x55f115b884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 0x55f115b7dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 0x55f115b884c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:38842/0 (socket says 172.21.15.86:38842) 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 -- 172.21.15.86:0/1791074343 learned_addr learned my addr 172.21.15.86:0/1791074343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 0x55f115b7dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:57778/0 (socket says 172.21.15.86:57778) 2024-04-25T19:10:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3400640 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 -- 172.21.15.86:0/1791074343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 msgr2=0x55f115b7dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 0x55f115b7dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 -- 172.21.15.86:0/1791074343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 msgr2=0x55f115b8adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3e00640 1 -- 172.21.15.86:0/1791074343 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1157f6210 con 0x55f115b7e140 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a9200640 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 0x55f115b7dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.534+0000 7f87a3400640 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.535+0000 7f87a3e00640 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 0x55f115b884c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87940edad0 tx=0x7f87940f8490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f03069c0359b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.535+0000 7f8799600640 1 -- 172.21.15.86:0/1791074343 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940f22d0 con 0x55f115b7e140 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.535+0000 7f87a0c00640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87880a3dc0 0x7f87880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.535+0000 7f8799600640 1 -- 172.21.15.86:0/1791074343 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87940fd030 con 0x55f115b7e140 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.535+0000 7f8799600640 1 -- 172.21.15.86:0/1791074343 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940b8680 con 0x55f115b7e140 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.535+0000 7f87a0c00640 1 -- 172.21.15.86:0/348557937 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f87940f0400 con 0x7f87a4076480 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87a9200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87880a3dc0 0x7f87880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 -- 172.21.15.86:0/1791074343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 msgr2=0x55f115b884c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 0x55f115b884c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87940edad0 tx=0x7f87940f8490 comp rx=0 tx=0).stop 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 -- 172.21.15.86:0/1791074343 shutdown_connections 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7e140 0x55f115b884c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/1791074343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7d820 0x55f115b7dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 -- 172.21.15.86:0/1791074343 >> 172.21.15.86:0/1791074343 conn(0x55f115b79d30 msgr2=0x55f115b7a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 -- 172.21.15.86:0/1791074343 shutdown_connections 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87a9200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87880a3dc0 0x7f87880a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f87a406b450 tx=0x7f879c010040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.536+0000 7f87ae1d7740 1 -- 172.21.15.86:0/1791074343 wait complete. 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.537+0000 7f87ae1d7740 1 -- start start 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.537+0000 7f87ae1d7740 1 Processor -- start 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.537+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.537+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 0x55f115b85f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.537+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b88a00 0x55f115b86450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.537+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f115b8b3f0 con 0x55f115b7e140 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f115a9d310 con 0x55f115b88a00 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f115b8b570 con 0x55f115b7d820 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87ae1d7740 10 client.0 _ll_register_callbacks cb 0x55f115a6bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 0x55f115b85f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b88a00 0x55f115b86450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 0x55f115b85f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40098/0 (socket says 172.21.15.86:40098) 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3400640 1 -- 172.21.15.86:0/2710483288 learned_addr learned my addr 172.21.15.86:0/2710483288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 -- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b88a00 msgr2=0x55f115b86450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b88a00 0x55f115b86450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 -- 172.21.15.86:0/2710483288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 msgr2=0x55f115b85f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 0x55f115b85f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 -- 172.21.15.86:0/2710483288 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1158c2620 con 0x55f115b7d820 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3400640 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 0x55f115b85f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f87a3e00640 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b859d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87940c0950 tx=0x7f87940f8150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6383ac1f25254c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.538+0000 7f878e000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f878e000640 1 -- 172.21.15.86:0/2710483288 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940bbb60 con 0x55f115b7d820 2024-04-25T19:10:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f878e000640 1 -- 172.21.15.86:0/2710483288 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f87940c0530 con 0x55f115b7d820 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87940fd030 con 0x55f115b7d820 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f878e000640 1 -- 172.21.15.86:0/2710483288 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87940bbd40 con 0x55f115b7d820 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 10 client.26134 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f115b8b3f0 con 0x55f115b7d820 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f878e000640 1 -- 172.21.15.86:0/2710483288 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87940fd030 con 0x55f115b7d820 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f878e000640 1 -- 172.21.15.86:0/2710483288 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87940bb830 con 0x55f115b7d820 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 10 client.26134 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 10 client.26134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 20 client.26134 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115b98790 0x55f115b9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 4 client.26134 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.539+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f115af78b0 con 0x55f115b98790 2024-04-25T19:10:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.540+0000 7f87a3400640 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115b98790 0x55f115b9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.540+0000 7f87a3400640 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115b98790 0x55f115b9ab80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d258ddc24743d62 server_cookie=786bfb9410f5388c in_seq=0 out_seq=0 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.540+0000 7f878e000640 10 client.26134 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f878e000640 1 -- 172.21.15.86:0/2710483288 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f115af78b0 con 0x55f115b98790 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f878e000640 10 client.26134 handle_command_reply: tid=0 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 client.26134 shutdown 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 msgr2=0x55f115b859d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b859d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87940c0950 tx=0x7f87940f8150 comp rx=0 tx=0).stop 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 shutdown_connections 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115b98790 0x55f115b9ab80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b88a00 0x55f115b86450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b7e140 0x55f115b85f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f878e000640 0 client.26134 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2710483288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b859d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.541+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 >> 172.21.15.86:0/2710483288 conn(0x55f115b79d30 msgr2=0x55f115b7fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.542+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 shutdown_connections 2024-04-25T19:10:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.542+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2710483288 wait complete. 2024-04-25T19:10:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.542+0000 7f87ae1d7740 20 client.26134 trim_cache size 0 max 16384 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 Processor -- start 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 -- start start 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b94150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b7a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115ba0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87a41b43c0 con 0x55f115b88a00 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87a41b5f00 con 0x55f115b7e140 2024-04-25T19:10:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.558+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a41b4a90 con 0x55f115b7d820 2024-04-25T19:10:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b7a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b94150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b7a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:57790/0 (socket says 172.21.15.86:57790) 2024-04-25T19:10:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b94150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:38872/0 (socket says 172.21.15.86:38872) 2024-04-25T19:10:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3400640 1 -- 172.21.15.86:0/3387526214 learned_addr learned my addr 172.21.15.86:0/3387526214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115ba0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 -- 172.21.15.86:0/3387526214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 msgr2=0x55f115b7a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b7a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 -- 172.21.15.86:0/3387526214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 msgr2=0x55f115ba0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115ba0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3e00640 1 -- 172.21.15.86:0/3387526214 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f115b84980 con 0x55f115b7d820 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.559+0000 7f87a3400640 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b7a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.560+0000 7f87a3e00640 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b94150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f87940bc0e0 tx=0x7f879402a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc37ffb3d00c7869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.560+0000 7f8798c00640 1 -- 172.21.15.86:0/3387526214 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940bbf20 con 0x55f115b7d820 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.560+0000 7f8798c00640 1 -- 172.21.15.86:0/3387526214 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87940fd070 con 0x55f115b7d820 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.560+0000 7f8798c00640 1 -- 172.21.15.86:0/3387526214 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940bc1b0 con 0x55f115b7d820 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 -- 172.21.15.86:0/3387526214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 msgr2=0x55f115b94150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b94150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f87940bc0e0 tx=0x7f879402a360 comp rx=0 tx=0).stop 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 -- 172.21.15.86:0/3387526214 shutdown_connections 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115ba0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b7a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/3387526214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b94150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 -- 172.21.15.86:0/3387526214 >> 172.21.15.86:0/3387526214 conn(0x55f115b79d30 msgr2=0x55f115b7fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.561+0000 7f87ae1d7740 1 -- 172.21.15.86:0/3387526214 shutdown_connections 2024-04-25T19:10:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.562+0000 7f87ae1d7740 1 -- 172.21.15.86:0/3387526214 wait complete. 2024-04-25T19:10:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.562+0000 7f87ae1d7740 1 -- start start 2024-04-25T19:10:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.562+0000 7f87ae1d7740 1 Processor -- start 2024-04-25T19:10:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b8e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87a41b5830 con 0x55f115b88a00 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87a41b5160 con 0x55f115b7e140 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a41b23a0 con 0x55f115b7d820 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87ae1d7740 10 client.0 _ll_register_callbacks cb 0x7f87940f8250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b8e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b8dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:38880/0 (socket says 172.21.15.86:38880) 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b8e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:57806/0 (socket says 172.21.15.86:57806) 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3e00640 1 -- 172.21.15.86:0/2376781467 learned_addr learned my addr 172.21.15.86:0/2376781467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a9200640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 -- 172.21.15.86:0/2376781467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 msgr2=0x55f115b8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 -- 172.21.15.86:0/2376781467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 msgr2=0x55f115b8ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3400640 1 -- 172.21.15.86:0/2376781467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f115b7df70 con 0x55f115b7e140 2024-04-25T19:10:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.563+0000 7f87a3e00640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.564+0000 7f87a3400640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b8e520 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8790004700 tx=0x7f8790012b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e029eae83f64f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.564+0000 7f878e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:10:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.565+0000 7f878e000640 1 -- 172.21.15.86:0/2376781467 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790020710 con 0x55f115b7e140 2024-04-25T19:10:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.565+0000 7f878e000640 1 -- 172.21.15.86:0/2376781467 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f87900208b0 con 0x55f115b7e140 2024-04-25T19:10:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.565+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87940fd070 con 0x55f115b7e140 2024-04-25T19:10:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.566+0000 7f878e000640 1 -- 172.21.15.86:0/2376781467 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8790030050 con 0x55f115b7e140 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.566+0000 7f87ae1d7740 10 client.26147 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.566+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f115bfecc0 con 0x55f115b7e140 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.566+0000 7f878e000640 1 -- 172.21.15.86:0/2376781467 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87940fd070 con 0x55f115b7e140 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.566+0000 7f878e000640 1 -- 172.21.15.86:0/2376781467 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8790010430 con 0x55f115b7e140 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87ae1d7740 10 client.26147 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87ae1d7740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87ae1d7740 20 client.26147 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115bff3f0 0x55f115c01810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87ae1d7740 4 client.26147 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f115af1320 con 0x55f115bff3f0 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87a3e00640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115bff3f0 0x55f115c01810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f87a3e00640 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115bff3f0 0x55f115c01810 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6115d7963ddae1a server_cookie=8424aa3e75b984a8 in_seq=0 out_seq=0 2024-04-25T19:10:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.567+0000 7f878e000640 10 client.26147 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f878e000640 1 -- 172.21.15.86:0/2376781467 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f115af1320 con 0x55f115bff3f0 2024-04-25T19:10:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f878e000640 10 client.26147 handle_command_reply: tid=0 2024-04-25T19:10:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 client.26147 shutdown 2024-04-25T19:10:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 msgr2=0x55f115b8e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b8e520 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8790004700 tx=0x7f8790012b20 comp rx=0 tx=0).stop 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 shutdown_connections 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f115bff3f0 0x55f115c01810 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f115b88a00 0x55f115b8ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f115b7e140 0x55f115b8e520 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f87ae1d7740 1 --2- 172.21.15.86:0/2376781467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f115b7d820 0x55f115b8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.579+0000 7f878e000640 0 client.26147 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.580+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 >> 172.21.15.86:0/2376781467 conn(0x55f115b79d30 msgr2=0x55f115b7fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.580+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 shutdown_connections 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.580+0000 7f87ae1d7740 1 -- 172.21.15.86:0/2376781467 wait complete. 2024-04-25T19:10:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.580+0000 7f87ae1d7740 20 client.26147 trim_cache size 0 max 16384 2024-04-25T19:10:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:10:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:10:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:10:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87880a3dc0 msgr2=0x7f87880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87880a3dc0 0x7f87880a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f87a406b450 tx=0x7f879c010040 comp rx=0 tx=0).stop 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 msgr2=0x7f87a419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a419ed80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8794009860 tx=0x7f8794032d20 comp rx=0 tx=0).stop 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 shutdown_connections 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87880a3dc0 0x7f87880a6280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a4112de0 0x7f87a41a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a4076da0 0x7f87a419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 --2- 172.21.15.86:0/348557937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4076480 0x7f87a419ed80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.581+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 >> 172.21.15.86:0/348557937 conn(0x7f87a40ffc50 msgr2=0x7f87a4114570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.582+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 shutdown_connections 2024-04-25T19:10:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:15.582+0000 7f87ab200640 1 -- 172.21.15.86:0/348557937 wait complete. 2024-04-25T19:10:15.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:15 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:10:15.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:15 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:10:15.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:15 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:10:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:16 smithi086 ceph-mon[27783]: pgmap v226: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:16.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:16 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:10:16.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:16 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:10:16.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:16 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:10:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:16 smithi183 ceph-mon[32344]: pgmap v226: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:16 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:10:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:16 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:10:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:16 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:10:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:16 smithi194 ceph-mon[32740]: pgmap v226: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:16.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:16 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:10:16.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:16 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:10:16.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:16 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:10:18.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:18 smithi086 ceph-mon[27783]: pgmap v227: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:18 smithi183 ceph-mon[32344]: pgmap v227: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:18 smithi194 ceph-mon[32740]: pgmap v227: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:20 smithi183 ceph-mon[32344]: pgmap v228: 97 pgs: 97 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T19:10:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:20 smithi194 ceph-mon[32740]: pgmap v228: 97 pgs: 97 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T19:10:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:20 smithi086 ceph-mon[27783]: pgmap v228: 97 pgs: 97 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T19:10:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:22 smithi183 ceph-mon[32344]: pgmap v229: 97 pgs: 97 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:22 smithi194 ceph-mon[32740]: pgmap v229: 97 pgs: 97 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:22 smithi086 ceph-mon[27783]: pgmap v229: 97 pgs: 97 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:10:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:24 smithi183 ceph-mon[32344]: pgmap v230: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-25T19:10:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:24 smithi194 ceph-mon[32740]: pgmap v230: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-25T19:10:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:24 smithi086 ceph-mon[27783]: pgmap v230: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-25T19:10:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:26 smithi183 ceph-mon[32344]: pgmap v231: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T19:10:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:26 smithi194 ceph-mon[32740]: pgmap v231: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T19:10:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:26 smithi086 ceph-mon[27783]: pgmap v231: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 -- 172.21.15.86:0/1332653578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08501041d0 msgr2=0x7f08501045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 --2- 172.21.15.86:0/1332653578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08501041d0 0x7f08501045b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0840009900 tx=0x7f084002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 -- 172.21.15.86:0/1332653578 shutdown_connections 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 --2- 172.21.15.86:0/1332653578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501092a0 0x7f0850110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 --2- 172.21.15.86:0/1332653578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0850104b80 0x7f0850108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 --2- 172.21.15.86:0/1332653578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08501041d0 0x7f08501045b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 -- 172.21.15.86:0/1332653578 >> 172.21.15.86:0/1332653578 conn(0x7f08500ffa70 msgr2=0x7f0850101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 -- 172.21.15.86:0/1332653578 shutdown_connections 2024-04-25T19:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.731+0000 7f0854a00640 1 -- 172.21.15.86:0/1332653578 wait complete. 2024-04-25T19:10:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.732+0000 7f0854a00640 1 Processor -- start 2024-04-25T19:10:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 -- start start 2024-04-25T19:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08501041d0 0x7f08501a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501092a0 0x7f08501a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 0x7f085019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0850114200 con 0x7f08501092a0 2024-04-25T19:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0850114080 con 0x7f08501041d0 2024-04-25T19:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f0854a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0850114380 con 0x7f085019a600 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f084f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 0x7f085019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f084e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08501041d0 0x7f08501a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f084f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 0x7f085019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33304/0 (socket says 172.21.15.86:33304) 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f084f200640 1 -- 172.21.15.86:0/3449676881 learned_addr learned my addr 172.21.15.86:0/3449676881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.733+0000 7f084de00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501092a0 0x7f08501a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f084f200640 1 -- 172.21.15.86:0/3449676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08501041d0 msgr2=0x7f08501a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f084f200640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08501041d0 0x7f08501a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f084f200640 1 -- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501092a0 msgr2=0x7f08501a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f084f200640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501092a0 0x7f08501a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f084f200640 1 -- 172.21.15.86:0/3449676881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085019b120 con 0x7f085019a600 2024-04-25T19:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f084f200640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 0x7f085019aab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f084800eb10 tx=0x7f084800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f0847400640 1 -- 172.21.15.86:0/3449676881 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848004b20 con 0x7f085019a600 2024-04-25T19:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.734+0000 7f0847400640 1 -- 172.21.15.86:0/3449676881 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0848004cc0 con 0x7f085019a600 2024-04-25T19:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.735+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085019b410 con 0x7f085019a600 2024-04-25T19:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.735+0000 7f0847400640 1 -- 172.21.15.86:0/3449676881 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848010670 con 0x7f085019a600 2024-04-25T19:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.735+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08501a7660 con 0x7f085019a600 2024-04-25T19:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.738+0000 7f085839c740 1 Processor -- start 2024-04-25T19:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.738+0000 7f0847400640 1 -- 172.21.15.86:0/3449676881 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0848010830 con 0x7f085019a600 2024-04-25T19:10:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 -- start start 2024-04-25T19:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 0x55f2b8336d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83372d0 0x55f2b8341650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b8341b90 0x55f2b8343f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2b829c900 con 0x55f2b83372d0 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8242ef0 con 0x55f2b8341b90 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f085839c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f2b82988c0 con 0x55f2b83369b0 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f084de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 0x55f2b8336d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f084de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 0x55f2b8336d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33306/0 (socket says 172.21.15.86:33306) 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f084de00640 1 -- 172.21.15.86:0/2844344213 learned_addr learned my addr 172.21.15.86:0/2844344213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.739+0000 7f084e800640 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83372d0 0x55f2b8341650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b8341b90 0x55f2b8343f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 -- 172.21.15.86:0/2844344213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b8341b90 msgr2=0x55f2b8343f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b8341b90 0x55f2b8343f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 -- 172.21.15.86:0/2844344213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83372d0 msgr2=0x55f2b8341650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83372d0 0x55f2b8341650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 -- 172.21.15.86:0/2844344213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b7faf210 con 0x55f2b83369b0 2024-04-25T19:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f0847400640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f08340a3d90 0x7f08340a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f0847400640 1 -- 172.21.15.86:0/3449676881 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f08480d0d60 con 0x7f085019a600 2024-04-25T19:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084de00640 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 0x55f2b8336d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08380058c0 tx=0x7f0838005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955f02cb4be75616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.740+0000 7f084e800640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f08340a3d90 0x7f08340a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.741+0000 7f083f400640 1 -- 172.21.15.86:0/2844344213 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838004050 con 0x55f2b83369b0 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.741+0000 7f083f400640 1 -- 172.21.15.86:0/2844344213 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08380054d0 con 0x55f2b83369b0 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.741+0000 7f083f400640 1 -- 172.21.15.86:0/2844344213 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838004ac0 con 0x55f2b83369b0 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.741+0000 7f085839c740 1 -- 172.21.15.86:0/2844344213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 msgr2=0x55f2b8336d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 0x55f2b8336d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08380058c0 tx=0x7f0838005850 comp rx=0 tx=0).stop 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.741+0000 7f084e800640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f08340a3d90 0x7f08340a6250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0840031440 tx=0x7f0840005b90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 -- 172.21.15.86:0/2844344213 shutdown_connections 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b8341b90 0x55f2b8343f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83372d0 0x55f2b8341650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 --2- 172.21.15.86:0/2844344213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83369b0 0x55f2b8336d90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 -- 172.21.15.86:0/2844344213 >> 172.21.15.86:0/2844344213 conn(0x55f2b8332ec0 msgr2=0x55f2b83332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 -- 172.21.15.86:0/2844344213 shutdown_connections 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 -- 172.21.15.86:0/2844344213 wait complete. 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 -- start start 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.742+0000 7f085839c740 1 Processor -- start 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83369b0 0x55f2b833eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83372d0 0x55f2b833f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 0x55f2b833f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2b83447c0 con 0x55f2b8341b90 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8256310 con 0x55f2b83369b0 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8344940 con 0x55f2b83372d0 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f085839c740 10 client.0 _ll_register_callbacks cb 0x55f2b8224cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f084f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 0x55f2b833f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f084de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83372d0 0x55f2b833f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f084f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 0x55f2b833f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48080/0 (socket says 172.21.15.86:48080) 2024-04-25T19:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f084e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83369b0 0x55f2b833eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.743+0000 7f084f200640 1 -- 172.21.15.86:0/2594602561 learned_addr learned my addr 172.21.15.86:0/2594602561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084f200640 1 -- 172.21.15.86:0/2594602561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83372d0 msgr2=0x55f2b833f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084f200640 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83372d0 0x55f2b833f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084f200640 1 -- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83369b0 msgr2=0x55f2b833eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084f200640 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83369b0 0x55f2b833eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084f200640 1 -- 172.21.15.86:0/2594602561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b807b620 con 0x55f2b8341b90 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084e800640 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83369b0 0x55f2b833eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f084f200640 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 0x55f2b833f5a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f08480049d0 tx=0x7f084809c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff215dce52a9a107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f0833e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f0833e00640 1 -- 172.21.15.86:0/2594602561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08480b6060 con 0x55f2b8341b90 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f0833e00640 1 -- 172.21.15.86:0/2594602561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f08480bd0c0 con 0x55f2b8341b90 2024-04-25T19:10:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.744+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08380054d0 con 0x55f2b8341b90 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f0833e00640 1 -- 172.21.15.86:0/2594602561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08480b6200 con 0x55f2b8341b90 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2b8350a20 con 0x55f2b8341b90 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f0833e00640 1 -- 172.21.15.86:0/2594602561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08380054d0 con 0x55f2b8341b90 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f0833e00640 1 -- 172.21.15.86:0/2594602561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f08480a5470 con 0x55f2b8341b90 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 20 client.16062 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b8351ee0 0x55f2b83542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 4 client.16062 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:10:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.745+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b82b08b0 con 0x55f2b8351ee0 2024-04-25T19:10:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.746+0000 7f084e800640 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b8351ee0 0x55f2b83542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.746+0000 7f084e800640 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b8351ee0 0x55f2b83542d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2fa9f0afbd8658d server_cookie=3ae1b89006b1c952 in_seq=0 out_seq=0 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.746+0000 7f0833e00640 10 client.16062 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.747+0000 7f0833e00640 1 -- 172.21.15.86:0/2594602561 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2b82b08b0 con 0x55f2b8351ee0 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.747+0000 7f0833e00640 10 client.16062 handle_command_reply: tid=0 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.747+0000 7f085839c740 1 client.16062 shutdown 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.747+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 msgr2=0x55f2b833f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.747+0000 7f085839c740 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 0x55f2b833f5a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f08480049d0 tx=0x7f084809c890 comp rx=0 tx=0).stop 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.747+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 shutdown_connections 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b8351ee0 0x55f2b83542d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b8341b90 0x55f2b833f5a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b83372d0 0x55f2b833f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 --2- 172.21.15.86:0/2594602561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83369b0 0x55f2b833eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 >> 172.21.15.86:0/2594602561 conn(0x55f2b8332ec0 msgr2=0x55f2b8338c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f0833e00640 0 client.16062 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 shutdown_connections 2024-04-25T19:10:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 1 -- 172.21.15.86:0/2594602561 wait complete. 2024-04-25T19:10:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.748+0000 7f085839c740 20 client.16062 trim_cache size 0 max 16384 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.764+0000 7f085839c740 1 Processor -- start 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.764+0000 7f085839c740 1 -- start start 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f085839c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b834d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f085839c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8359b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f085839c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b835a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f085839c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08501afe30 con 0x55f2b83369b0 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f085839c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08501b1970 con 0x55f2b83372d0 2024-04-25T19:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f085839c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08501b0500 con 0x55f2b8341b90 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8359b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8359b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:36834/0 (socket says 172.21.15.86:36834) 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 -- 172.21.15.86:0/2110500118 learned_addr learned my addr 172.21.15.86:0/2110500118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084e800640 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b834d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084f200640 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b835a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 -- 172.21.15.86:0/2110500118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 msgr2=0x55f2b835a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b835a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 -- 172.21.15.86:0/2110500118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 msgr2=0x55f2b834d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b834d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084de00640 1 -- 172.21.15.86:0/2110500118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b8337100 con 0x55f2b83372d0 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.765+0000 7f084f200640 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b835a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.766+0000 7f084de00640 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8359b70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0838005670 tx=0x7f08380056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957a623055a19864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.766+0000 7f083ea00640 1 -- 172.21.15.86:0/2110500118 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838005c50 con 0x55f2b83372d0 2024-04-25T19:10:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.766+0000 7f083ea00640 1 -- 172.21.15.86:0/2110500118 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800f9b0 con 0x55f2b83372d0 2024-04-25T19:10:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.766+0000 7f083ea00640 1 -- 172.21.15.86:0/2110500118 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083801f740 con 0x55f2b83372d0 2024-04-25T19:10:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 -- 172.21.15.86:0/2110500118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 msgr2=0x55f2b8359b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8359b70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0838005670 tx=0x7f08380056c0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 -- 172.21.15.86:0/2110500118 shutdown_connections 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b835a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8359b70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 --2- 172.21.15.86:0/2110500118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b834d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 -- 172.21.15.86:0/2110500118 >> 172.21.15.86:0/2110500118 conn(0x55f2b8332ec0 msgr2=0x55f2b8339e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 -- 172.21.15.86:0/2110500118 shutdown_connections 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 -- 172.21.15.86:0/2110500118 wait complete. 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.767+0000 7f085839c740 1 -- start start 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 Processor -- start 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b8349db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8347140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b8347680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08501b12a0 con 0x55f2b83369b0 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08501b0bd0 con 0x55f2b83372d0 2024-04-25T19:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ade30 con 0x55f2b8341b90 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.768+0000 7f085839c740 10 client.0 _ll_register_callbacks cb 0x7f0848098f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b8349db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b8349db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48098/0 (socket says 172.21.15.86:48098) 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084e800640 1 -- 172.21.15.86:0/2356315234 learned_addr learned my addr 172.21.15.86:0/2356315234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8347140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084f200640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b8347680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 -- 172.21.15.86:0/2356315234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 msgr2=0x55f2b8347680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b8347680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 -- 172.21.15.86:0/2356315234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 msgr2=0x55f2b8349db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b8349db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 -- 172.21.15.86:0/2356315234 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b8347cc0 con 0x55f2b83372d0 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084e800640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b8349db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f084de00640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8347140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0838005670 tx=0x7f0838013510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d2fc75362d0bed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.769+0000 7f0833e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f0833e00640 1 -- 172.21.15.86:0/2356315234 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838005c50 con 0x55f2b83372d0 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083800f9b0 con 0x55f2b83372d0 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f0833e00640 1 -- 172.21.15.86:0/2356315234 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f0838005050 con 0x55f2b83372d0 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f0833e00640 1 -- 172.21.15.86:0/2356315234 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0838005270 con 0x55f2b83372d0 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f085839c740 10 client.26165 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2b83b8270 con 0x55f2b83372d0 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.770+0000 7f0833e00640 1 -- 172.21.15.86:0/2356315234 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800f9b0 con 0x55f2b83372d0 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f0833e00640 1 -- 172.21.15.86:0/2356315234 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f083801d020 con 0x55f2b83372d0 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f085839c740 10 client.26165 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f085839c740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f085839c740 20 client.26165 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f085839c740 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b83b86f0 0x55f2b83baae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f085839c740 4 client.26165 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2b82aa320 con 0x55f2b83b86f0 2024-04-25T19:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.771+0000 7f084e800640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b83b86f0 0x55f2b83baae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.772+0000 7f0833e00640 10 client.26165 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.772+0000 7f084e800640 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b83b86f0 0x55f2b83baae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87832d2ecc0c317c server_cookie=f8af297b04dfbfc7 in_seq=0 out_seq=0 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f0833e00640 1 -- 172.21.15.86:0/2356315234 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2b82aa320 con 0x55f2b83b86f0 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f0833e00640 10 client.26165 handle_command_reply: tid=0 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 client.26165 shutdown 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 msgr2=0x55f2b8347140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8347140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0838005670 tx=0x7f0838013510 comp rx=0 tx=0).stop 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 shutdown_connections 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f2b83b86f0 0x55f2b83baae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8341b90 0x55f2b8347680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2b83372d0 0x55f2b8347140 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 --2- 172.21.15.86:0/2356315234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2b83369b0 0x55f2b8349db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f0833e00640 0 client.26165 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 >> 172.21.15.86:0/2356315234 conn(0x55f2b8332ec0 msgr2=0x55f2b833eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 shutdown_connections 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.786+0000 7f085839c740 1 -- 172.21.15.86:0/2356315234 wait complete. 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f085839c740 20 client.26165 trim_cache size 0 max 16384 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:10:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:10:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f08340a3d90 msgr2=0x7f08340a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f08340a3d90 0x7f08340a6250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0840031440 tx=0x7f0840005b90 comp rx=0 tx=0).stop 2024-04-25T19:10:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 msgr2=0x7f085019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 0x7f085019aab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f084800eb10 tx=0x7f084800c2d0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 shutdown_connections 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f08340a3d90 0x7f08340a6250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f085019a600 0x7f085019aab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08501092a0 0x7f08501a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 --2- 172.21.15.86:0/3449676881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08501041d0 0x7f08501a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.787+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 >> 172.21.15.86:0/3449676881 conn(0x7f08500ffa70 msgr2=0x7f0850100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.788+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 shutdown_connections 2024-04-25T19:10:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:28.788+0000 7f0854a00640 1 -- 172.21.15.86:0/3449676881 wait complete. 2024-04-25T19:10:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:28 smithi183 ceph-mon[32344]: pgmap v232: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T19:10:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:28 smithi194 ceph-mon[32740]: pgmap v232: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T19:10:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:28 smithi086 ceph-mon[27783]: pgmap v232: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T19:10:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:30 smithi183 ceph-mon[32344]: pgmap v233: 97 pgs: 97 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:30 smithi194 ceph-mon[32740]: pgmap v233: 97 pgs: 97 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:30 smithi086 ceph-mon[27783]: pgmap v233: 97 pgs: 97 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:32.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:32 smithi183 ceph-mon[32344]: pgmap v234: 97 pgs: 97 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T19:10:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:32 smithi194 ceph-mon[32740]: pgmap v234: 97 pgs: 97 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T19:10:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:32 smithi086 ceph-mon[27783]: pgmap v234: 97 pgs: 97 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T19:10:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:34 smithi183 ceph-mon[32344]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:34 smithi194 ceph-mon[32740]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:34 smithi086 ceph-mon[27783]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:36 smithi183 ceph-mon[32344]: pgmap v236: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:10:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:36 smithi194 ceph-mon[32740]: pgmap v236: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:10:37.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:36 smithi086 ceph-mon[27783]: pgmap v236: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:10:38.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:38 smithi183 ceph-mon[32344]: pgmap v237: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:10:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:38 smithi194 ceph-mon[32740]: pgmap v237: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:10:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:38 smithi086 ceph-mon[27783]: pgmap v237: 97 pgs: 97 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:10:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:40 smithi183 ceph-mon[32344]: pgmap v238: 97 pgs: 97 active+clean; 5.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:40 smithi194 ceph-mon[32740]: pgmap v238: 97 pgs: 97 active+clean; 5.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:41.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:40 smithi086 ceph-mon[27783]: pgmap v238: 97 pgs: 97 active+clean; 5.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T19:10:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 -- 172.21.15.86:0/2950256026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa990108490 msgr2=0x7fa9901088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 --2- 172.21.15.86:0/2950256026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa990108490 0x7fa9901088d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa978009930 tx=0x7fa97802a420 comp rx=0 tx=0).stop 2024-04-25T19:10:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 -- 172.21.15.86:0/2950256026 shutdown_connections 2024-04-25T19:10:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 --2- 172.21.15.86:0/2950256026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990108e10 0x7fa990115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 --2- 172.21.15.86:0/2950256026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa990108490 0x7fa9901088d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 --2- 172.21.15.86:0/2950256026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa99010e420 0x7fa99010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.926+0000 7fa994e00640 1 -- 172.21.15.86:0/2950256026 >> 172.21.15.86:0/2950256026 conn(0x7fa990077d30 msgr2=0x7fa990106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.927+0000 7fa994e00640 1 -- 172.21.15.86:0/2950256026 shutdown_connections 2024-04-25T19:10:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.927+0000 7fa994e00640 1 -- 172.21.15.86:0/2950256026 wait complete. 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.927+0000 7fa994e00640 1 Processor -- start 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 -- start start 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa990108490 0x7fa9901a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990108e10 0x7fa9901a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 0x7fa99019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901185f0 con 0x7fa99010e420 2024-04-25T19:10:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa990118470 con 0x7fa990108490 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa994e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa990118770 con 0x7fa990108e10 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 0x7fa99019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 0x7fa99019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42392/0 (socket says 172.21.15.86:42392) 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa98fe00640 1 -- 172.21.15.86:0/3232164366 learned_addr learned my addr 172.21.15.86:0/3232164366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa986a00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990108e10 0x7fa9901a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.928+0000 7fa98f400640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa990108490 0x7fa9901a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98fe00640 1 -- 172.21.15.86:0/3232164366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990108e10 msgr2=0x7fa9901a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98fe00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990108e10 0x7fa9901a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98fe00640 1 -- 172.21.15.86:0/3232164366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa990108490 msgr2=0x7fa9901a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98fe00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa990108490 0x7fa9901a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98fe00640 1 -- 172.21.15.86:0/3232164366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99019f0f0 con 0x7fa99010e420 2024-04-25T19:10:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98fe00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 0x7fa99019e970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa98800eb10 tx=0x7fa98800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98cc00640 1 -- 172.21.15.86:0/3232164366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988004b20 con 0x7fa99010e420 2024-04-25T19:10:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98cc00640 1 -- 172.21.15.86:0/3232164366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa988004cc0 con 0x7fa99010e420 2024-04-25T19:10:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99019f3e0 con 0x7fa99010e420 2024-04-25T19:10:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.929+0000 7fa98cc00640 1 -- 172.21.15.86:0/3232164366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988010670 con 0x7fa99010e420 2024-04-25T19:10:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.930+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9901ab680 con 0x7fa99010e420 2024-04-25T19:10:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 Processor -- start 2024-04-25T19:10:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa98cc00640 1 -- 172.21.15.86:0/3232164366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa988005020 con 0x7fa99010e420 2024-04-25T19:10:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 -- start start 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d629b0 0x559d41d62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 0x559d41d6d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d6db90 0x559d41d6ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d41cc8900 con 0x559d41d6db90 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.933+0000 7fa999238740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d41c6eef0 con 0x559d41d629b0 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa999238740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d41cc48c0 con 0x559d41d632d0 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 0x559d41d6d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa986a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d629b0 0x559d41d62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 0x559d41d6d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48878/0 (socket says 172.21.15.86:48878) 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 -- 172.21.15.86:0/1990531475 learned_addr learned my addr 172.21.15.86:0/1990531475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa986a00640 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d6db90 0x559d41d6ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 -- 172.21.15.86:0/1990531475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d629b0 msgr2=0x559d41d62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d629b0 0x559d41d62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 -- 172.21.15.86:0/1990531475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d6db90 msgr2=0x559d41d6ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d6db90 0x559d41d6ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 -- 172.21.15.86:0/1990531475 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d419db210 con 0x559d41d632d0 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa986a00640 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d629b0 0x559d41d62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98cc00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9740a3d10 0x7fa9740a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.934+0000 7fa98f400640 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 0x559d41d6d650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa980009c10 tx=0x7fa980011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a1f1b553833d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa98cc00640 1 -- 172.21.15.86:0/3232164366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7fa9880d0e00 con 0x7fa99010e420 2024-04-25T19:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa984c00640 1 -- 172.21.15.86:0/1990531475 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980004230 con 0x559d41d632d0 2024-04-25T19:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa984c00640 1 -- 172.21.15.86:0/1990531475 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9800043d0 con 0x559d41d632d0 2024-04-25T19:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa984c00640 1 -- 172.21.15.86:0/1990531475 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980019640 con 0x559d41d632d0 2024-04-25T19:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa986a00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9740a3d10 0x7fa9740a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa999238740 1 -- 172.21.15.86:0/1990531475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 msgr2=0x559d41d6d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa999238740 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 0x559d41d6d650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa980009c10 tx=0x7fa980011810 comp rx=0 tx=0).stop 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.935+0000 7fa986a00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9740a3d10 0x7fa9740a61d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa978009900 tx=0x7fa978000bc0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 -- 172.21.15.86:0/1990531475 shutdown_connections 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d6db90 0x559d41d6ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d632d0 0x559d41d6d650 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 --2- 172.21.15.86:0/1990531475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d629b0 0x559d41d62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 -- 172.21.15.86:0/1990531475 >> 172.21.15.86:0/1990531475 conn(0x559d41d5eec0 msgr2=0x559d41d5f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 -- 172.21.15.86:0/1990531475 shutdown_connections 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 -- 172.21.15.86:0/1990531475 wait complete. 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 -- start start 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.936+0000 7fa999238740 1 Processor -- start 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d6aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d6b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41d6b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d41d707c0 con 0x559d41d629b0 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d41c82310 con 0x559d41d632d0 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d41d70940 con 0x559d41d6db90 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa999238740 10 client.0 _ll_register_callbacks cb 0x559d41c50cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa98f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d6aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41d6b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa98f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d6aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42420/0 (socket says 172.21.15.86:42420) 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa98f400640 1 -- 172.21.15.86:0/418912189 learned_addr learned my addr 172.21.15.86:0/418912189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.937+0000 7fa986a00640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d6b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98f400640 1 -- 172.21.15.86:0/418912189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 msgr2=0x559d41d6b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98f400640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41d6b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98f400640 1 -- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 msgr2=0x559d41d6b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98f400640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d6b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98f400640 1 -- 172.21.15.86:0/418912189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d41aa7620 con 0x559d41d629b0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa986a00640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d6b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98fe00640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41d6b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa98f400640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d6aad0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa980000c00 tx=0x7fa980003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1747f0f48b520bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa97d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa97d600640 1 -- 172.21.15.86:0/418912189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98001f040 con 0x559d41d629b0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa97d600640 1 -- 172.21.15.86:0/418912189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7fa980005050 con 0x559d41d629b0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.938+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9800043d0 con 0x559d41d629b0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa97d600640 1 -- 172.21.15.86:0/418912189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa980032050 con 0x559d41d629b0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 10 client.16080 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d41d7c9d0 con 0x559d41d629b0 2024-04-25T19:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa97d600640 1 -- 172.21.15.86:0/418912189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9800043d0 con 0x559d41d629b0 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa97d600640 1 -- 172.21.15.86:0/418912189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa980018070 con 0x559d41d629b0 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 20 client.16080 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41d7dd80 0x559d41d80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 4 client.16080 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.939+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d41cdc8b0 con 0x559d41d7dd80 2024-04-25T19:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.940+0000 7fa986a00640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41d7dd80 0x559d41d80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.941+0000 7fa986a00640 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41d7dd80 0x559d41d80170 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14be5df6691ca3af server_cookie=d3203d4606aff149 in_seq=0 out_seq=0 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.941+0000 7fa97d600640 10 client.16080 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa97d600640 1 -- 172.21.15.86:0/418912189 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d41cdc8b0 con 0x559d41d7dd80 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa97d600640 10 client.16080 handle_command_reply: tid=0 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 client.16080 shutdown 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 msgr2=0x559d41d6aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d6aad0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa980000c00 tx=0x7fa980003820 comp rx=0 tx=0).stop 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 shutdown_connections 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41d7dd80 0x559d41d80170 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41d6b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d6b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 --2- 172.21.15.86:0/418912189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d6aad0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa97d600640 0 client.16080 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 >> 172.21.15.86:0/418912189 conn(0x559d41d5eec0 msgr2=0x559d41d64bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 shutdown_connections 2024-04-25T19:10:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 1 -- 172.21.15.86:0/418912189 wait complete. 2024-04-25T19:10:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.942+0000 7fa999238740 20 client.16080 trim_cache size 0 max 16384 2024-04-25T19:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.958+0000 7fa999238740 1 Processor -- start 2024-04-25T19:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.958+0000 7fa999238740 1 -- start start 2024-04-25T19:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa999238740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d629b0 0x559d41d79680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa999238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d632d0 0x559d41d858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa999238740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 0x559d41d85e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa999238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b4210 con 0x559d41d632d0 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa999238740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b5d50 con 0x559d41d6db90 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa999238740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b48e0 con 0x559d41d629b0 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 0x559d41d85e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa986a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d632d0 0x559d41d858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 0x559d41d85e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:52604/0 (socket says 172.21.15.86:52604) 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa98fe00640 1 -- 172.21.15.86:0/1541683815 learned_addr learned my addr 172.21.15.86:0/1541683815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.959+0000 7fa98f400640 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d629b0 0x559d41d79680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa98fe00640 1 -- 172.21.15.86:0/1541683815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d629b0 msgr2=0x559d41d79680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa98fe00640 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d629b0 0x559d41d79680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa98fe00640 1 -- 172.21.15.86:0/1541683815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d632d0 msgr2=0x559d41d858e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa98fe00640 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d632d0 0x559d41d858e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa98fe00640 1 -- 172.21.15.86:0/1541683815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d41d63c40 con 0x559d41d6db90 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa986a00640 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d632d0 0x559d41d858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa98fe00640 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 0x559d41d85e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa98809cc60 tx=0x7fa98800cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58248581aae17e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa97fe00640 1 -- 172.21.15.86:0/1541683815 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98809ccd0 con 0x559d41d6db90 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa97fe00640 1 -- 172.21.15.86:0/1541683815 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9880a1080 con 0x559d41d6db90 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.960+0000 7fa97fe00640 1 -- 172.21.15.86:0/1541683815 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98809d190 con 0x559d41d6db90 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 -- 172.21.15.86:0/1541683815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 msgr2=0x559d41d85e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 0x559d41d85e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa98809cc60 tx=0x7fa98800cee0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 -- 172.21.15.86:0/1541683815 shutdown_connections 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d6db90 0x559d41d85e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d632d0 0x559d41d858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 --2- 172.21.15.86:0/1541683815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d629b0 0x559d41d79680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 -- 172.21.15.86:0/1541683815 >> 172.21.15.86:0/1541683815 conn(0x559d41d5eec0 msgr2=0x559d41d65dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 -- 172.21.15.86:0/1541683815 shutdown_connections 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.961+0000 7fa999238740 1 -- 172.21.15.86:0/1541683815 wait complete. 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 -- start start 2024-04-25T19:10:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 Processor -- start 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d72b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41de2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b5680 con 0x559d41d629b0 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b4fb0 con 0x559d41d632d0 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901b2210 con 0x559d41d6db90 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa999238740 10 client.0 _ll_register_callbacks cb 0x7fa980009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa98f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d72b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa98f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d72b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42448/0 (socket says 172.21.15.86:42448) 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.962+0000 7fa98f400640 1 -- 172.21.15.86:0/1445283272 learned_addr learned my addr 172.21.15.86:0/1445283272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98fe00640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41de2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa986a00640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98f400640 1 -- 172.21.15.86:0/1445283272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 msgr2=0x559d41de2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98f400640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41de2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98f400640 1 -- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 msgr2=0x559d41d730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98f400640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98f400640 1 -- 172.21.15.86:0/1445283272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d41de2690 con 0x559d41d629b0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa98f400640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d72b60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa980004cf0 tx=0x7fa980009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1134ba12f47824b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa97d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa97d600640 1 -- 172.21.15.86:0/1445283272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980015070 con 0x559d41d629b0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9880a1080 con 0x559d41d629b0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.963+0000 7fa97d600640 1 -- 172.21.15.86:0/1445283272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7fa980004650 con 0x559d41d629b0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa97d600640 1 -- 172.21.15.86:0/1445283272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa980004850 con 0x559d41d629b0 2024-04-25T19:10:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 10 client.16086 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d41de3840 con 0x559d41d629b0 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa97d600640 1 -- 172.21.15.86:0/1445283272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9880a1080 con 0x559d41d629b0 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa97d600640 1 -- 172.21.15.86:0/1445283272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa980018040 con 0x559d41d629b0 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 10 client.16086 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 20 client.16086 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41de3f50 0x559d41de6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 4 client.16086 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.964+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d41cd6320 con 0x559d41de3f50 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.965+0000 7fa986a00640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41de3f50 0x559d41de6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.965+0000 7fa986a00640 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41de3f50 0x559d41de6340 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d127fe6becc3b30 server_cookie=6fa67bb9f771ac89 in_seq=0 out_seq=0 2024-04-25T19:10:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.965+0000 7fa97d600640 10 client.16086 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa97d600640 1 -- 172.21.15.86:0/1445283272 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d41cd6320 con 0x559d41de3f50 2024-04-25T19:10:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa97d600640 10 client.16086 handle_command_reply: tid=0 2024-04-25T19:10:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 client.16086 shutdown 2024-04-25T19:10:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 msgr2=0x559d41d72b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d72b60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa980004cf0 tx=0x7fa980009850 comp rx=0 tx=0).stop 2024-04-25T19:10:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 shutdown_connections 2024-04-25T19:10:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d41de3f50 0x559d41de6340 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d41d6db90 0x559d41de2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa97d600640 0 client.16086 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d41d632d0 0x559d41d730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 --2- 172.21.15.86:0/1445283272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d41d629b0 0x559d41d72b60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.979+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 >> 172.21.15.86:0/1445283272 conn(0x559d41d5eec0 msgr2=0x559d41d5d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.980+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 shutdown_connections 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.980+0000 7fa999238740 1 -- 172.21.15.86:0/1445283272 wait complete. 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.980+0000 7fa999238740 20 client.16086 trim_cache size 0 max 16384 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:10:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:10:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:10:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9740a3d10 msgr2=0x7fa9740a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9740a3d10 0x7fa9740a61d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa978009900 tx=0x7fa978000bc0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 msgr2=0x7fa99019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 0x7fa99019e970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa98800eb10 tx=0x7fa98800c2d0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 shutdown_connections 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa9740a3d10 0x7fa9740a61d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99010e420 0x7fa99019e970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa990108e10 0x7fa9901a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 --2- 172.21.15.86:0/3232164366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa990108490 0x7fa9901a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 >> 172.21.15.86:0/3232164366 conn(0x7fa990077d30 msgr2=0x7fa990078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 shutdown_connections 2024-04-25T19:10:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:41.981+0000 7fa994e00640 1 -- 172.21.15.86:0/3232164366 wait complete. 2024-04-25T19:10:42.351 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-04-25T19:10:42.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:10:42.804 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:42 smithi086 ceph-mon[27783]: pgmap v239: 97 pgs: 97 active+clean; 5.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-25T19:10:42.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:42 smithi183 ceph-mon[32344]: pgmap v239: 97 pgs: 97 active+clean; 5.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-25T19:10:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:42 smithi194 ceph-mon[32740]: pgmap v239: 97 pgs: 97 active+clean; 5.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-25T19:10:44.836 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:44 smithi086 ceph-mon[27783]: pgmap v240: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 67 KiB/s wr, 25 op/s 2024-04-25T19:10:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:44 smithi183 ceph-mon[32344]: pgmap v240: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 67 KiB/s wr, 25 op/s 2024-04-25T19:10:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:44 smithi194 ceph-mon[32740]: pgmap v240: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 67 KiB/s wr, 25 op/s 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 -- 172.21.15.86:0/2254462244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 msgr2=0x7f3a74111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 --2- 172.21.15.86:0/2254462244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 0x7f3a74111830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009f90 tx=0x7f3a6c00b330 comp rx=0 tx=0).stop 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 -- 172.21.15.86:0/2254462244 shutdown_connections 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 --2- 172.21.15.86:0/2254462244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 0x7f3a74111830 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 --2- 172.21.15.86:0/2254462244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a74075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 --2- 172.21.15.86:0/2254462244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 0x7f3a7410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 -- 172.21.15.86:0/2254462244 >> 172.21.15.86:0/2254462244 conn(0x7f3a74079ba0 msgr2=0x7f3a74079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 -- 172.21.15.86:0/2254462244 shutdown_connections 2024-04-25T19:10:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.115+0000 7f3a7c000700 1 -- 172.21.15.86:0/2254462244 wait complete. 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.116+0000 7f3a7c000700 1 Processor -- start 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.116+0000 7f3a7c000700 1 -- start start 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a7419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7c000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 0x7f3a7419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7c000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 0x7f3a7419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74114e50 con 0x7f3a74074c70 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7c000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74114cd0 con 0x7f3a7410f440 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7c000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74114fd0 con 0x7f3a74075590 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a79600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a7419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a79600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a7419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42480/0 (socket says 172.21.15.86:42480) 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a79600700 1 -- 172.21.15.86:0/1839425226 learned_addr learned my addr 172.21.15.86:0/1839425226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a78c00700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 0x7f3a7419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.117+0000 7f3a7a000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 0x7f3a7419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a79600700 1 -- 172.21.15.86:0/1839425226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 msgr2=0x7f3a7419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a79600700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 0x7f3a7419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a79600700 1 -- 172.21.15.86:0/1839425226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 msgr2=0x7f3a7419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a79600700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 0x7f3a7419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a79600700 1 -- 172.21.15.86:0/1839425226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a741a02f0 con 0x7f3a74074c70 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a78c00700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 0x7f3a7419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a79600700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a7419b2e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800ec10 tx=0x7f3a6800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800cc90 con 0x7f3a74074c70 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a6800ce30 con 0x7f3a74074c70 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68010430 con 0x7f3a74074c70 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.118+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a741a05e0 con 0x7f3a74074c70 2024-04-25T19:10:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.119+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a741a7e60 con 0x7f3a74074c70 2024-04-25T19:10:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.120+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a74069730 con 0x7f3a74074c70 2024-04-25T19:10:45.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.121+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a68004000 con 0x7f3a74074c70 2024-04-25T19:10:45.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.123+0000 7f3a72000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a5c0a3e30 0x7f3a5c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.123+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7f3a680d0ab0 con 0x7f3a74074c70 2024-04-25T19:10:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.123+0000 7f3a78c00700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a5c0a3e30 0x7f3a5c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:45.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.124+0000 7f3a78c00700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a5c0a3e30 0x7f3a5c0a62f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a74111040 tx=0x7f3a60009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:45.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.128+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a6809d080 con 0x7f3a74074c70 2024-04-25T19:10:45.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.403+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a7402df80 con 0x7f3a74074c70 2024-04-25T19:10:45.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.405+0000 7f3a72000700 1 -- 172.21.15.86:0/1839425226 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f3a6809c980 con 0x7f3a74074c70 2024-04-25T19:10:45.409 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:10:45.409 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:10:45.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a5c0a3e30 msgr2=0x7f3a5c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:45.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a5c0a3e30 0x7f3a5c0a62f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a74111040 tx=0x7f3a60009260 comp rx=0 tx=0).stop 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 msgr2=0x7f3a7419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a7419b2e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800ec10 tx=0x7f3a6800ef20 comp rx=0 tx=0).stop 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 shutdown_connections 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a7410f440 0x7f3a7419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a74074c70 0x7f3a7419b2e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3a5c0a3e30 0x7f3a5c0a62f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 --2- 172.21.15.86:0/1839425226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a74075590 0x7f3a7419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 >> 172.21.15.86:0/1839425226 conn(0x7f3a74079ba0 msgr2=0x7f3a740ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 shutdown_connections 2024-04-25T19:10:45.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:45.409+0000 7f3a7c000700 1 -- 172.21.15.86:0/1839425226 wait complete. 2024-04-25T19:10:45.413 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:10:45.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:45 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1839425226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:45.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:45 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1839425226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:45.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:45 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1839425226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:46.224 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 86.8 --await-for 42.0' 2024-04-25T19:10:46.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 86.8 --await-for 42.0 2024-04-25T19:10:46.672 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:46.848 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:46 smithi086 ceph-mon[27783]: pgmap v241: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:10:46.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:46 smithi183 ceph-mon[32344]: pgmap v241: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:10:46.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:46 smithi194 ceph-mon[32740]: pgmap v241: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:10:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:48 smithi086 ceph-mon[27783]: pgmap v242: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:10:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:48 smithi183 ceph-mon[32344]: pgmap v242: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:10:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:48 smithi194 ceph-mon[32740]: pgmap v242: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:10:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 -- 172.21.15.86:0/334575215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f01092d0 msgr2=0x7ff0f01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 --2- 172.21.15.86:0/334575215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f01092d0 0x7ff0f01096b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40099c0 tx=0x7ff0e4009cd0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 -- 172.21.15.86:0/334575215 shutdown_connections 2024-04-25T19:10:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 --2- 172.21.15.86:0/334575215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f010e3b0 0x7ff0f0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 --2- 172.21.15.86:0/334575215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f01092d0 0x7ff0f01096b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 --2- 172.21.15.86:0/334575215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 -- 172.21.15.86:0/334575215 >> 172.21.15.86:0/334575215 conn(0x7ff0f0076210 msgr2=0x7ff0f0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 -- 172.21.15.86:0/334575215 shutdown_connections 2024-04-25T19:10:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.001+0000 7ff0f8000700 1 -- 172.21.15.86:0/334575215 wait complete. 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.002+0000 7ff0f8000700 1 Processor -- start 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.002+0000 7ff0f8000700 1 -- start start 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f8000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f01092d0 0x7ff0f019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f8000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f8000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f010e3b0 0x7ff0f01a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f8000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01192c0 con 0x7ff0f0109c80 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f8000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0119140 con 0x7ff0f01092d0 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f8000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0119440 con 0x7ff0f010e3b0 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f5600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f01092d0 0x7ff0f019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f019fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42496/0 (socket says 172.21.15.86:42496) 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f4c00700 1 -- 172.21.15.86:0/1229422140 learned_addr learned my addr 172.21.15.86:0/1229422140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.003+0000 7ff0f6000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f010e3b0 0x7ff0f01a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f5600700 1 -- 172.21.15.86:0/1229422140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f010e3b0 msgr2=0x7ff0f01a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f5600700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f010e3b0 0x7ff0f01a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f5600700 1 -- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 msgr2=0x7ff0f019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f5600700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f5600700 1 -- 172.21.15.86:0/1229422140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01a46a0 con 0x7ff0f01092d0 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0f5600700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f01092d0 0x7ff0f019f690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40098e0 tx=0x7ff0e40055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.004+0000 7ff0ee000700 1 -- 172.21.15.86:0/1229422140 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4026070 con 0x7ff0f01092d0 2024-04-25T19:10:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.005+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01a4930 con 0x7ff0f01092d0 2024-04-25T19:10:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.005+0000 7ff0ee000700 1 -- 172.21.15.86:0/1229422140 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e400f460 con 0x7ff0f01092d0 2024-04-25T19:10:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.005+0000 7ff0ee000700 1 -- 172.21.15.86:0/1229422140 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4009e60 con 0x7ff0f01092d0 2024-04-25T19:10:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.005+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01ac210 con 0x7ff0f01092d0 2024-04-25T19:10:49.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.008+0000 7ff0ee000700 1 -- 172.21.15.86:0/1229422140 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0e4029070 con 0x7ff0f01092d0 2024-04-25T19:10:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 Processor -- start 2024-04-25T19:10:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 -- start start 2024-04-25T19:10:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fb9c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b08faef020 con 0x55b08fba7560 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b08fa958c0 con 0x55b08fb9c6f0 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b08faeafe0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.010+0000 7ff0ee000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff0d80a3e50 0x7ff0d80a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fb9c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0ee000700 1 -- 172.21.15.86:0/1229422140 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7ff0e40d0fe0 con 0x7ff0f01092d0 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fb9c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48950/0 (socket says 172.21.15.86:48950) 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42510/0 (socket says 172.21.15.86:42510) 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f6000700 1 -- 172.21.15.86:0/2078546938 learned_addr learned my addr 172.21.15.86:0/2078546938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:49.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f6000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff0d80a3e50 0x7ff0d80a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f5600700 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 -- 172.21.15.86:0/2078546938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 msgr2=0x55b08fb9c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fb9c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 -- 172.21.15.86:0/2078546938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 msgr2=0x55b08fba7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.011+0000 7ff0f4c00700 1 -- 172.21.15.86:0/2078546938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08fa8b810 con 0x55b08fba7560 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.012+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba9950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005c50 tx=0x7ff0dc010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958a5925059e18a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.012+0000 7ff0f6000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff0d80a3e50 0x7ff0d80a6310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009fd0 tx=0x7ff0e800b040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.012+0000 7ff0e2a00700 1 -- 172.21.15.86:0/2078546938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc011440 con 0x55b08fba7560 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.012+0000 7ff0e2a00700 1 -- 172.21.15.86:0/2078546938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0dc011a90 con 0x55b08fba7560 2024-04-25T19:10:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.012+0000 7ff0e2a00700 1 -- 172.21.15.86:0/2078546938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc019a10 con 0x55b08fba7560 2024-04-25T19:10:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 -- 172.21.15.86:0/2078546938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 msgr2=0x55b08fba9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba9950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005c50 tx=0x7ff0dc010730 comp rx=0 tx=0).stop 2024-04-25T19:10:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 -- 172.21.15.86:0/2078546938 shutdown_connections 2024-04-25T19:10:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba9950 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fb9c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/2078546938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 -- 172.21.15.86:0/2078546938 >> 172.21.15.86:0/2078546938 conn(0x55b08facb0e0 msgr2=0x55b08fb982e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 -- 172.21.15.86:0/2078546938 shutdown_connections 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.013+0000 7ff1028b0b80 1 -- 172.21.15.86:0/2078546938 wait complete. 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.014+0000 7ff1028b0b80 1 -- start start 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.014+0000 7ff1028b0b80 1 Processor -- start 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fba6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b08fbaa050 con 0x55b08fba7560 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b08faa89f0 con 0x55b08fb9c6f0 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b08fbaa1d0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff1028b0b80 10 client.0 _ll_register_callbacks cb 0x55b08fa878b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fba6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fba6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48952/0 (socket says 172.21.15.86:48952) 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 -- 172.21.15.86:0/4261771188 learned_addr learned my addr 172.21.15.86:0/4261771188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 -- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 msgr2=0x55b08fba3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 -- 172.21.15.86:0/4261771188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 msgr2=0x55b08fba4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.015+0000 7ff0f5600700 1 -- 172.21.15.86:0/4261771188 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08fa3ca60 con 0x55b08fb9bdd0 2024-04-25T19:10:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff0f5600700 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fba6c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff0e409cef0 tx=0x7ff0e409caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ad756f5d3d6e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff0d7400700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff0d7400700 1 -- 172.21.15.86:0/4261771188 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e409d660 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff0d7400700 1 -- 172.21.15.86:0/4261771188 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7ff0e40094b0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0dc011a90 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff0d7400700 1 -- 172.21.15.86:0/4261771188 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0e409cf60 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff1028b0b80 10 client.26194 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b08fa85c80 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.016+0000 7ff0d7400700 1 -- 172.21.15.86:0/4261771188 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0dc011a90 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff0d7400700 1 -- 172.21.15.86:0/4261771188 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e40b6260 con 0x55b08fb9bdd0 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff1028b0b80 10 client.26194 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff1028b0b80 10 client.26194 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:10:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff1028b0b80 20 client.26194 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fbb6ca0 0x55b08fbb9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff1028b0b80 4 client.26194 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:10:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b08fafc810 con 0x55b08fbb6ca0 2024-04-25T19:10:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.017+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fbb6ca0 0x55b08fbb9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.018+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fbb6ca0 0x55b08fbb9120 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be25ebc15e9cfaab server_cookie=b75e1c5aa9b98c96 in_seq=0 out_seq=0 2024-04-25T19:10:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.018+0000 7ff0d7400700 10 client.26194 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff0d7400700 1 -- 172.21.15.86:0/4261771188 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b08fafc810 con 0x55b08fbb6ca0 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff0d7400700 10 client.26194 handle_command_reply: tid=0 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 client.26194 shutdown 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 msgr2=0x55b08fba6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fba6c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff0e409cef0 tx=0x7ff0e409caa0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 shutdown_connections 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fbb6ca0 0x55b08fbb9120 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fba7560 0x55b08fba4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fba6c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff0d7400700 0 client.26194 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/4261771188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fba3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.019+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 >> 172.21.15.86:0/4261771188 conn(0x55b08facb0e0 msgr2=0x55b08fb97e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.020+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 shutdown_connections 2024-04-25T19:10:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.020+0000 7ff1028b0b80 1 -- 172.21.15.86:0/4261771188 wait complete. 2024-04-25T19:10:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.020+0000 7ff1028b0b80 20 client.26194 trim_cache size 0 max 16384 2024-04-25T19:10:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.041+0000 7ff1028b0b80 1 Processor -- start 2024-04-25T19:10:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 -- start start 2024-04-25T19:10:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9bdd0 0x55b08fbb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 0x55b08fbaf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b38a0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b53e0 con 0x55b08fb9c6f0 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b3f70 con 0x55b08fba7560 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff0f6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 0x55b08fbaf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff0f5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9bdd0 0x55b08fbb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff0f6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 0x55b08fbaf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48966/0 (socket says 172.21.15.86:48966) 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff0f6000700 1 -- 172.21.15.86:0/684168921 learned_addr learned my addr 172.21.15.86:0/684168921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.042+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0f6000700 1 -- 172.21.15.86:0/684168921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 msgr2=0x55b08fbaf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0f6000700 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0f6000700 1 -- 172.21.15.86:0/684168921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9bdd0 msgr2=0x55b08fbb2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0f6000700 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9bdd0 0x55b08fbb2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0f6000700 1 -- 172.21.15.86:0/684168921 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b08fb9dc40 con 0x55b08fba7560 2024-04-25T19:10:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0f6000700 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 0x55b08fbaf9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8008fd0 tx=0x7ff0e8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=395df6932a577f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0e2000700 1 -- 172.21.15.86:0/684168921 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8004bb0 con 0x55b08fba7560 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0e2000700 1 -- 172.21.15.86:0/684168921 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e8004d50 con 0x55b08fba7560 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.043+0000 7ff0e2000700 1 -- 172.21.15.86:0/684168921 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e801b450 con 0x55b08fba7560 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 -- 172.21.15.86:0/684168921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 msgr2=0x55b08fbaf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 0x55b08fbaf9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8008fd0 tx=0x7ff0e8009920 comp rx=0 tx=0).stop 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 -- 172.21.15.86:0/684168921 shutdown_connections 2024-04-25T19:10:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fba7560 0x55b08fbaf9f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9bdd0 0x55b08fbb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/684168921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 -- 172.21.15.86:0/684168921 >> 172.21.15.86:0/684168921 conn(0x55b08facb0e0 msgr2=0x55b08fb97490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 -- 172.21.15.86:0/684168921 shutdown_connections 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.044+0000 7ff1028b0b80 1 -- 172.21.15.86:0/684168921 wait complete. 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.045+0000 7ff1028b0b80 1 -- start start 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.045+0000 7ff1028b0b80 1 Processor -- start 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.045+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fbaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.045+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.045+0000 7ff1028b0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fba7560 0x55b08fbaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.045+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b4d10 con 0x55b08fb9c6f0 2024-04-25T19:10:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b4640 con 0x55b08fba7560 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff1028b0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f01b1860 con 0x55b08fb9bdd0 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff1028b0b80 10 client.0 _ll_register_callbacks cb 0x55b08fa878b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42538/0 (socket says 172.21.15.86:42538) 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f4c00700 1 -- 172.21.15.86:0/499135143 learned_addr learned my addr 172.21.15.86:0/499135143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f5600700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fbaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f6000700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fba7560 0x55b08fbaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f5600700 1 -- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fba7560 msgr2=0x55b08fbaa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f5600700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fba7560 0x55b08fbaa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f5600700 1 -- 172.21.15.86:0/499135143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 msgr2=0x55b08fbaf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f5600700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f5600700 1 -- 172.21.15.86:0/499135143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b08fbaadb0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f6000700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fba7560 0x55b08fbaa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.046+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0f5600700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fbaf120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40993c0 tx=0x7ff0e4009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc964c29ec6e50c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0d7400700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0d7400700 1 -- 172.21.15.86:0/499135143 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40a58b0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0d7400700 1 -- 172.21.15.86:0/499135143 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7ff0e409c3b0 con 0x55b08fb9bdd0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e8004d50 con 0x55b08fb9bdd0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0d7400700 1 -- 172.21.15.86:0/499135143 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0e409d660 con 0x55b08fb9bdd0 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff1028b0b80 10 client.26200 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b08fc1ec70 con 0x55b08fb9bdd0 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0d7400700 1 -- 172.21.15.86:0/499135143 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e8004d50 con 0x55b08fb9bdd0 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.047+0000 7ff0d7400700 1 -- 172.21.15.86:0/499135143 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e40b6060 con 0x55b08fb9bdd0 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff1028b0b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff1028b0b80 10 client.26200 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff1028b0b80 20 client.26200 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fc1f1b0 0x55b08fc215a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff1028b0b80 4 client.26200 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.8, "await_for": 42.0}] 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.8, "await_for": 42.0}) -- 0x55b08faf68b0 con 0x55b08fc1f1b0 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.048+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fc1f1b0 0x55b08fc215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.049+0000 7ff0f4c00700 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fc1f1b0 0x55b08fc215a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d933cc244b01f6 server_cookie=dc4c95f25ee1e20f in_seq=0 out_seq=0 2024-04-25T19:10:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.049+0000 7ff0d7400700 10 client.26200 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.063+0000 7ff0d7400700 1 -- 172.21.15.86:0/499135143 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b08faf68b0 con 0x55b08fc1f1b0 2024-04-25T19:10:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.063+0000 7ff0d7400700 10 client.26200 handle_command_reply: tid=0 2024-04-25T19:10:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.063+0000 7ff1028b0b80 1 client.26200 shutdown 2024-04-25T19:10:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 msgr2=0x55b08fbaf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fbaf120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40993c0 tx=0x7ff0e4009290 comp rx=0 tx=0).stop 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 shutdown_connections 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b08fc1f1b0 0x55b08fc215a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b08fba7560 0x55b08fbaa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b08fb9bdd0 0x55b08fbaf120 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 --2- 172.21.15.86:0/499135143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b08fb9c6f0 0x55b08fbaf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff0d7400700 0 client.26200 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 >> 172.21.15.86:0/499135143 conn(0x55b08facb0e0 msgr2=0x55b08fb97d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 shutdown_connections 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 1 -- 172.21.15.86:0/499135143 wait complete. 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.064+0000 7ff1028b0b80 20 client.26200 trim_cache size 0 max 16384 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 22, 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "9afd9dba": { 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 22, 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:10:49.068 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 86.8, 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:10:49.069 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:10:49.070 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:10:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff0d80a3e50 msgr2=0x7ff0d80a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff0d80a3e50 0x7ff0d80a6310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009fd0 tx=0x7ff0e800b040 comp rx=0 tx=0).stop 2024-04-25T19:10:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f01092d0 msgr2=0x7ff0f019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f01092d0 0x7ff0f019f690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40098e0 tx=0x7ff0e40055f0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 shutdown_connections 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff0d80a3e50 0x7ff0d80a6310 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f010e3b0 0x7ff0f01a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f01092d0 0x7ff0f019f690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 --2- 172.21.15.86:0/1229422140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f0109c80 0x7ff0f019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 >> 172.21.15.86:0/1229422140 conn(0x7ff0f0076210 msgr2=0x7ff0f00776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 shutdown_connections 2024-04-25T19:10:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:10:49.065+0000 7ff0f8000700 1 -- 172.21.15.86:0/1229422140 wait complete. 2024-04-25T19:10:50.017 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9afd9dba', quiesce took 7.7 seconds. Will release after: 26.8 2024-04-25T19:10:50.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:50 smithi183 ceph-mon[32344]: pgmap v243: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T19:10:50.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:50 smithi194 ceph-mon[32740]: pgmap v243: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T19:10:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:50 smithi086 ceph-mon[27783]: pgmap v243: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T19:10:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:52 smithi194 ceph-mon[32740]: pgmap v244: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:10:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:52 smithi086 ceph-mon[27783]: pgmap v244: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:10:53.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:52 smithi183 ceph-mon[32344]: pgmap v244: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:10:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:54 smithi194 ceph-mon[32740]: pgmap v245: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-25T19:10:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:54 smithi086 ceph-mon[27783]: pgmap v245: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-25T19:10:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 -- 172.21.15.86:0/3643659757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 msgr2=0x7fd01c111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 --2- 172.21.15.86:0/3643659757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 0x7fd01c111060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd014009ca0 tx=0x7fd01402a8b0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 -- 172.21.15.86:0/3643659757 shutdown_connections 2024-04-25T19:10:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 --2- 172.21.15.86:0/3643659757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 0x7fd01c111060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 --2- 172.21.15.86:0/3643659757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd01c104d70 0x7fd01c108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 --2- 172.21.15.86:0/3643659757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 -- 172.21.15.86:0/3643659757 >> 172.21.15.86:0/3643659757 conn(0x7fd01c0ffc20 msgr2=0x7fd01c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 -- 172.21.15.86:0/3643659757 shutdown_connections 2024-04-25T19:10:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.126+0000 7fd022e00640 1 -- 172.21.15.86:0/3643659757 wait complete. 2024-04-25T19:10:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.127+0000 7fd022e00640 1 Processor -- start 2024-04-25T19:10:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 -- start start 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 0x7fd01c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd01c19a6e0 0x7fd01c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c114400 con 0x7fd01c1043c0 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c114280 con 0x7fd01c19a6e0 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd022e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c114580 con 0x7fd01c109490 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd01be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd01be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35294/0 (socket says 172.21.15.86:35294) 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd01be00640 1 -- 172.21.15.86:0/3414064378 learned_addr learned my addr 172.21.15.86:0/3414064378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd01b400640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 0x7fd01c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.128+0000 7fd020e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd01c19a6e0 0x7fd01c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd01be00640 1 -- 172.21.15.86:0/3414064378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 msgr2=0x7fd01c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd01be00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 0x7fd01c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd01be00640 1 -- 172.21.15.86:0/3414064378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd01c19a6e0 msgr2=0x7fd01c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd01be00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd01c19a6e0 0x7fd01c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd01be00640 1 -- 172.21.15.86:0/3414064378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c19b2e0 con 0x7fd01c1043c0 2024-04-25T19:10:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd01be00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1a0620 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00d950 tx=0x7fd00c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd018c00640 1 -- 172.21.15.86:0/3414064378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c017070 con 0x7fd01c1043c0 2024-04-25T19:10:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd018c00640 1 -- 172.21.15.86:0/3414064378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c009c30 con 0x7fd01c1043c0 2024-04-25T19:10:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c1a73c0 con 0x7fd01c1043c0 2024-04-25T19:10:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.129+0000 7fd018c00640 1 -- 172.21.15.86:0/3414064378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00f620 con 0x7fd01c1043c0 2024-04-25T19:10:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.130+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1a77c0 con 0x7fd01c1043c0 2024-04-25T19:10:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.133+0000 7fd025c23740 1 Processor -- start 2024-04-25T19:10:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.133+0000 7fd018c00640 1 -- 172.21.15.86:0/3414064378 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd00c010460 con 0x7fd01c1043c0 2024-04-25T19:10:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.133+0000 7fd025c23740 1 -- start start 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.133+0000 7fd025c23740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x562457464c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.133+0000 7fd025c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.133+0000 7fd025c23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x562457471df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd025c23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5624573ca900 con 0x56245746fa00 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd025c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562457370ef0 con 0x562457465140 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd025c23740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5624573c68c0 con 0x562457464820 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd01b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x562457471df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd01be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd020e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x562457464c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd01b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x562457471df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35310/0 (socket says 172.21.15.86:35310) 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd01be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:42468/0 (socket says 172.21.15.86:42468) 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd01b400640 1 -- 172.21.15.86:0/2967325527 learned_addr learned my addr 172.21.15.86:0/2967325527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd020e00640 1 -- 172.21.15.86:0/2967325527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 msgr2=0x56245746f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd020e00640 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd020e00640 1 -- 172.21.15.86:0/2967325527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 msgr2=0x562457471df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd020e00640 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x562457471df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.134+0000 7fd020e00640 1 -- 172.21.15.86:0/2967325527 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624570dd210 con 0x562457464820 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd018c00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0000a3dc0 0x7fd0000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd020e00640 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x562457464c00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd014005e30 tx=0x7fd014030f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b272e885e2368cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd018c00640 1 -- 172.21.15.86:0/3414064378 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7fd00c0d0420 con 0x7fd01c1043c0 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd01be00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0000a3dc0 0x7fd0000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd011600640 1 -- 172.21.15.86:0/2967325527 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0140305a0 con 0x562457464820 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd011600640 1 -- 172.21.15.86:0/2967325527 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd014036030 con 0x562457464820 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.135+0000 7fd011600640 1 -- 172.21.15.86:0/2967325527 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0140042a0 con 0x562457464820 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 -- 172.21.15.86:0/2967325527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 msgr2=0x562457464c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd01be00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0000a3dc0 0x7fd0000a6280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0cdbb0 tx=0x7fd00c0d3f70 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x562457464c00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd014005e30 tx=0x7fd014030f50 comp rx=0 tx=0).stop 2024-04-25T19:10:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 -- 172.21.15.86:0/2967325527 shutdown_connections 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x562457471df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 --2- 172.21.15.86:0/2967325527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x562457464c00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 -- 172.21.15.86:0/2967325527 >> 172.21.15.86:0/2967325527 conn(0x562457460d30 msgr2=0x562457461110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 -- 172.21.15.86:0/2967325527 shutdown_connections 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.136+0000 7fd025c23740 1 -- 172.21.15.86:0/2967325527 wait complete. 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 -- start start 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 Processor -- start 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x56245746c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x56245746d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5624574725d0 con 0x56245746fa00 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562457384310 con 0x562457465140 2024-04-25T19:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.137+0000 7fd025c23740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562457472750 con 0x562457464820 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd025c23740 10 client.0 _ll_register_callbacks cb 0x562457352cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd020e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x56245746d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd01b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd020e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x56245746d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35324/0 (socket says 172.21.15.86:35324) 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd020e00640 1 -- 172.21.15.86:0/3429841725 learned_addr learned my addr 172.21.15.86:0/3429841725 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd020e00640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x56245746c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd01b400640 1 -- 172.21.15.86:0/3429841725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 msgr2=0x56245746c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd01b400640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x56245746c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd01b400640 1 -- 172.21.15.86:0/3429841725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 msgr2=0x56245746d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd01b400640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x56245746d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd01b400640 1 -- 172.21.15.86:0/3429841725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624571a9620 con 0x562457465140 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.138+0000 7fd020e00640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x56245746c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd01b400640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746ce90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x562457461ec0 tx=0x7fd008012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c1b8227f393b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd006000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:10:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd006000640 1 -- 172.21.15.86:0/3429841725 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004020 con 0x562457465140 2024-04-25T19:10:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd014036030 con 0x562457465140 2024-04-25T19:10:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd006000640 1 -- 172.21.15.86:0/3429841725 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7fd0080041c0 con 0x562457465140 2024-04-25T19:10:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd006000640 1 -- 172.21.15.86:0/3429841725 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0080043c0 con 0x562457465140 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd025c23740 10 client.26219 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.139+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56245747e8d0 con 0x562457465140 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd006000640 1 -- 172.21.15.86:0/3429841725 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd014036030 con 0x562457465140 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd006000640 1 -- 172.21.15.86:0/3429841725 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd008003970 con 0x562457465140 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd025c23740 10 client.26219 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd025c23740 10 client.26219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd025c23740 20 client.26219 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd025c23740 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56245747fde0 0x5624574821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd025c23740 4 client.26219 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.140+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624573de8b0 con 0x56245747fde0 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.141+0000 7fd020e00640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56245747fde0 0x5624574821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.141+0000 7fd020e00640 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56245747fde0 0x5624574821d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3619e11261a841a4 server_cookie=dc0f783a870222a0 in_seq=0 out_seq=0 2024-04-25T19:10:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.141+0000 7fd006000640 10 client.26219 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd006000640 1 -- 172.21.15.86:0/3429841725 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624573de8b0 con 0x56245747fde0 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd006000640 10 client.26219 handle_command_reply: tid=0 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 client.26219 shutdown 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 msgr2=0x56245746ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746ce90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x562457461ec0 tx=0x7fd008012b60 comp rx=0 tx=0).stop 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 shutdown_connections 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56245747fde0 0x5624574821d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56245746fa00 0x56245746d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x56245746ce90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd006000640 0 client.26219 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 --2- 172.21.15.86:0/3429841725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562457464820 0x56245746c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.142+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 >> 172.21.15.86:0/3429841725 conn(0x562457460d30 msgr2=0x562457466a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.143+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 shutdown_connections 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.143+0000 7fd025c23740 1 -- 172.21.15.86:0/3429841725 wait complete. 2024-04-25T19:10:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.143+0000 7fd025c23740 20 client.26219 trim_cache size 0 max 16384 2024-04-25T19:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 Processor -- start 2024-04-25T19:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 -- start start 2024-04-25T19:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457464820 0x56245747b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 0x562457487ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457488020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1afe60 con 0x562457465140 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1b19a0 con 0x562457464820 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.159+0000 7fd025c23740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1b0530 con 0x56245746fa00 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 0x562457487ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 0x562457487ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35334/0 (socket says 172.21.15.86:35334) 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 -- 172.21.15.86:0/922940336 learned_addr learned my addr 172.21.15.86:0/922940336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd020e00640 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457488020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd020e00640 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457464820 0x56245747b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 -- 172.21.15.86:0/922940336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 msgr2=0x562457488020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457488020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 -- 172.21.15.86:0/922940336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457464820 msgr2=0x56245747b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457464820 0x56245747b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd01b400640 1 -- 172.21.15.86:0/922940336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562457482540 con 0x562457465140 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.160+0000 7fd020e00640 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457488020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd01b400640 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 0x562457487ae0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd008009d00 tx=0x7fd008005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02b4f4a161a8b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd010c00640 1 -- 172.21.15.86:0/922940336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004000 con 0x562457465140 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd010c00640 1 -- 172.21.15.86:0/922940336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0080041a0 con 0x562457465140 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd010c00640 1 -- 172.21.15.86:0/922940336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00801c470 con 0x562457465140 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 -- 172.21.15.86:0/922940336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 msgr2=0x562457487ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 0x562457487ae0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd008009d00 tx=0x7fd008005870 comp rx=0 tx=0).stop 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 -- 172.21.15.86:0/922940336 shutdown_connections 2024-04-25T19:10:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457488020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457465140 0x562457487ae0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 --2- 172.21.15.86:0/922940336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457464820 0x56245747b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.161+0000 7fd025c23740 1 -- 172.21.15.86:0/922940336 >> 172.21.15.86:0/922940336 conn(0x562457460d30 msgr2=0x562457461110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.162+0000 7fd025c23740 1 -- 172.21.15.86:0/922940336 shutdown_connections 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.162+0000 7fd025c23740 1 -- 172.21.15.86:0/922940336 wait complete. 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.162+0000 7fd025c23740 1 -- start start 2024-04-25T19:10:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.162+0000 7fd025c23740 1 Processor -- start 2024-04-25T19:10:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 0x562457475460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x5624574759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457475ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1b12d0 con 0x562457464820 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1b0c00 con 0x562457465140 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ade60 con 0x56245746fa00 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd025c23740 10 client.0 _ll_register_callbacks cb 0x7fd008009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 0x562457475460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 0x562457475460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35350/0 (socket says 172.21.15.86:35350) 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 -- 172.21.15.86:0/3614765297 learned_addr learned my addr 172.21.15.86:0/3614765297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd01b400640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x5624574759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457475ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 -- 172.21.15.86:0/3614765297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 msgr2=0x562457475ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457475ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 -- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 msgr2=0x5624574759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x5624574759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd020e00640 1 -- 172.21.15.86:0/3614765297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624574e53e0 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.163+0000 7fd01b400640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x5624574759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd020e00640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 0x562457475460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd014009d90 tx=0x7fd014058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe428149878d509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd006000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd006000640 1 -- 172.21.15.86:0/3614765297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014030a00 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd006000640 1 -- 172.21.15.86:0/3614765297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8637+0+0 (secure 0 0 0) 0x7fd014030ba0 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0080041a0 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd006000640 1 -- 172.21.15.86:0/3614765297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd014004040 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd025c23740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624574e66d0 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.164+0000 7fd006000640 1 -- 172.21.15.86:0/3614765297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0080041a0 con 0x562457464820 2024-04-25T19:10:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd006000640 1 -- 172.21.15.86:0/3614765297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd014043070 con 0x562457464820 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd025c23740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd025c23740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd025c23740 20 client.16128 populate_metadata read hostname 'smithi086' 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd025c23740 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624574e6a10 0x5624574e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd025c23740 4 client.16128 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624573d8320 con 0x5624574e6a10 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.165+0000 7fd01b400640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624574e6a10 0x5624574e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.166+0000 7fd01b400640 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624574e6a10 0x5624574e8e00 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc549676f9a91fa server_cookie=1993794b5bb30e36 in_seq=0 out_seq=0 2024-04-25T19:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.166+0000 7fd006000640 10 client.16128 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:55.171 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:54 smithi183 ceph-mon[32344]: pgmap v245: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd006000640 1 -- 172.21.15.86:0/3614765297 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624573d8320 con 0x5624574e6a10 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd006000640 10 client.16128 handle_command_reply: tid=0 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 client.16128 shutdown 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 msgr2=0x562457475460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 0x562457475460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd014009d90 tx=0x7fd014058040 comp rx=0 tx=0).stop 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 shutdown_connections 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624574e6a10 0x5624574e8e00 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56245746fa00 0x562457475ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562457465140 0x5624574759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 --2- 172.21.15.86:0/3614765297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562457464820 0x562457475460 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 >> 172.21.15.86:0/3614765297 conn(0x562457460d30 msgr2=0x56245745f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd006000640 0 client.16128 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:10:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 shutdown_connections 2024-04-25T19:10:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 1 -- 172.21.15.86:0/3614765297 wait complete. 2024-04-25T19:10:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.180+0000 7fd025c23740 20 client.16128 trim_cache size 0 max 16384 2024-04-25T19:10:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:10:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:10:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:10:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:10:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0000a3dc0 msgr2=0x7fd0000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0000a3dc0 0x7fd0000a6280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0cdbb0 tx=0x7fd00c0d3f70 comp rx=0 tx=0).stop 2024-04-25T19:10:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 msgr2=0x7fd01c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1a0620 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00d950 tx=0x7fd00c00de10 comp rx=0 tx=0).stop 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 shutdown_connections 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0000a3dc0 0x7fd0000a6280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd01c19a6e0 0x7fd01c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd01c109490 0x7fd01c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 --2- 172.21.15.86:0/3414064378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd01c1043c0 0x7fd01c1a0620 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.181+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 >> 172.21.15.86:0/3414064378 conn(0x7fd01c0ffc20 msgr2=0x7fd01c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.182+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 shutdown_connections 2024-04-25T19:10:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:10:55.182+0000 7fd022e00640 1 -- 172.21.15.86:0/3414064378 wait complete. 2024-04-25T19:10:56.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:56 smithi194 ceph-mon[32740]: pgmap v246: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T19:10:57.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:56 smithi086 ceph-mon[27783]: pgmap v246: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T19:10:57.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:56 smithi183 ceph-mon[32344]: pgmap v246: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T19:10:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:10:58 smithi194 ceph-mon[32740]: pgmap v247: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T19:10:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:10:58 smithi086 ceph-mon[27783]: pgmap v247: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T19:10:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:10:58 smithi183 ceph-mon[32344]: pgmap v247: 97 pgs: 97 active+clean; 1.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-04-25T19:11:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:00 smithi194 ceph-mon[32740]: pgmap v248: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-25T19:11:01.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:00 smithi086 ceph-mon[27783]: pgmap v248: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-25T19:11:01.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:00 smithi183 ceph-mon[32344]: pgmap v248: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-25T19:11:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:02 smithi194 ceph-mon[32740]: pgmap v249: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-25T19:11:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:02 smithi086 ceph-mon[27783]: pgmap v249: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-25T19:11:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:02 smithi183 ceph-mon[32344]: pgmap v249: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-25T19:11:04.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:04 smithi194 ceph-mon[32740]: pgmap v250: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-25T19:11:05.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:04 smithi086 ceph-mon[27783]: pgmap v250: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-25T19:11:05.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:04 smithi183 ceph-mon[32344]: pgmap v250: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-25T19:11:06.969 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:06 smithi183 ceph-mon[32344]: pgmap v251: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-25T19:11:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:06 smithi194 ceph-mon[32740]: pgmap v251: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-25T19:11:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:06 smithi086 ceph-mon[27783]: pgmap v251: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-25T19:11:07.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:07 smithi194 ceph-mon[32740]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T19:11:08.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:07 smithi086 ceph-mon[27783]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T19:11:08.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:07 smithi183 ceph-mon[32344]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T19:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 -- 172.21.15.86:0/3231123373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec102f10 msgr2=0x7f5dec1032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 --2- 172.21.15.86:0/3231123373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec102f10 0x7f5dec1032f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009900 tx=0x7f5de002a480 comp rx=0 tx=0).stop 2024-04-25T19:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 -- 172.21.15.86:0/3231123373 shutdown_connections 2024-04-25T19:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 --2- 172.21.15.86:0/3231123373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec10c330 0x7f5dec10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 --2- 172.21.15.86:0/3231123373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec103830 0x7f5dec10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 --2- 172.21.15.86:0/3231123373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec102f10 0x7f5dec1032f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 -- 172.21.15.86:0/3231123373 >> 172.21.15.86:0/3231123373 conn(0x7f5dec0fc770 msgr2=0x7f5dec0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 -- 172.21.15.86:0/3231123373 shutdown_connections 2024-04-25T19:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.324+0000 7f5df2a00640 1 -- 172.21.15.86:0/3231123373 wait complete. 2024-04-25T19:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.325+0000 7f5df2a00640 1 Processor -- start 2024-04-25T19:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.325+0000 7f5df2a00640 1 -- start start 2024-04-25T19:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 0x7f5dec1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec103830 0x7f5dec1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec10c330 0x7f5dec19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112010 con 0x7f5dec103830 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec111e90 con 0x7f5dec102f10 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112190 con 0x7f5dec10c330 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 0x7f5dec1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 0x7f5dec1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46098/0 (socket says 172.21.15.86:46098) 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df1600640 1 -- 172.21.15.86:0/1565715336 learned_addr learned my addr 172.21.15.86:0/1565715336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df2000640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec10c330 0x7f5dec19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.326+0000 7f5df0c00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec103830 0x7f5dec1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df1600640 1 -- 172.21.15.86:0/1565715336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec10c330 msgr2=0x7f5dec19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df1600640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec10c330 0x7f5dec19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df1600640 1 -- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec103830 msgr2=0x7f5dec1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df1600640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec103830 0x7f5dec1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df1600640 1 -- 172.21.15.86:0/1565715336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19ad30 con 0x7f5dec102f10 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df2000640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec10c330 0x7f5dec19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df1600640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 0x7f5dec1a0400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005ab0 tx=0x7f5de0003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5dea000640 1 -- 172.21.15.86:0/1565715336 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de003f070 con 0x7f5dec102f10 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19afc0 con 0x7f5dec102f10 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5dea000640 1 -- 172.21.15.86:0/1565715336 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de0031d60 con 0x7f5dec102f10 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.327+0000 7f5dea000640 1 -- 172.21.15.86:0/1565715336 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0035030 con 0x7f5dec102f10 2024-04-25T19:11:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.328+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a72c0 con 0x7f5dec102f10 2024-04-25T19:11:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 Processor -- start 2024-04-25T19:11:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 -- start start 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 0x55cb5242cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242d2d0 0x55cb52437650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52439f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb52392900 con 0x55cb5242d2d0 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb52338ef0 con 0x55cb5242c9b0 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5df6e13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb5238e8c0 con 0x55cb52437b90 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.332+0000 7f5dea000640 1 -- 172.21.15.86:0/1565715336 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5de0042070 con 0x7f5dec102f10 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 0x55cb5242cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 0x55cb5242cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46114/0 (socket says 172.21.15.86:46114) 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 -- 172.21.15.86:0/279670012 learned_addr learned my addr 172.21.15.86:0/279670012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df0c00640 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52439f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df1600640 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242d2d0 0x55cb52437650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 -- 172.21.15.86:0/279670012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 msgr2=0x55cb52439f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52439f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 -- 172.21.15.86:0/279670012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242d2d0 msgr2=0x55cb52437650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242d2d0 0x55cb52437650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.333+0000 7f5df2000640 1 -- 172.21.15.86:0/279670012 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb520a5210 con 0x55cb5242c9b0 2024-04-25T19:11:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5df2000640 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 0x55cb5242cd90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5de4011790 tx=0x7f5de4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f9487f5c283e9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5dde600640 1 -- 172.21.15.86:0/279670012 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4003c30 con 0x55cb5242c9b0 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5dde600640 1 -- 172.21.15.86:0/279670012 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de400fe50 con 0x55cb5242c9b0 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5dde600640 1 -- 172.21.15.86:0/279670012 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40193e0 con 0x55cb5242c9b0 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5dea000640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5dd40a3d10 0x7f5dd40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5dea000640 1 -- 172.21.15.86:0/1565715336 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8681+0+0 (secure 0 0 0) 0x7f5de00f10e0 con 0x7f5dec102f10 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5df6e13740 1 -- 172.21.15.86:0/279670012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 msgr2=0x55cb5242cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5df6e13740 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 0x55cb5242cd90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5de4011790 tx=0x7f5de4011c50 comp rx=0 tx=0).stop 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.334+0000 7f5df1600640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5dd40a3d10 0x7f5dd40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 -- 172.21.15.86:0/279670012 shutdown_connections 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52439f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242d2d0 0x55cb52437650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 --2- 172.21.15.86:0/279670012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242c9b0 0x55cb5242cd90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 -- 172.21.15.86:0/279670012 >> 172.21.15.86:0/279670012 conn(0x55cb52428ec0 msgr2=0x55cb524292a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 -- 172.21.15.86:0/279670012 shutdown_connections 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 -- 172.21.15.86:0/279670012 wait complete. 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df1600640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5dd40a3d10 0x7f5dd40a61d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5de00ee840 tx=0x7f5de0009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.335+0000 7f5df6e13740 1 -- start start 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 Processor -- start 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242c9b0 0x55cb52434b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb52435040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52435580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb5243a7c0 con 0x55cb5242c9b0 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb5234c310 con 0x55cb5242d2d0 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb5243a940 con 0x55cb52437b90 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df6e13740 10 client.0 _ll_register_callbacks cb 0x55cb5231acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242c9b0 0x55cb52434b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242c9b0 0x55cb52434b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57620/0 (socket says 172.21.15.86:57620) 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df2000640 1 -- 172.21.15.86:0/4277608932 learned_addr learned my addr 172.21.15.86:0/4277608932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.336+0000 7f5df0c00640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb52435040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df2000640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52435580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df0c00640 1 -- 172.21.15.86:0/4277608932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 msgr2=0x55cb52435580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df0c00640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52435580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df0c00640 1 -- 172.21.15.86:0/4277608932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242c9b0 msgr2=0x55cb52434b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df0c00640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242c9b0 0x55cb52434b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df0c00640 1 -- 172.21.15.86:0/4277608932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb52171620 con 0x55cb5242d2d0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5df0c00640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb52435040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55cb5242a050 tx=0x7f5dd8012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d990183e20126d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5dd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.337+0000 7f5dd2a00640 1 -- 172.21.15.86:0/4277608932 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80107e0 con 0x55cb5242d2d0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de400fe50 con 0x55cb5242d2d0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5dd2a00640 1 -- 172.21.15.86:0/4277608932 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8681+0+0 (secure 0 0 0) 0x7f5dd8010980 con 0x55cb5242d2d0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5dd2a00640 1 -- 172.21.15.86:0/4277608932 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dd8010b80 con 0x55cb5242d2d0 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 10 client.26237 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb52446a60 con 0x55cb5242d2d0 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5dd2a00640 1 -- 172.21.15.86:0/4277608932 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de400fe50 con 0x55cb5242d2d0 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5dd2a00640 1 -- 172.21.15.86:0/4277608932 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5dd8019070 con 0x55cb5242d2d0 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 10 client.26237 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 10 client.26237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 20 client.26237 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb52447ed0 0x55cb5244a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 4 client.26237 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.338+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb523a68b0 con 0x55cb52447ed0 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.339+0000 7f5df2000640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb52447ed0 0x55cb5244a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.339+0000 7f5df2000640 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb52447ed0 0x55cb5244a300 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e154e8b195c8486 server_cookie=8c282fc194102dbe in_seq=0 out_seq=0 2024-04-25T19:11:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.339+0000 7f5dd2a00640 10 client.26237 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5dd2a00640 1 -- 172.21.15.86:0/4277608932 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb523a68b0 con 0x55cb52447ed0 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5dd2a00640 10 client.26237 handle_command_reply: tid=0 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 client.26237 shutdown 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 msgr2=0x55cb52435040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb52435040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55cb5242a050 tx=0x7f5dd8012820 comp rx=0 tx=0).stop 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 shutdown_connections 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb52447ed0 0x55cb5244a300 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb52437b90 0x55cb52435580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb52435040 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5dd2a00640 0 client.26237 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 --2- 172.21.15.86:0/4277608932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5242c9b0 0x55cb52434b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 >> 172.21.15.86:0/4277608932 conn(0x55cb52428ec0 msgr2=0x55cb5242ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.340+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 shutdown_connections 2024-04-25T19:11:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.341+0000 7f5df6e13740 1 -- 172.21.15.86:0/4277608932 wait complete. 2024-04-25T19:11:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.341+0000 7f5df6e13740 20 client.26237 trim_cache size 0 max 16384 2024-04-25T19:11:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 Processor -- start 2024-04-25T19:11:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 -- start start 2024-04-25T19:11:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb52443680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb524351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5244fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1afe80 con 0x55cb52437b90 2024-04-25T19:11:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1b19c0 con 0x55cb5242d2d0 2024-04-25T19:11:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df6e13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1b0550 con 0x55cb5242c9b0 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5244fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5244fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57632/0 (socket says 172.21.15.86:57632) 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df2000640 1 -- 172.21.15.86:0/944358221 learned_addr learned my addr 172.21.15.86:0/944358221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.357+0000 7f5df2000640 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb52443680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df0c00640 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb524351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df2000640 1 -- 172.21.15.86:0/944358221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 msgr2=0x55cb52443680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df2000640 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb52443680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df2000640 1 -- 172.21.15.86:0/944358221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 msgr2=0x55cb524351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df2000640 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb524351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df2000640 1 -- 172.21.15.86:0/944358221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5244a980 con 0x55cb52437b90 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5df2000640 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5244fc10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55cb52429950 tx=0x7f5de4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501f7804c384993b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5dddc00640 1 -- 172.21.15.86:0/944358221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de401f040 con 0x55cb52437b90 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5dddc00640 1 -- 172.21.15.86:0/944358221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de400fd30 con 0x55cb52437b90 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.358+0000 7f5dddc00640 1 -- 172.21.15.86:0/944358221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4003c90 con 0x55cb52437b90 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 -- 172.21.15.86:0/944358221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 msgr2=0x55cb5244fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5244fc10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55cb52429950 tx=0x7f5de4004570 comp rx=0 tx=0).stop 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 -- 172.21.15.86:0/944358221 shutdown_connections 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5244fc10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb524351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 --2- 172.21.15.86:0/944358221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb52443680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 -- 172.21.15.86:0/944358221 >> 172.21.15.86:0/944358221 conn(0x55cb52428ec0 msgr2=0x55cb52434ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 -- 172.21.15.86:0/944358221 shutdown_connections 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 -- 172.21.15.86:0/944358221 wait complete. 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.359+0000 7f5df6e13740 1 -- start start 2024-04-25T19:11:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 Processor -- start 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb524400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb5243d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5243db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1b12f0 con 0x55cb52437b90 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1b0c20 con 0x55cb5242d2d0 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1ade80 con 0x55cb5242c9b0 2024-04-25T19:11:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df6e13740 10 client.0 _ll_register_callbacks cb 0x7f5dd800ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5243db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5243db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57646/0 (socket says 172.21.15.86:57646) 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df2000640 1 -- 172.21.15.86:0/2818790173 learned_addr learned my addr 172.21.15.86:0/2818790173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df0c00640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb5243d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.360+0000 7f5df2000640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb524400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df2000640 1 -- 172.21.15.86:0/2818790173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 msgr2=0x55cb524400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df2000640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb524400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df2000640 1 -- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 msgr2=0x55cb5243d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df2000640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb5243d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df2000640 1 -- 172.21.15.86:0/2818790173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb52439f80 con 0x55cb52437b90 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df0c00640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb5243d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df2000640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5243db70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5de401c040 tx=0x7f5de40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5810cb960e0238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5dd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5dd2a00640 1 -- 172.21.15.86:0/2818790173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4004860 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5dd2a00640 1 -- 172.21.15.86:0/2818790173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8681+0+0 (secure 0 0 0) 0x7f5de4004a00 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.361+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de400fd30 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5dd2a00640 1 -- 172.21.15.86:0/2818790173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5de402e040 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb524ae450 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5dd2a00640 1 -- 172.21.15.86:0/2818790173 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de400fd30 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5dd2a00640 1 -- 172.21.15.86:0/2818790173 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5de4025040 con 0x55cb52437b90 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 20 client.16146 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb524ae800 0x55cb524b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 4 client.16146 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.362+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb523a0320 con 0x55cb524ae800 2024-04-25T19:11:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.363+0000 7f5df0c00640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb524ae800 0x55cb524b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.363+0000 7f5df0c00640 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb524ae800 0x55cb524b0c20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10749ba8ed5ccc62 server_cookie=bde31c839b409a20 in_seq=0 out_seq=0 2024-04-25T19:11:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.363+0000 7f5dd2a00640 10 client.16146 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.375+0000 7f5dd2a00640 1 -- 172.21.15.86:0/2818790173 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb523a0320 con 0x55cb524ae800 2024-04-25T19:11:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.375+0000 7f5dd2a00640 10 client.16146 handle_command_reply: tid=0 2024-04-25T19:11:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.375+0000 7f5df6e13740 1 client.16146 shutdown 2024-04-25T19:11:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 msgr2=0x55cb5243db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5243db70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5de401c040 tx=0x7f5de40046d0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 shutdown_connections 2024-04-25T19:11:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb524ae800 0x55cb524b0c20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb52437b90 0x55cb5243db70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb5242d2d0 0x55cb5243d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 --2- 172.21.15.86:0/2818790173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb5242c9b0 0x55cb524400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 >> 172.21.15.86:0/2818790173 conn(0x55cb52428ec0 msgr2=0x55cb524352e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5dd2a00640 0 client.16146 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 shutdown_connections 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 1 -- 172.21.15.86:0/2818790173 wait complete. 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.376+0000 7f5df6e13740 20 client.16146 trim_cache size 0 max 16384 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:11:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5dd40a3d10 msgr2=0x7f5dd40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5dd40a3d10 0x7f5dd40a61d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5de00ee840 tx=0x7f5de0009260 comp rx=0 tx=0).stop 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 msgr2=0x7f5dec1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 0x7f5dec1a0400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005ab0 tx=0x7f5de0003c80 comp rx=0 tx=0).stop 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 shutdown_connections 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5dd40a3d10 0x7f5dd40a61d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dec10c330 0x7f5dec19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec103830 0x7f5dec1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 --2- 172.21.15.86:0/1565715336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dec102f10 0x7f5dec1a0400 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 >> 172.21.15.86:0/1565715336 conn(0x7f5dec0fc770 msgr2=0x7f5dec101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 shutdown_connections 2024-04-25T19:11:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:08.377+0000 7f5df2a00640 1 -- 172.21.15.86:0/1565715336 wait complete. 2024-04-25T19:11:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:08 smithi194 ceph-mon[32740]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 34 op/s 2024-04-25T19:11:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:08 smithi086 ceph-mon[27783]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 34 op/s 2024-04-25T19:11:09.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:08 smithi183 ceph-mon[32344]: pgmap v253: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 34 op/s 2024-04-25T19:11:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:10 smithi194 ceph-mon[32740]: pgmap v254: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-25T19:11:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:10 smithi086 ceph-mon[27783]: pgmap v254: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-25T19:11:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:10 smithi183 ceph-mon[32344]: pgmap v254: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-25T19:11:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:12 smithi086 ceph-mon[27783]: pgmap v255: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-25T19:11:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:12 smithi183 ceph-mon[32344]: pgmap v255: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-25T19:11:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:12 smithi194 ceph-mon[32740]: pgmap v255: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-25T19:11:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:14 smithi194 ceph-mon[32740]: pgmap v256: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:11:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:14 smithi194 ceph-mon[32740]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T19:11:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:14 smithi086 ceph-mon[27783]: pgmap v256: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:11:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:14 smithi086 ceph-mon[27783]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T19:11:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:14 smithi183 ceph-mon[32344]: pgmap v256: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:11:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:14 smithi183 ceph-mon[32344]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T19:11:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:16 smithi194 ceph-mon[32740]: pgmap v258: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T19:11:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:16 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:11:16.818 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:11:16.827 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:16 smithi086 ceph-mon[27783]: pgmap v258: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T19:11:16.827 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:16 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:11:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:16 smithi183 ceph-mon[32344]: pgmap v258: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-04-25T19:11:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:16 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:11:17.257 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:11:18.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:18 smithi086 ceph-mon[27783]: pgmap v259: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:11:18.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:18 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:18.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:18 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:18 smithi183 ceph-mon[32344]: pgmap v259: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:11:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:18 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:18.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:18 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:18.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:18 smithi194 ceph-mon[32740]: pgmap v259: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:11:18.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:18 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:18.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:18 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:19.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:19 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:11:19.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:19 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:11:19.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:19 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:19.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:19 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:11:19.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:19 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:11:19.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:19 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 -- 172.21.15.86:0/3887945241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40103320 msgr2=0x7f4d4010f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 --2- 172.21.15.86:0/3887945241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40103320 0x7f4d4010f830 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800b0a0 tx=0x7f4d3800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 -- 172.21.15.86:0/3887945241 shutdown_connections 2024-04-25T19:11:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 --2- 172.21.15.86:0/3887945241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40103320 0x7f4d4010f830 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 --2- 172.21.15.86:0/3887945241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 0x7f4d40108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 --2- 172.21.15.86:0/3887945241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40102960 0x7f4d40102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 -- 172.21.15.86:0/3887945241 >> 172.21.15.86:0/3887945241 conn(0x7f4d400fe490 msgr2=0x7f4d401008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:19.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 -- 172.21.15.86:0/3887945241 shutdown_connections 2024-04-25T19:11:19.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.458+0000 7f4d46200700 1 -- 172.21.15.86:0/3887945241 wait complete. 2024-04-25T19:11:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.459+0000 7f4d46200700 1 Processor -- start 2024-04-25T19:11:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 -- start start 2024-04-25T19:11:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 0x7f4d4019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40103320 0x7f4d4019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 0x7f4d4019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40112bc0 con 0x7f4d40102960 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40112a40 con 0x7f4d40103320 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d46200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40112d40 con 0x7f4d40108960 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d44e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 0x7f4d4019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d44e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 0x7f4d4019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59946/0 (socket says 172.21.15.86:59946) 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d44e00700 1 -- 172.21.15.86:0/361574577 learned_addr learned my addr 172.21.15.86:0/361574577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d45800700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 0x7f4d4019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.460+0000 7f4d37e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40103320 0x7f4d4019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d44e00700 1 -- 172.21.15.86:0/361574577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 msgr2=0x7f4d4019fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d44e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 0x7f4d4019fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d44e00700 1 -- 172.21.15.86:0/361574577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40103320 msgr2=0x7f4d4019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d44e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40103320 0x7f4d4019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d44e00700 1 -- 172.21.15.86:0/361574577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d401a0200 con 0x7f4d40102960 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d45800700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 0x7f4d4019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d37e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40103320 0x7f4d4019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.461+0000 7f4d44e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 0x7f4d4019b240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e9c0 tx=0x7f4d3000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.462+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000cbd0 con 0x7f4d40102960 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.462+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d3000ee60 con 0x7f4d40102960 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.462+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d401a04f0 con 0x7f4d40102960 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.462+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d300055c0 con 0x7f4d40102960 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.462+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d401a7d70 con 0x7f4d40102960 2024-04-25T19:11:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.463+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d40069730 con 0x7f4d40102960 2024-04-25T19:11:19.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.465+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d30004750 con 0x7f4d40102960 2024-04-25T19:11:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.466+0000 7f4d3e000700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4d240a3db0 0x7f4d240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.466+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f4d300d0ac0 con 0x7f4d40102960 2024-04-25T19:11:19.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.467+0000 7f4d37e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4d240a3db0 0x7f4d240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.467+0000 7f4d37e00700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4d240a3db0 0x7f4d240a6270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d4010fce0 tx=0x7f4d28009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.471+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d3009d090 con 0x7f4d40102960 2024-04-25T19:11:19.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:19 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:11:19.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:19 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:11:19.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:19 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:11:19.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.751+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d4019c550 con 0x7f4d40102960 2024-04-25T19:11:19.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.752+0000 7f4d3e000700 1 -- 172.21.15.86:0/361574577 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f4d3009c990 con 0x7f4d40102960 2024-04-25T19:11:19.754 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:11:19.754 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4d240a3db0 msgr2=0x7f4d240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4d240a3db0 0x7f4d240a6270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4d4010fce0 tx=0x7f4d28009260 comp rx=0 tx=0).stop 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 msgr2=0x7f4d4019b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 0x7f4d4019b240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e9c0 tx=0x7f4d3000ecd0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 shutdown_connections 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4d240a3db0 0x7f4d240a6270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d40108960 0x7f4d4019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d40102960 0x7f4d4019b240 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 --2- 172.21.15.86:0/361574577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d40103320 0x7f4d4019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:19.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.755+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 >> 172.21.15.86:0/361574577 conn(0x7f4d400fe490 msgr2=0x7f4d400fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:19.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.756+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 shutdown_connections 2024-04-25T19:11:19.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:19.756+0000 7f4d46200700 1 -- 172.21.15.86:0/361574577 wait complete. 2024-04-25T19:11:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:11:20.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:20 smithi086 ceph-mon[27783]: pgmap v260: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T19:11:20.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:20 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/361574577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:11:20.383 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:20 smithi194 ceph-mon[32740]: pgmap v260: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T19:11:20.383 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:20 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/361574577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:11:20.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:20 smithi183 ceph-mon[32344]: pgmap v260: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T19:11:20.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:20 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/361574577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:11:20.517 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 9afd9dba --release --await' 2024-04-25T19:11:20.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 9afd9dba --release --await 2024-04-25T19:11:20.975 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 -- 172.21.15.86:0/3740158338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28104d60 msgr2=0x7fcf28108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/3740158338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28104d60 0x7fcf28108da0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009860 tx=0x7fcf1402a340 comp rx=0 tx=0).stop 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 -- 172.21.15.86:0/3740158338 shutdown_connections 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/3740158338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf28109480 0x7fcf28111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/3740158338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28104d60 0x7fcf28108da0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/3740158338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf281043b0 0x7fcf28104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 -- 172.21.15.86:0/3740158338 >> 172.21.15.86:0/3740158338 conn(0x7fcf280ffc50 msgr2=0x7fcf28102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 -- 172.21.15.86:0/3740158338 shutdown_connections 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.521+0000 7fcf2ea00640 1 -- 172.21.15.86:0/3740158338 wait complete. 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.522+0000 7fcf2ea00640 1 Processor -- start 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.522+0000 7fcf2ea00640 1 -- start start 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 0x7fcf281a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28109480 0x7fcf281a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2819a720 0x7fcf2819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281143a0 con 0x7fcf281043b0 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28114220 con 0x7fcf2819a720 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28114520 con 0x7fcf28109480 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 0x7fcf281a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 0x7fcf281a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32780/0 (socket says 172.21.15.86:32780) 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 -- 172.21.15.86:0/2099522012 learned_addr learned my addr 172.21.15.86:0/2099522012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2819a720 0x7fcf2819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27400640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28109480 0x7fcf281a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 -- 172.21.15.86:0/2099522012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28109480 msgr2=0x7fcf281a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28109480 0x7fcf281a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 -- 172.21.15.86:0/2099522012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2819a720 msgr2=0x7fcf2819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2819a720 0x7fcf2819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.523+0000 7fcf27e00640 1 -- 172.21.15.86:0/2099522012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2819b320 con 0x7fcf281043b0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.524+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 0x7fcf281a0660 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9f0 tx=0x7fcf1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.524+0000 7fcf24c00640 1 -- 172.21.15.86:0/2099522012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800cd80 con 0x7fcf281043b0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.524+0000 7fcf24c00640 1 -- 172.21.15.86:0/2099522012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf180108e0 con 0x7fcf281043b0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.524+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf281a7400 con 0x7fcf281043b0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.524+0000 7fcf24c00640 1 -- 172.21.15.86:0/2099522012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18018980 con 0x7fcf281043b0 2024-04-25T19:11:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.524+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf281a77b0 con 0x7fcf281043b0 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.527+0000 7fcf3182d740 1 Processor -- start 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.527+0000 7fcf24c00640 1 -- 172.21.15.86:0/2099522012 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf18010430 con 0x7fcf281043b0 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.527+0000 7fcf3182d740 1 -- start start 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x560326457c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x5603264624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326462a00 0x560326464df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf3182d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603263bd900 con 0x560326457820 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf3182d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560326363ef0 con 0x560326458140 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf3182d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5603263b98c0 con 0x560326462a00 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x560326457c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x560326457c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32794/0 (socket says 172.21.15.86:32794) 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 -- 172.21.15.86:0/2455984542 learned_addr learned my addr 172.21.15.86:0/2455984542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf27400640 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326462a00 0x560326464df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x5603264624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 -- 172.21.15.86:0/2455984542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326462a00 msgr2=0x560326464df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326462a00 0x560326464df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 -- 172.21.15.86:0/2455984542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 msgr2=0x5603264624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x5603264624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf2ca00640 1 -- 172.21.15.86:0/2455984542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603260d0210 con 0x560326457820 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.528+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x5603264624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf24c00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcf0c0a3dc0 0x7fcf0c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf24c00640 1 -- 172.21.15.86:0/2099522012 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7fcf180d0b90 con 0x7fcf281043b0 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x560326457c00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf200099f0 tx=0x7fcf200095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310a910195426489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcf0c0a3dc0 0x7fcf0c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf1d200640 1 -- 172.21.15.86:0/2455984542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2001d070 con 0x560326457820 2024-04-25T19:11:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf1d200640 1 -- 172.21.15.86:0/2455984542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf20003e40 con 0x560326457820 2024-04-25T19:11:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.529+0000 7fcf1d200640 1 -- 172.21.15.86:0/2455984542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000da30 con 0x560326457820 2024-04-25T19:11:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 -- 172.21.15.86:0/2455984542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 msgr2=0x560326457c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf27e00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcf0c0a3dc0 0x7fcf0c0a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x560326458dc0 tx=0x7fcf18000bc0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x560326457c00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf200099f0 tx=0x7fcf200095c0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 -- 172.21.15.86:0/2455984542 shutdown_connections 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326462a00 0x560326464df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x5603264624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 --2- 172.21.15.86:0/2455984542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x560326457c00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 -- 172.21.15.86:0/2455984542 >> 172.21.15.86:0/2455984542 conn(0x560326453d30 msgr2=0x560326454110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 -- 172.21.15.86:0/2455984542 shutdown_connections 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.530+0000 7fcf3182d740 1 -- 172.21.15.86:0/2455984542 wait complete. 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 -- start start 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 Processor -- start 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x56032645f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x56032645fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x5603264603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560326465630 con 0x560326462a00 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560326377310 con 0x560326458140 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.531+0000 7fcf3182d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5603264657b0 con 0x560326457820 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf3182d740 10 client.0 _ll_register_callbacks cb 0x560326345cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x5603264603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf27400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x56032645fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x5603264603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32800/0 (socket says 172.21.15.86:32800) 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf2ca00640 1 -- 172.21.15.86:0/4032519400 learned_addr learned my addr 172.21.15.86:0/4032519400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x56032645f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf27400640 1 -- 172.21.15.86:0/4032519400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 msgr2=0x56032645f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf27400640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x56032645f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf27400640 1 -- 172.21.15.86:0/4032519400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 msgr2=0x5603264603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf27400640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x5603264603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf27400640 1 -- 172.21.15.86:0/4032519400 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032619c620 con 0x560326458140 2024-04-25T19:11:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.532+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x5603264603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf27400640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x56032645fe70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf14035040 tx=0x7fcf14030d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63d676e59b0ddb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf11600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf11600640 1 -- 172.21.15.86:0/4032519400 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14037040 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603263d18b0 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf11600640 1 -- 172.21.15.86:0/4032519400 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7fcf14004250 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf11600640 1 -- 172.21.15.86:0/4032519400 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf1404d050 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf3182d740 10 client.26261 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.533+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560326471880 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf11600640 1 -- 172.21.15.86:0/4032519400 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603263d18b0 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf11600640 1 -- 172.21.15.86:0/4032519400 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcf14003990 con 0x560326458140 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf3182d740 10 client.26261 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf3182d740 10 client.26261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf3182d740 20 client.26261 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf3182d740 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560326472d90 0x5603264751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf3182d740 4 client.26261 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:11:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603263cb320 con 0x560326472d90 2024-04-25T19:11:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.534+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560326472d90 0x5603264751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.535+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560326472d90 0x5603264751a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653226c66a583747 server_cookie=c1615fc6ba2b99ba in_seq=0 out_seq=0 2024-04-25T19:11:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.535+0000 7fcf11600640 10 client.26261 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf11600640 1 -- 172.21.15.86:0/4032519400 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603263cb320 con 0x560326472d90 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf11600640 10 client.26261 handle_command_reply: tid=0 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 client.26261 shutdown 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 msgr2=0x56032645fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x56032645fe70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf14035040 tx=0x7fcf14030d80 comp rx=0 tx=0).stop 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 shutdown_connections 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560326472d90 0x5603264751a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x5603264603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x56032645fe70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf11600640 0 client.26261 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 --2- 172.21.15.86:0/4032519400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x56032645f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 >> 172.21.15.86:0/4032519400 conn(0x560326453d30 msgr2=0x560326459ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 shutdown_connections 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 1 -- 172.21.15.86:0/4032519400 wait complete. 2024-04-25T19:11:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.536+0000 7fcf3182d740 20 client.26261 trim_cache size 0 max 16384 2024-04-25T19:11:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.552+0000 7fcf3182d740 1 Processor -- start 2024-04-25T19:11:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.552+0000 7fcf3182d740 1 -- start start 2024-04-25T19:11:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x56032646e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326458140 0x5603264601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326462a00 0x56032647aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf3182d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281afeb0 con 0x560326457820 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf3182d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281b19f0 con 0x560326462a00 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf3182d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281b0580 con 0x560326458140 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x56032646e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x56032646e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32802/0 (socket says 172.21.15.86:32802) 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf2ca00640 1 -- 172.21.15.86:0/3017293814 learned_addr learned my addr 172.21.15.86:0/3017293814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf27400640 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326458140 0x5603264601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.553+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326462a00 0x56032647aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf2ca00640 1 -- 172.21.15.86:0/3017293814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326458140 msgr2=0x5603264601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326458140 0x5603264601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf2ca00640 1 -- 172.21.15.86:0/3017293814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326462a00 msgr2=0x56032647aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326462a00 0x56032647aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf2ca00640 1 -- 172.21.15.86:0/3017293814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560326457f70 con 0x560326457820 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf27400640 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326458140 0x5603264601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x56032646e4f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000f80 tx=0x7fcf20010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9582321ad775923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf13e00640 1 -- 172.21.15.86:0/3017293814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2001d050 con 0x560326457820 2024-04-25T19:11:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf13e00640 1 -- 172.21.15.86:0/3017293814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2001e040 con 0x560326457820 2024-04-25T19:11:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.554+0000 7fcf13e00640 1 -- 172.21.15.86:0/3017293814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200047d0 con 0x560326457820 2024-04-25T19:11:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 -- 172.21.15.86:0/3017293814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 msgr2=0x56032646e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x56032646e4f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000f80 tx=0x7fcf20010040 comp rx=0 tx=0).stop 2024-04-25T19:11:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 -- 172.21.15.86:0/3017293814 shutdown_connections 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326462a00 0x56032647aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326458140 0x5603264601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 --2- 172.21.15.86:0/3017293814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326457820 0x56032646e4f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 -- 172.21.15.86:0/3017293814 >> 172.21.15.86:0/3017293814 conn(0x560326453d30 msgr2=0x560326454110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 -- 172.21.15.86:0/3017293814 shutdown_connections 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 -- 172.21.15.86:0/3017293814 wait complete. 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.555+0000 7fcf3182d740 1 -- start start 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 Processor -- start 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x5603264679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x560326467f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x560326468450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281b1320 con 0x560326462a00 2024-04-25T19:11:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281b0c50 con 0x560326458140 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf281adeb0 con 0x560326457820 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf3182d740 10 client.0 _ll_register_callbacks cb 0x7fcf140096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.556+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x560326468450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x560326468450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32816/0 (socket says 172.21.15.86:32816) 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 -- 172.21.15.86:0/371282996 learned_addr learned my addr 172.21.15.86:0/371282996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf27400640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x560326467f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x5603264679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 -- 172.21.15.86:0/371282996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 msgr2=0x5603264679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x5603264679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 -- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 msgr2=0x560326467f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x560326467f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 -- 172.21.15.86:0/371282996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603264dfb30 con 0x560326462a00 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf2ca00640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x560326468450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009620 tx=0x7fcf2000d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7e864c394f1b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf11600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:11:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf11600640 1 -- 172.21.15.86:0/371282996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2001d070 con 0x560326462a00 2024-04-25T19:11:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf11600640 1 -- 172.21.15.86:0/371282996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7fcf2000d9a0 con 0x560326462a00 2024-04-25T19:11:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.557+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2001e040 con 0x560326462a00 2024-04-25T19:11:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf11600640 1 -- 172.21.15.86:0/371282996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf2000dba0 con 0x560326462a00 2024-04-25T19:11:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:21.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603264e0e10 con 0x560326462a00 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf11600640 1 -- 172.21.15.86:0/371282996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2001e040 con 0x560326462a00 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf11600640 1 -- 172.21.15.86:0/371282996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcf2000b040 con 0x560326462a00 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 20 client.16170 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5603264e1370 0x5603264e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 4 client.16170 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.558+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603263c5190 con 0x5603264e1370 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.559+0000 7fcf27400640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5603264e1370 0x5603264e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.559+0000 7fcf27400640 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5603264e1370 0x5603264e3760 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93b10e524bacbf9 server_cookie=347df3b1a01cfef3 in_seq=0 out_seq=0 2024-04-25T19:11:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.559+0000 7fcf11600640 10 client.16170 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.569+0000 7fcf11600640 1 -- 172.21.15.86:0/371282996 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603263c5190 con 0x5603264e1370 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.569+0000 7fcf11600640 10 client.16170 handle_command_reply: tid=0 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.569+0000 7fcf3182d740 1 client.16170 shutdown 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 msgr2=0x560326468450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x560326468450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009620 tx=0x7fcf2000d5d0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 shutdown_connections 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5603264e1370 0x5603264e3760 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560326462a00 0x560326468450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560326458140 0x560326467f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 --2- 172.21.15.86:0/371282996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560326457820 0x5603264679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf11600640 0 client.16170 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 >> 172.21.15.86:0/371282996 conn(0x560326453d30 msgr2=0x560326452600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 shutdown_connections 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 1 -- 172.21.15.86:0/371282996 wait complete. 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.570+0000 7fcf3182d740 20 client.16170 trim_cache size 0 max 16384 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:11:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:11:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcf0c0a3dc0 msgr2=0x7fcf0c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcf0c0a3dc0 0x7fcf0c0a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x560326458dc0 tx=0x7fcf18000bc0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 msgr2=0x7fcf281a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 0x7fcf281a0660 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9f0 tx=0x7fcf1800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 shutdown_connections 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcf0c0a3dc0 0x7fcf0c0a6280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf2819a720 0x7fcf2819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf28109480 0x7fcf281a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 --2- 172.21.15.86:0/2099522012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf281043b0 0x7fcf281a0660 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.571+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 >> 172.21.15.86:0/2099522012 conn(0x7fcf280ffc50 msgr2=0x7fcf28100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.572+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 shutdown_connections 2024-04-25T19:11:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:21.572+0000 7fcf2ea00640 1 -- 172.21.15.86:0/2099522012 wait complete. 2024-04-25T19:11:22.319 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:22 smithi086 ceph-mon[27783]: pgmap v261: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T19:11:22.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:22 smithi183 ceph-mon[32344]: pgmap v261: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T19:11:22.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:22 smithi194 ceph-mon[32740]: pgmap v261: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T19:11:23.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 -- 172.21.15.86:0/443545103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 msgr2=0x7f880810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 --2- 172.21.15.86:0/443545103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f880810dcd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87f40098f0 tx=0x7f87f4009c00 comp rx=0 tx=0).stop 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 -- 172.21.15.86:0/443545103 shutdown_connections 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 --2- 172.21.15.86:0/443545103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880810e3b0 0x7f8808115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 --2- 172.21.15.86:0/443545103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88081092d0 0x7f88081096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 --2- 172.21.15.86:0/443545103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f880810dcd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 -- 172.21.15.86:0/443545103 >> 172.21.15.86:0/443545103 conn(0x7f8808076210 msgr2=0x7f8808107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 -- 172.21.15.86:0/443545103 shutdown_connections 2024-04-25T19:11:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.186+0000 7f880dc00700 1 -- 172.21.15.86:0/443545103 wait complete. 2024-04-25T19:11:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.187+0000 7f880dc00700 1 Processor -- start 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 -- start start 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 0x7f88081a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f88081a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880810e3b0 0x7f880819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8808119220 con 0x7f88081092d0 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88081190a0 con 0x7f8808109c80 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.188+0000 7f880dc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88081193a0 con 0x7f880810e3b0 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 0x7f88081a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 0x7f88081a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32828/0 (socket says 172.21.15.86:32828) 2024-04-25T19:11:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 -- 172.21.15.86:0/1368967162 learned_addr learned my addr 172.21.15.86:0/1368967162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8807400700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880810e3b0 0x7f880819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806000700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f88081a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 -- 172.21.15.86:0/1368967162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880810e3b0 msgr2=0x7f880819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880810e3b0 0x7f880819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 -- 172.21.15.86:0/1368967162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 msgr2=0x7f88081a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f88081a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806a00700 1 -- 172.21.15.86:0/1368967162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880819fd20 con 0x7f88081092d0 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8807400700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880810e3b0 0x7f880819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.189+0000 7f8806000700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f88081a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.190+0000 7f8806a00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 0x7f88081a5400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87f800ec10 tx=0x7f87f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.190+0000 7f87ff400700 1 -- 172.21.15.86:0/1368967162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800cc80 con 0x7f88081092d0 2024-04-25T19:11:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.190+0000 7f87ff400700 1 -- 172.21.15.86:0/1368967162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f800ce20 con 0x7f88081092d0 2024-04-25T19:11:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.190+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081a0010 con 0x7f88081092d0 2024-04-25T19:11:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.190+0000 7f87ff400700 1 -- 172.21.15.86:0/1368967162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8010430 con 0x7f88081092d0 2024-04-25T19:11:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.190+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081ac2d0 con 0x7f88081092d0 2024-04-25T19:11:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.193+0000 7f87ff400700 1 -- 172.21.15.86:0/1368967162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f87f8004000 con 0x7f88081092d0 2024-04-25T19:11:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 Processor -- start 2024-04-25T19:11:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f87ff400700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87ec0a3e30 0x7f87ec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 -- start start 2024-04-25T19:11:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f87ff400700 1 -- 172.21.15.86:0/1368967162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f87f80d0ab0 con 0x7f88081092d0 2024-04-25T19:11:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f8807400700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87ec0a3e30 0x7f87ec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb7041ef90 con 0x55eb704cc660 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb703c5830 con 0x55eb704cbd40 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.196+0000 7f88184dab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eb7041af50 con 0x55eb704d74d0 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32836/0 (socket says 172.21.15.86:32836) 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 -- 172.21.15.86:0/1228868130 learned_addr learned my addr 172.21.15.86:0/1228868130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806a00700 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8807400700 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 -- 172.21.15.86:0/1228868130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 msgr2=0x55eb704d98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 -- 172.21.15.86:0/1228868130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 msgr2=0x55eb704cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8806000700 1 -- 172.21.15.86:0/1228868130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb703bb780 con 0x55eb704cc660 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.197+0000 7f8807400700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87ec0a3e30 0x7f87ec0a62f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fd0 tx=0x7f8800009920 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.198+0000 7f8806000700 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d6f90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f880806c260 tx=0x7f87f4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86009c878c48eaaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.198+0000 7f87f3e00700 1 -- 172.21.15.86:0/1228868130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f400f660 con 0x55eb704cc660 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.198+0000 7f87f3e00700 1 -- 172.21.15.86:0/1228868130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f40054f0 con 0x55eb704cc660 2024-04-25T19:11:23.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.198+0000 7f87f3e00700 1 -- 172.21.15.86:0/1228868130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004c20 con 0x55eb704cc660 2024-04-25T19:11:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 -- 172.21.15.86:0/1228868130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 msgr2=0x55eb704d6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d6f90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f880806c260 tx=0x7f87f4005dc0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 -- 172.21.15.86:0/1228868130 shutdown_connections 2024-04-25T19:11:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 --2- 172.21.15.86:0/1228868130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d6f90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 -- 172.21.15.86:0/1228868130 >> 172.21.15.86:0/1228868130 conn(0x55eb703fb050 msgr2=0x55eb704c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 -- 172.21.15.86:0/1228868130 shutdown_connections 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.199+0000 7f88184dab80 1 -- 172.21.15.86:0/1228868130 wait complete. 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 -- start start 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 Processor -- start 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704d6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb704d9f60 con 0x55eb704cc660 2024-04-25T19:11:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb703d8960 con 0x55eb704cbd40 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.200+0000 7f88184dab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eb704da0e0 con 0x55eb704d74d0 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f88184dab80 10 client.0 _ll_register_callbacks cb 0x55eb703b7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32850/0 (socket says 172.21.15.86:32850) 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 -- 172.21.15.86:0/1081064915 learned_addr learned my addr 172.21.15.86:0/1081064915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8807400700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806a00700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704d6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 -- 172.21.15.86:0/1081064915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 msgr2=0x55eb704d4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 -- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 msgr2=0x55eb704d6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704d6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8806000700 1 -- 172.21.15.86:0/1081064915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7036c9d0 con 0x55eb704cc660 2024-04-25T19:11:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.201+0000 7f8807400700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.202+0000 7f8806000700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d3f30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87f400fb70 tx=0x7f87f4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b261f9e3333428c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.202+0000 7f87f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:11:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.202+0000 7f87f0c00700 1 -- 172.21.15.86:0/1081064915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4031070 con 0x55eb704cc660 2024-04-25T19:11:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.202+0000 7f87f0c00700 1 -- 172.21.15.86:0/1081064915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f87f4004460 con 0x55eb704cc660 2024-04-25T19:11:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.202+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f40054f0 con 0x55eb704cc660 2024-04-25T19:11:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.202+0000 7f87f0c00700 1 -- 172.21.15.86:0/1081064915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87f403d050 con 0x55eb704cc660 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 10 client.16182 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb703b5bf0 con 0x55eb704cc660 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f87f0c00700 1 -- 172.21.15.86:0/1081064915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f40054f0 con 0x55eb704cc660 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f87f0c00700 1 -- 172.21.15.86:0/1081064915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87f400b930 con 0x55eb704cc660 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 10 client.16182 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 20 client.16182 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb704e6af0 0x55eb704e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 4 client.16182 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:11:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.203+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7042c780 con 0x55eb704e6af0 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.204+0000 7f8806a00700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb704e6af0 0x55eb704e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.204+0000 7f8806a00700 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb704e6af0 0x55eb704e8ee0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddbfed2f5017886 server_cookie=aac2ca0409b12d81 in_seq=0 out_seq=0 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.204+0000 7f87f0c00700 10 client.16182 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.205+0000 7f87f0c00700 1 -- 172.21.15.86:0/1081064915 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb7042c780 con 0x55eb704e6af0 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.205+0000 7f87f0c00700 10 client.16182 handle_command_reply: tid=0 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.205+0000 7f88184dab80 1 client.16182 shutdown 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.205+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 msgr2=0x55eb704d3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.205+0000 7f88184dab80 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d3f30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87f400fb70 tx=0x7f87f4004710 comp rx=0 tx=0).stop 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 shutdown_connections 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb704e6af0 0x55eb704e8ee0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704d74d0 0x55eb704d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704cbd40 0x55eb704d6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f87f0c00700 0 client.16182 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 --2- 172.21.15.86:0/1081064915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704d3f30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 >> 172.21.15.86:0/1081064915 conn(0x55eb703fb050 msgr2=0x55eb704c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 shutdown_connections 2024-04-25T19:11:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 1 -- 172.21.15.86:0/1081064915 wait complete. 2024-04-25T19:11:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.206+0000 7f88184dab80 20 client.16182 trim_cache size 0 max 16384 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 Processor -- start 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 -- start start 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 0x55eb704c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cc660 0x55eb704e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b3a90 con 0x55eb704cbd40 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b55d0 con 0x55eb704d74d0 2024-04-25T19:11:23.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.227+0000 7f88184dab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b4160 con 0x55eb704cc660 2024-04-25T19:11:23.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 0x55eb704c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 0x55eb704c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32856/0 (socket says 172.21.15.86:32856) 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806a00700 1 -- 172.21.15.86:0/1886970853 learned_addr learned my addr 172.21.15.86:0/1886970853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cc660 0x55eb704e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8807400700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 -- 172.21.15.86:0/1886970853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 msgr2=0x55eb704e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 -- 172.21.15.86:0/1886970853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 msgr2=0x55eb704c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 0x55eb704c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 -- 172.21.15.86:0/1886970853 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb704cdbb0 con 0x55eb704cc660 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8807400700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806a00700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 0x55eb704c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.228+0000 7f8806000700 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cc660 0x55eb704e2a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87f400bd30 tx=0x7f87f40044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d2ee999714ec96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:23.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f87f3400700 1 -- 172.21.15.86:0/1886970853 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4031070 con 0x55eb704cc660 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f87f3400700 1 -- 172.21.15.86:0/1886970853 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f4005030 con 0x55eb704cc660 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f87f3400700 1 -- 172.21.15.86:0/1886970853 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f40049e0 con 0x55eb704cc660 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 -- 172.21.15.86:0/1886970853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cc660 msgr2=0x55eb704e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cc660 0x55eb704e2a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87f400bd30 tx=0x7f87f40044d0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 -- 172.21.15.86:0/1886970853 shutdown_connections 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cbd40 0x55eb704c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 --2- 172.21.15.86:0/1886970853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cc660 0x55eb704e2a80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.229+0000 7f88184dab80 1 -- 172.21.15.86:0/1886970853 >> 172.21.15.86:0/1886970853 conn(0x55eb703fb050 msgr2=0x55eb704e2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.230+0000 7f88184dab80 1 -- 172.21.15.86:0/1886970853 shutdown_connections 2024-04-25T19:11:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.230+0000 7f88184dab80 1 -- 172.21.15.86:0/1886970853 wait complete. 2024-04-25T19:11:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.230+0000 7f88184dab80 1 -- start start 2024-04-25T19:11:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.230+0000 7f88184dab80 1 Processor -- start 2024-04-25T19:11:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cbd40 0x55eb704ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b4f00 con 0x55eb704cc660 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b4830 con 0x55eb704d74d0 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b1a50 con 0x55eb704cbd40 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f88184dab80 10 client.0 _ll_register_callbacks cb 0x55eb703b7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f8806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f8806000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704e8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32868/0 (socket says 172.21.15.86:32868) 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f8806000700 1 -- 172.21.15.86:0/1809235738 learned_addr learned my addr 172.21.15.86:0/1809235738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f8807400700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.231+0000 7f8806a00700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cbd40 0x55eb704ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f8806000700 1 -- 172.21.15.86:0/1809235738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cbd40 msgr2=0x55eb704ec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f8806000700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cbd40 0x55eb704ec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f8806000700 1 -- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 msgr2=0x55eb704e9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f8806000700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f8806000700 1 -- 172.21.15.86:0/1809235738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb704e9ab0 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f8806000700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704e8ee0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f87f4000c00 tx=0x7f87f402d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a5c58c4a026a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f87f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f87f0c00700 1 -- 172.21.15.86:0/1809235738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4031070 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f87f0c00700 1 -- 172.21.15.86:0/1809235738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f87f40054f0 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.232+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f4005030 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f87f0c00700 1 -- 172.21.15.86:0/1809235738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87f400b560 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 10 client.16188 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb7054dbd0 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f87f0c00700 1 -- 172.21.15.86:0/1809235738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f4005030 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f87f0c00700 1 -- 172.21.15.86:0/1809235738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87f402dc60 con 0x55eb704cc660 2024-04-25T19:11:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 10 client.16188 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 20 client.16188 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb7054e210 0x55eb70550630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 4 client.16188 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9afd9dba", "release": true, "await": true}] 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.233+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9afd9dba", "release": true, "await": true}) -- 0x55eb70426820 con 0x55eb7054e210 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.234+0000 7f8806a00700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb7054e210 0x55eb70550630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.234+0000 7f8806a00700 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb7054e210 0x55eb70550630 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9265e4897568cdf5 server_cookie=8b99c15e88679b8e in_seq=0 out_seq=0 2024-04-25T19:11:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.234+0000 7f87f0c00700 10 client.16188 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:23.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.237+0000 7f87f0c00700 1 -- 172.21.15.86:0/1809235738 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55eb70426820 con 0x55eb7054e210 2024-04-25T19:11:23.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.237+0000 7f87f0c00700 10 client.16188 handle_command_reply: tid=0 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.237+0000 7f88184dab80 1 client.16188 shutdown 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 msgr2=0x55eb704e8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704e8ee0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f87f4000c00 tx=0x7f87f402d670 comp rx=0 tx=0).stop 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 shutdown_connections 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55eb7054e210 0x55eb70550630 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb704d74d0 0x55eb704e9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb704cbd40 0x55eb704ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f87f0c00700 0 client.16188 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 --2- 172.21.15.86:0/1809235738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb704cc660 0x55eb704e8ee0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 >> 172.21.15.86:0/1809235738 conn(0x55eb703fb050 msgr2=0x55eb704e2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 shutdown_connections 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 1 -- 172.21.15.86:0/1809235738 wait complete. 2024-04-25T19:11:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.238+0000 7f88184dab80 20 client.16188 trim_cache size 0 max 16384 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 24, 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "9afd9dba": { 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 24, 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:11:23.242 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 86.8, 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:11:23.243 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:11:23.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:11:23.244 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:11:23.244 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:11:23.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87ec0a3e30 msgr2=0x7f87ec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87ec0a3e30 0x7f87ec0a62f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fd0 tx=0x7f8800009920 comp rx=0 tx=0).stop 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 msgr2=0x7f88081a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 0x7f88081a5400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87f800ec10 tx=0x7f87f800ef20 comp rx=0 tx=0).stop 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 shutdown_connections 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f87ec0a3e30 0x7f87ec0a62f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880810e3b0 0x7f880819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88081092d0 0x7f88081a5400 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 --2- 172.21.15.86:0/1368967162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8808109c80 0x7f88081a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 >> 172.21.15.86:0/1368967162 conn(0x7f8808076210 msgr2=0x7f880810f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 shutdown_connections 2024-04-25T19:11:23.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:11:23.239+0000 7f880dc00700 1 -- 172.21.15.86:0/1368967162 wait complete. 2024-04-25T19:11:24.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9afd9dba', seconds elapsed: 41.9 2024-04-25T19:11:24.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:24 smithi194 ceph-mon[32740]: pgmap v262: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-25T19:11:24.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:24 smithi086 ceph-mon[27783]: pgmap v262: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-25T19:11:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:24 smithi183 ceph-mon[32344]: pgmap v262: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-25T19:11:26.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:26 smithi194 ceph-mon[32740]: pgmap v263: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-25T19:11:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:26 smithi086 ceph-mon[27783]: pgmap v263: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-25T19:11:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:26 smithi183 ceph-mon[32344]: pgmap v263: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-25T19:11:27.274 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.274 INFO:tasks.workunit.client.0.smithi086.stdout:189/600: open fd = 3 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:190/600: open fd = 3 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:191/600: open fd = 3 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:192/600: open fd = 3 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.275 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:193/600: open fd = 3 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:194/600: open fd = 3 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:195/600: open fd = 3 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:196/600: open fd = 3 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.276 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:197/600: open fd = 3 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:198/600: open fd = 3 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:199/600: open fd = 3 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:200/600: open fd = 3 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.277 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:201/600: open fd = 3 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:202/600: open fd = 3 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:203/600: open fd = 3 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:204/600: open fd = 3 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.278 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:205/600: open fd = 3 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:206/600: open fd = 3 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:207/600: open fd = 3 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:208/600: open fd = 3 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.279 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:209/600: open fd = 3 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:210/600: open fd = 3 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:211/600: open fd = 3 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:212/600: open fd = 3 2024-04-25T19:11:27.280 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:213/600: open fd = 3 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:214/600: open fd = 3 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:215/600: open fd = 3 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:216/600: open fd = 3 2024-04-25T19:11:27.281 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:217/600: open fd = 3 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:218/600: open fd = 3 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:219/600: open fd = 3 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:220/600: open fd = 3 2024-04-25T19:11:27.282 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:221/600: open fd = 3 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:222/600: open fd = 3 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:223/600: open fd = 3 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:224/600: open fd = 3 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.283 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:225/600: open fd = 3 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:226/600: open fd = 3 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:227/600: open fd = 3 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:228/600: open fd = 3 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.284 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:229/600: open fd = 3 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:230/600: open fd = 3 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:231/600: open fd = 3 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:232/600: open fd = 3 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.285 INFO:tasks.workunit.client.0.smithi086.stdout:233/600: open fd = 3 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:234/600: open fd = 3 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:235/600: open fd = 3 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:236/600: open fd = 3 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.286 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:237/600: open fd = 3 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:238/600: open fd = 3 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:239/600: open fd = 3 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:240/600: open fd = 3 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.287 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:241/600: open fd = 3 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:242/600: open fd = 3 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:243/600: open fd = 3 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:244/600: open fd = 3 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.288 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:245/600: open fd = 3 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:246/600: open fd = 3 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:247/600: open fd = 3 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:248/600: open fd = 3 2024-04-25T19:11:27.289 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:249/600: open fd = 3 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:250/600: open fd = 3 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:27.290 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:11:28.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:28 smithi194 ceph-mon[32740]: pgmap v264: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-04-25T19:11:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:28 smithi086 ceph-mon[27783]: pgmap v264: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-04-25T19:11:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:28 smithi183 ceph-mon[32344]: pgmap v264: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-04-25T19:11:30.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:30 smithi194 ceph-mon[32740]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T19:11:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:30 smithi086 ceph-mon[27783]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T19:11:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:30 smithi183 ceph-mon[32344]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T19:11:32.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:32 smithi194 ceph-mon[32740]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-04-25T19:11:32.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:32 smithi086 ceph-mon[27783]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-04-25T19:11:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:32 smithi183 ceph-mon[32344]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 -- 172.21.15.86:0/2604676637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 msgr2=0x7f8af810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 --2- 172.21.15.86:0/2604676637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af810e800 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009930 tx=0x7f8ae802a420 comp rx=0 tx=0).stop 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 -- 172.21.15.86:0/2604676637 shutdown_connections 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 --2- 172.21.15.86:0/2604676637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108e10 0x7f8af8115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 --2- 172.21.15.86:0/2604676637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108490 0x7f8af81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 --2- 172.21.15.86:0/2604676637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af810e800 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 -- 172.21.15.86:0/2604676637 >> 172.21.15.86:0/2604676637 conn(0x7f8af8077d30 msgr2=0x7f8af8106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 -- 172.21.15.86:0/2604676637 shutdown_connections 2024-04-25T19:11:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.712+0000 7f8afe600640 1 -- 172.21.15.86:0/2604676637 wait complete. 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.713+0000 7f8afe600640 1 Processor -- start 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.713+0000 7f8afe600640 1 -- start start 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.713+0000 7f8afe600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 0x7f8af81a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.713+0000 7f8afe600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108e10 0x7f8af81a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.713+0000 7f8afe600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afe600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8118640 con 0x7f8af810e420 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afe600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81184c0 con 0x7f8af8108490 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afe600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81187c0 con 0x7f8af8108e10 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 0x7f8af81a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 0x7f8af81a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:52100/0 (socket says 172.21.15.86:52100) 2024-04-25T19:11:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 -- 172.21.15.86:0/4205218347 learned_addr learned my addr 172.21.15.86:0/4205218347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afdc00640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8aefe00640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108e10 0x7f8af81a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 -- 172.21.15.86:0/4205218347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108e10 msgr2=0x7f8af81a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108e10 0x7f8af81a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 -- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 msgr2=0x7f8af819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afd200640 1 -- 172.21.15.86:0/4205218347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af819f180 con 0x7f8af8108490 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8afdc00640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.714+0000 7f8aefe00640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108e10 0x7f8af81a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.715+0000 7f8afd200640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 0x7f8af81a4850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005ab0 tx=0x7f8ae8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.715+0000 7f8af6000640 1 -- 172.21.15.86:0/4205218347 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae803f070 con 0x7f8af8108490 2024-04-25T19:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.715+0000 7f8af6000640 1 -- 172.21.15.86:0/4205218347 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae8031b50 con 0x7f8af8108490 2024-04-25T19:11:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.715+0000 7f8af6000640 1 -- 172.21.15.86:0/4205218347 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8035030 con 0x7f8af8108490 2024-04-25T19:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.715+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af819f410 con 0x7f8af8108490 2024-04-25T19:11:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.715+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af81ab710 con 0x7f8af8108490 2024-04-25T19:11:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.718+0000 7f8b0296f740 1 Processor -- start 2024-04-25T19:11:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.718+0000 7f8b0296f740 1 -- start start 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8af6000640 1 -- 172.21.15.86:0/4205218347 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ae8042070 con 0x7f8af8108490 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 0x560297980d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x56029798b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56029798bb90 0x56029798df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8b0296f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5602978e6900 con 0x56029798bb90 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8b0296f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56029788cef0 con 0x5602979812d0 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8b0296f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5602978e28c0 con 0x5602979809b0 2024-04-25T19:11:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 0x560297980d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 0x560297980d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:55986/0 (socket says 172.21.15.86:55986) 2024-04-25T19:11:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 -- 172.21.15.86:0/2816839039 learned_addr learned my addr 172.21.15.86:0/2816839039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afd200640 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x56029798b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8aefe00640 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56029798bb90 0x56029798df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 -- 172.21.15.86:0/2816839039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 msgr2=0x56029798b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x56029798b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 -- 172.21.15.86:0/2816839039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56029798bb90 msgr2=0x56029798df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56029798bb90 0x56029798df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.719+0000 7f8afdc00640 1 -- 172.21.15.86:0/2816839039 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602975f9210 con 0x5602979809b0 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8afd200640 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x56029798b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 0x560297980d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8af0011790 tx=0x7f8af0011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4096c081d17b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8aee000640 1 -- 172.21.15.86:0/2816839039 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0003c30 con 0x5602979809b0 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8aee000640 1 -- 172.21.15.86:0/2816839039 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af000fe50 con 0x5602979809b0 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8aee000640 1 -- 172.21.15.86:0/2816839039 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af00193e0 con 0x5602979809b0 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8af6000640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f8adc0a3d90 0x7f8adc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.720+0000 7f8af6000640 1 -- 172.21.15.86:0/4205218347 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f8ae80f0de0 con 0x7f8af8108490 2024-04-25T19:11:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8afd200640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f8adc0a3d90 0x7f8adc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 -- 172.21.15.86:0/2816839039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 msgr2=0x560297980d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 0x560297980d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8af0011790 tx=0x7f8af0011c50 comp rx=0 tx=0).stop 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 -- 172.21.15.86:0/2816839039 shutdown_connections 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56029798bb90 0x56029798df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x56029798b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2816839039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979809b0 0x560297980d90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 -- 172.21.15.86:0/2816839039 >> 172.21.15.86:0/2816839039 conn(0x56029797cec0 msgr2=0x56029797d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 -- 172.21.15.86:0/2816839039 shutdown_connections 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 -- 172.21.15.86:0/2816839039 wait complete. 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8afd200640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f8adc0a3d90 0x7f8adc0a6250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x560297981f50 tx=0x7f8ae8009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.721+0000 7f8b0296f740 1 -- start start 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 Processor -- start 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297988b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297989040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x560297989580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56029798e7c0 con 0x5602979809b0 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5602978a0310 con 0x56029798bb90 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56029798e940 con 0x5602979812d0 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8b0296f740 10 client.0 _ll_register_callbacks cb 0x56029786ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297989040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297988b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297989040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56002/0 (socket says 172.21.15.86:56002) 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297988b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59682/0 (socket says 172.21.15.86:59682) 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.722+0000 7f8aefe00640 1 -- 172.21.15.86:0/2820524210 learned_addr learned my addr 172.21.15.86:0/2820524210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x560297989580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8aefe00640 1 -- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 msgr2=0x560297989580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8aefe00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x560297989580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8aefe00640 1 -- 172.21.15.86:0/2820524210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 msgr2=0x560297988b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8aefe00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297988b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8aefe00640 1 -- 172.21.15.86:0/2820524210 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602976c5620 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297988b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8aefe00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297989040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0012a10 tx=0x7f8ae0012ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52a84c1203cf067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8ae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.723+0000 7f8ae6a00640 1 -- 172.21.15.86:0/2820524210 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0004050 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.724+0000 7f8ae6a00640 1 -- 172.21.15.86:0/2820524210 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f8ae00041f0 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.724+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af000fe50 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.724+0000 7f8ae6a00640 1 -- 172.21.15.86:0/2820524210 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ae00043f0 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.724+0000 7f8b0296f740 10 client.26290 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.724+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56029799aad0 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.724+0000 7f8ae6a00640 1 -- 172.21.15.86:0/2820524210 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af000fe50 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8ae6a00640 1 -- 172.21.15.86:0/2820524210 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ae0003c00 con 0x5602979812d0 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8b0296f740 10 client.26290 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8b0296f740 10 client.26290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8b0296f740 20 client.26290 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56029799bf90 0x56029799e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8b0296f740 4 client.26290 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602978fa8b0 con 0x56029799bf90 2024-04-25T19:11:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.725+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56029799bf90 0x56029799e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:34 smithi194 ceph-mon[32740]: pgmap v267: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T19:11:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.726+0000 7f8afdc00640 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56029799bf90 0x56029799e3f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dad2d99f265bf14 server_cookie=89cc676d6e05470b in_seq=0 out_seq=0 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.726+0000 7f8ae6a00640 10 client.26290 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.726+0000 7f8ae6a00640 1 -- 172.21.15.86:0/2820524210 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602978fa8b0 con 0x56029799bf90 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.726+0000 7f8ae6a00640 10 client.26290 handle_command_reply: tid=0 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.726+0000 7f8b0296f740 1 client.26290 shutdown 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 msgr2=0x560297989040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297989040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0012a10 tx=0x7f8ae0012ed0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 shutdown_connections 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56029799bf90 0x56029799e3f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x560297989580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297989040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 --2- 172.21.15.86:0/2820524210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297988b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8ae6a00640 0 client.26290 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 >> 172.21.15.86:0/2820524210 conn(0x56029797cec0 msgr2=0x560297982bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 shutdown_connections 2024-04-25T19:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 1 -- 172.21.15.86:0/2820524210 wait complete. 2024-04-25T19:11:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.727+0000 7f8b0296f740 20 client.26290 trim_cache size 0 max 16384 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 Processor -- start 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 -- start start 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x5602979976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x5602979a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029798bb90 0x5602979a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b4230 con 0x5602979809b0 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b5d70 con 0x5602979812d0 2024-04-25T19:11:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.743+0000 7f8b0296f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b4900 con 0x56029798bb90 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x5602979976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x5602979976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59698/0 (socket says 172.21.15.86:59698) 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 -- 172.21.15.86:0/470809186 learned_addr learned my addr 172.21.15.86:0/470809186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8aefe00640 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x5602979a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029798bb90 0x5602979a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 -- 172.21.15.86:0/470809186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029798bb90 msgr2=0x5602979a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029798bb90 0x5602979a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 -- 172.21.15.86:0/470809186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 msgr2=0x5602979a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x5602979a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 -- 172.21.15.86:0/470809186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560297987090 con 0x5602979809b0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.744+0000 7f8afdc00640 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x5602979976b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8af0014040 tx=0x7f8af0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb44ce95d79d46a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8aed600640 1 -- 172.21.15.86:0/470809186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af001f040 con 0x5602979809b0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8aed600640 1 -- 172.21.15.86:0/470809186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af0003a30 con 0x5602979809b0 2024-04-25T19:11:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8aed600640 1 -- 172.21.15.86:0/470809186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af002e410 con 0x5602979809b0 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 -- 172.21.15.86:0/470809186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 msgr2=0x5602979976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x5602979976b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8af0014040 tx=0x7f8af0009500 comp rx=0 tx=0).stop 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 -- 172.21.15.86:0/470809186 shutdown_connections 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029798bb90 0x5602979a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5602979812d0 0x5602979a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 --2- 172.21.15.86:0/470809186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x5602979976b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 -- 172.21.15.86:0/470809186 >> 172.21.15.86:0/470809186 conn(0x56029797cec0 msgr2=0x560297983e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 -- 172.21.15.86:0/470809186 shutdown_connections 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.745+0000 7f8b0296f740 1 -- 172.21.15.86:0/470809186 wait complete. 2024-04-25T19:11:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.746+0000 7f8b0296f740 1 -- start start 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.746+0000 7f8b0296f740 1 Processor -- start 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.746+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297993bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297994130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8b0296f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x5602979912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8b0296f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b56a0 con 0x5602979809b0 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8b0296f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b4fd0 con 0x56029798bb90 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8b0296f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af81b2230 con 0x5602979812d0 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8b0296f740 10 client.0 _ll_register_callbacks cb 0x7f8ae0012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297993bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297993bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59714/0 (socket says 172.21.15.86:59714) 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 -- 172.21.15.86:0/362281962 learned_addr learned my addr 172.21.15.86:0/362281962 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8aefe00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297994130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x5602979912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 -- 172.21.15.86:0/362281962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 msgr2=0x560297994130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297994130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 -- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 msgr2=0x5602979912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x5602979912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.747+0000 7f8afdc00640 1 -- 172.21.15.86:0/362281962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560297991ad0 con 0x5602979809b0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8afdc00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297993bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009810 tx=0x7f8af0009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4da0cbbbbbaf5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8ae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8ae6a00640 1 -- 172.21.15.86:0/362281962 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af003d060 con 0x5602979809b0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af0003a30 con 0x5602979809b0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8ae6a00640 1 -- 172.21.15.86:0/362281962 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f8af0003ca0 con 0x5602979809b0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8ae6a00640 1 -- 172.21.15.86:0/362281962 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8af0032040 con 0x5602979809b0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8b0296f740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560297a021f0 con 0x5602979809b0 2024-04-25T19:11:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.748+0000 7f8ae6a00640 1 -- 172.21.15.86:0/362281962 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af0003a30 con 0x5602979809b0 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8ae6a00640 1 -- 172.21.15.86:0/362281962 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8af0018070 con 0x5602979809b0 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8b0296f740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8b0296f740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8b0296f740 20 client.16206 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8b0296f740 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560297a02620 0x560297a04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8b0296f740 4 client.16206 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602978f4320 con 0x560297a02620 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.749+0000 7f8aefe00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560297a02620 0x560297a04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.750+0000 7f8aefe00640 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560297a02620 0x560297a04a10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896c8bea44fb8825 server_cookie=9da9a4a9e24f0468 in_seq=0 out_seq=0 2024-04-25T19:11:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.750+0000 7f8ae6a00640 10 client.16206 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.763+0000 7f8ae6a00640 1 -- 172.21.15.86:0/362281962 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602978f4320 con 0x560297a02620 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.763+0000 7f8ae6a00640 10 client.16206 handle_command_reply: tid=0 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.763+0000 7f8b0296f740 1 client.16206 shutdown 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.763+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 msgr2=0x560297993bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.763+0000 7f8b0296f740 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297993bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009810 tx=0x7f8af0009fa0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 shutdown_connections 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560297a02620 0x560297a04a10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56029798bb90 0x5602979912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5602979812d0 0x560297994130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 --2- 172.21.15.86:0/362281962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602979809b0 0x560297993bf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8ae6a00640 0 client.16206 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 >> 172.21.15.86:0/362281962 conn(0x56029797cec0 msgr2=0x560297988aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 shutdown_connections 2024-04-25T19:11:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 1 -- 172.21.15.86:0/362281962 wait complete. 2024-04-25T19:11:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.764+0000 7f8b0296f740 20 client.16206 trim_cache size 0 max 16384 2024-04-25T19:11:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:11:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:11:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:11:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f8adc0a3d90 msgr2=0x7f8adc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f8adc0a3d90 0x7f8adc0a6250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x560297981f50 tx=0x7f8ae8009260 comp rx=0 tx=0).stop 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 msgr2=0x7f8af81a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 0x7f8af81a4850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005ab0 tx=0x7f8ae8003b90 comp rx=0 tx=0).stop 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 shutdown_connections 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f8adc0a3d90 0x7f8adc0a6250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8af810e420 0x7f8af819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af8108e10 0x7f8af81a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 --2- 172.21.15.86:0/4205218347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8af8108490 0x7f8af81a4850 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 >> 172.21.15.86:0/4205218347 conn(0x7f8af8077d30 msgr2=0x7f8af8078ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 shutdown_connections 2024-04-25T19:11:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:34.765+0000 7f8afe600640 1 -- 172.21.15.86:0/4205218347 wait complete. 2024-04-25T19:11:34.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:34 smithi086 ceph-mon[27783]: pgmap v267: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T19:11:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:34 smithi183 ceph-mon[32344]: pgmap v267: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T19:11:36.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:36 smithi194 ceph-mon[32740]: pgmap v268: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:36.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:36 smithi086 ceph-mon[27783]: pgmap v268: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:36 smithi183 ceph-mon[32344]: pgmap v268: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:38 smithi194 ceph-mon[32740]: pgmap v269: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:38 smithi086 ceph-mon[27783]: pgmap v269: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:38 smithi183 ceph-mon[32344]: pgmap v269: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:40.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:40 smithi086 ceph-mon[27783]: pgmap v270: 97 pgs: 97 active+clean; 2.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:11:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:40 smithi183 ceph-mon[32344]: pgmap v270: 97 pgs: 97 active+clean; 2.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:11:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:40 smithi194 ceph-mon[32740]: pgmap v270: 97 pgs: 97 active+clean; 2.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:11:42.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:42 smithi086 ceph-mon[27783]: pgmap v271: 97 pgs: 97 active+clean; 2.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:42.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:42 smithi183 ceph-mon[32344]: pgmap v271: 97 pgs: 97 active+clean; 2.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:42 smithi194 ceph-mon[32740]: pgmap v271: 97 pgs: 97 active+clean; 2.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:44 smithi086 ceph-mon[27783]: pgmap v272: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T19:11:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:44 smithi183 ceph-mon[32344]: pgmap v272: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T19:11:44.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:44 smithi194 ceph-mon[32740]: pgmap v272: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T19:11:46.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:46 smithi086 ceph-mon[27783]: pgmap v273: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:11:46.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:46 smithi183 ceph-mon[32344]: pgmap v273: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:11:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:46 smithi194 ceph-mon[32740]: pgmap v273: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:11:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 -- 172.21.15.86:0/368532595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0101860 msgr2=0x7f98c01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 --2- 172.21.15.86:0/368532595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0101860 0x7f98c01058a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009900 tx=0x7f98ac02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 -- 172.21.15.86:0/368532595 shutdown_connections 2024-04-25T19:11:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 --2- 172.21.15.86:0/368532595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c0105f80 0x7f98c010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 --2- 172.21.15.86:0/368532595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0101860 0x7f98c01058a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 --2- 172.21.15.86:0/368532595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98c0100eb0 0x7f98c0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 -- 172.21.15.86:0/368532595 >> 172.21.15.86:0/368532595 conn(0x7f98c00fc770 msgr2=0x7f98c00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 -- 172.21.15.86:0/368532595 shutdown_connections 2024-04-25T19:11:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.906+0000 7f98c5e00640 1 -- 172.21.15.86:0/368532595 wait complete. 2024-04-25T19:11:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.908+0000 7f98c5e00640 1 Processor -- start 2024-04-25T19:11:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.908+0000 7f98c5e00640 1 -- start start 2024-04-25T19:11:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.908+0000 7f98c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 0x7f98c0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98c0105f80 0x7f98c0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c019cee0 0x7f98c019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c0110ee0 con 0x7f98c019cee0 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98c0110d60 con 0x7f98c0100eb0 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98c0111060 con 0x7f98c0105f80 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c019cee0 0x7f98c019d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 0x7f98c0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98c0105f80 0x7f98c0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 0x7f98c0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:53140/0 (socket says 172.21.15.86:53140) 2024-04-25T19:11:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 -- 172.21.15.86:0/2873084845 learned_addr learned my addr 172.21.15.86:0/2873084845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 -- 172.21.15.86:0/2873084845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98c0105f80 msgr2=0x7f98c0198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98c0105f80 0x7f98c0198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 -- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c019cee0 msgr2=0x7f98c019d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c019cee0 0x7f98c019d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c4a00640 1 -- 172.21.15.86:0/2873084845 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c019d9d0 con 0x7f98c0100eb0 2024-04-25T19:11:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.909+0000 7f98c5400640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c019cee0 0x7f98c019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.910+0000 7f98c4a00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 0x7f98c0198670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f98b400e9f0 tx=0x7f98b400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.910+0000 7f98bd600640 1 -- 172.21.15.86:0/2873084845 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b400ccc0 con 0x7f98c0100eb0 2024-04-25T19:11:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.910+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c019dcc0 con 0x7f98c0100eb0 2024-04-25T19:11:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.910+0000 7f98bd600640 1 -- 172.21.15.86:0/2873084845 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b400ce60 con 0x7f98c0100eb0 2024-04-25T19:11:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.910+0000 7f98bd600640 1 -- 172.21.15.86:0/2873084845 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4010430 con 0x7f98c0100eb0 2024-04-25T19:11:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.910+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01a5480 con 0x7f98c0100eb0 2024-04-25T19:11:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.913+0000 7f98ca273740 1 Processor -- start 2024-04-25T19:11:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 -- start start 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcc1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dc15900 con 0x55bb9dcbefa0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dbbbef0 con 0x55bb9dcb01f0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98ca273740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dc118c0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98bd600640 1 -- 172.21.15.86:0/2873084845 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98b40046c0 con 0x7f98c0100eb0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcc1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcc1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39562/0 (socket says 172.21.15.86:39562) 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98bfe00640 1 -- 172.21.15.86:0/3180504993 learned_addr learned my addr 172.21.15.86:0/3180504993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98c5400640 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.914+0000 7f98c4a00640 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98bfe00640 1 -- 172.21.15.86:0/3180504993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcaf8d0 msgr2=0x55bb9dcafcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98bfe00640 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcafcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98bfe00640 1 -- 172.21.15.86:0/3180504993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 msgr2=0x55bb9dcbea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98bfe00640 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98bfe00640 1 -- 172.21.15.86:0/3180504993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9d928210 con 0x55bb9dcbefa0 2024-04-25T19:11:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98c5400640 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.915+0000 7f98bfe00640 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcc1390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009900 tx=0x7f98ac005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1acecbcd92eae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98b1c00640 1 -- 172.21.15.86:0/3180504993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac040070 con 0x55bb9dcbefa0 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98b1c00640 1 -- 172.21.15.86:0/3180504993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac004030 con 0x55bb9dcbefa0 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98b1c00640 1 -- 172.21.15.86:0/3180504993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac035040 con 0x55bb9dcbefa0 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98bd600640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98a80a3d10 0x7f98a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98bd600640 1 -- 172.21.15.86:0/2873084845 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f98b40d0cb0 con 0x7f98c0100eb0 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98c5400640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98a80a3d10 0x7f98a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98ca273740 1 -- 172.21.15.86:0/3180504993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 msgr2=0x55bb9dcc1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.916+0000 7f98ca273740 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcc1390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009900 tx=0x7f98ac005e40 comp rx=0 tx=0).stop 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 -- 172.21.15.86:0/3180504993 shutdown_connections 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcc1390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 --2- 172.21.15.86:0/3180504993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 -- 172.21.15.86:0/3180504993 >> 172.21.15.86:0/3180504993 conn(0x55bb9dcabec0 msgr2=0x55bb9dcac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98c5400640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98a80a3d10 0x7f98a80a61d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98b8005fd0 tx=0x7f98b8009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 -- 172.21.15.86:0/3180504993 shutdown_connections 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 -- 172.21.15.86:0/3180504993 wait complete. 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.917+0000 7f98ca273740 1 -- start start 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 Processor -- start 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcbc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dcc1b70 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dbcf310 con 0x55bb9dcbefa0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dcc1cf0 con 0x55bb9dcb01f0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98ca273740 10 client.0 _ll_register_callbacks cb 0x55bb9db9dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcbbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39578/0 (socket says 172.21.15.86:39578) 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98c4a00640 1 -- 172.21.15.86:0/1528709721 learned_addr learned my addr 172.21.15.86:0/1528709721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.918+0000 7f98c5400640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcbc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98c4a00640 1 -- 172.21.15.86:0/1528709721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 msgr2=0x55bb9dcbc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98c4a00640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98c4a00640 1 -- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcbefa0 msgr2=0x55bb9dcbc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98c4a00640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcbc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98c4a00640 1 -- 172.21.15.86:0/1528709721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9d9f4620 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98c4a00640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcbbf70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f98b409c880 tx=0x7f98b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86be2cae44029a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98a6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98a6000640 1 -- 172.21.15.86:0/1528709721 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b409c4e0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98a6000640 1 -- 172.21.15.86:0/1528709721 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f98b409cde0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.919+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac004030 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98a6000640 1 -- 172.21.15.86:0/1528709721 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f98b40ce7b0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 10 client.16218 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb9dccdeb0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98a6000640 1 -- 172.21.15.86:0/1528709721 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac004030 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98a6000640 1 -- 172.21.15.86:0/1528709721 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f98b40a5360 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 10 client.16218 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 20 client.16218 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dccf370 0x55bb9dcd1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 4 client.16218 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:11:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.920+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb9dc298b0 con 0x55bb9dccf370 2024-04-25T19:11:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.921+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dccf370 0x55bb9dcd1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.921+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dccf370 0x55bb9dcd1760 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cb29a9e50d64aa server_cookie=de9921fe34ac83e3 in_seq=0 out_seq=0 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.921+0000 7f98a6000640 10 client.16218 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98a6000640 1 -- 172.21.15.86:0/1528709721 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb9dc298b0 con 0x55bb9dccf370 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98a6000640 10 client.16218 handle_command_reply: tid=0 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 client.16218 shutdown 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 msgr2=0x55bb9dcbbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcbbf70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f98b409c880 tx=0x7f98b4005740 comp rx=0 tx=0).stop 2024-04-25T19:11:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 shutdown_connections 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dccf370 0x55bb9dcd1760 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcbc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcbc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 --2- 172.21.15.86:0/1528709721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcbbf70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 >> 172.21.15.86:0/1528709721 conn(0x55bb9dcabec0 msgr2=0x55bb9dcb1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98a6000640 0 client.16218 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.922+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 shutdown_connections 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.923+0000 7f98ca273740 1 -- 172.21.15.86:0/1528709721 wait complete. 2024-04-25T19:11:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.923+0000 7f98ca273740 20 client.16218 trim_cache size 0 max 16384 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.939+0000 7f98ca273740 1 Processor -- start 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 -- start start 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dccaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcd7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcd7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c01ab9c0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98c01ad500 con 0x55bb9dcb01f0 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98ca273740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98c01ac090 con 0x55bb9dcbefa0 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dccaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dccaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39592/0 (socket says 172.21.15.86:39592) 2024-04-25T19:11:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98c4a00640 1 -- 172.21.15.86:0/1552268046 learned_addr learned my addr 172.21.15.86:0/1552268046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcd7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.940+0000 7f98c5400640 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcd7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98bfe00640 1 -- 172.21.15.86:0/1552268046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcbefa0 msgr2=0x55bb9dcd7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcd7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98bfe00640 1 -- 172.21.15.86:0/1552268046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 msgr2=0x55bb9dccaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dccaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98bfe00640 1 -- 172.21.15.86:0/1552268046 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9dcb6d70 con 0x55bb9dcb01f0 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98c4a00640 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dccaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98bfe00640 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcd7000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55bb9dca64f0 tx=0x7f98ac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5de682a01bc4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98b1200640 1 -- 172.21.15.86:0/1552268046 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac040050 con 0x55bb9dcb01f0 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98b1200640 1 -- 172.21.15.86:0/1552268046 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac032e60 con 0x55bb9dcb01f0 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.941+0000 7f98b1200640 1 -- 172.21.15.86:0/1552268046 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac035040 con 0x55bb9dcb01f0 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 -- 172.21.15.86:0/1552268046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 msgr2=0x55bb9dcd7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcd7000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55bb9dca64f0 tx=0x7f98ac005fd0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 -- 172.21.15.86:0/1552268046 shutdown_connections 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcbefa0 0x55bb9dcd7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcd7000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 --2- 172.21.15.86:0/1552268046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dccaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.942+0000 7f98ca273740 1 -- 172.21.15.86:0/1552268046 >> 172.21.15.86:0/1552268046 conn(0x55bb9dcabec0 msgr2=0x55bb9dcb7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.943+0000 7f98ca273740 1 -- 172.21.15.86:0/1552268046 shutdown_connections 2024-04-25T19:11:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.943+0000 7f98ca273740 1 -- 172.21.15.86:0/1552268046 wait complete. 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.943+0000 7f98ca273740 1 -- start start 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.943+0000 7f98ca273740 1 Processor -- start 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcc4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dd33870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c01ace30 con 0x55bb9dcbefa0 2024-04-25T19:11:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98c01ac760 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98c01a99c0 con 0x55bb9dcb01f0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98ca273740 10 client.0 _ll_register_callbacks cb 0x7f98b4005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcc4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dd33870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98c4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcc4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:53176/0 (socket says 172.21.15.86:53176) 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcc4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41514/0 (socket says 172.21.15.86:41514) 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.944+0000 7f98c4a00640 1 -- 172.21.15.86:0/2430660913 learned_addr learned my addr 172.21.15.86:0/2430660913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98c4a00640 1 -- 172.21.15.86:0/2430660913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 msgr2=0x55bb9dcc4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98c4a00640 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcc4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98c4a00640 1 -- 172.21.15.86:0/2430660913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 msgr2=0x55bb9dd33870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98c4a00640 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dd33870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98c4a00640 1 -- 172.21.15.86:0/2430660913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9dd33db0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98c4a00640 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcc4100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f98b4099550 tx=0x7f98b4009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c5d4904ef5cfc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98a6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98a6000640 1 -- 172.21.15.86:0/2430660913 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4066060 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98a6000640 1 -- 172.21.15.86:0/2430660913 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f98b409cde0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.945+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac032e60 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98a6000640 1 -- 172.21.15.86:0/2430660913 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f98b40d4070 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 10 client.26321 fetch_fsmap learned FSMap version 29 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb9dd350c0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98a6000640 1 -- 172.21.15.86:0/2430660913 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac032e60 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98a6000640 1 -- 172.21.15.86:0/2430660913 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f98b4061bc0 con 0x55bb9dcaf8d0 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 10 client.26321 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 10 client.26321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:11:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 20 client.26321 populate_metadata read hostname 'smithi086' 2024-04-25T19:11:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dd35620 0x55bb9dd37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:11:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 4 client.26321 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:11:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb9dc23320 con 0x55bb9dd35620 2024-04-25T19:11:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.946+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dd35620 0x55bb9dd37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:11:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.947+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dd35620 0x55bb9dd37a40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8a43692a799aac server_cookie=1b5af4007327275b in_seq=0 out_seq=0 2024-04-25T19:11:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.947+0000 7f98a6000640 10 client.26321 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98a6000640 1 -- 172.21.15.86:0/2430660913 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb9dc23320 con 0x55bb9dd35620 2024-04-25T19:11:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98a6000640 10 client.26321 handle_command_reply: tid=0 2024-04-25T19:11:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 client.26321 shutdown 2024-04-25T19:11:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 msgr2=0x55bb9dcc4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcc4100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f98b4099550 tx=0x7f98b4009790 comp rx=0 tx=0).stop 2024-04-25T19:11:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 shutdown_connections 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55bb9dd35620 0x55bb9dd37a40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb9dcbefa0 0x55bb9dd33870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9dcb01f0 0x55bb9dcc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 --2- 172.21.15.86:0/2430660913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bb9dcaf8d0 0x55bb9dcc4100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 >> 172.21.15.86:0/2430660913 conn(0x55bb9dcabec0 msgr2=0x55bb9dcaa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.961+0000 7f98a6000640 0 client.26321 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.962+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 shutdown_connections 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.962+0000 7f98ca273740 1 -- 172.21.15.86:0/2430660913 wait complete. 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.962+0000 7f98ca273740 20 client.26321 trim_cache size 0 max 16384 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:11:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:11:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:11:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98a80a3d10 msgr2=0x7f98a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98a80a3d10 0x7f98a80a61d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98b8005fd0 tx=0x7f98b8009500 comp rx=0 tx=0).stop 2024-04-25T19:11:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 msgr2=0x7f98c0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 0x7f98c0198670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f98b400e9f0 tx=0x7f98b400edf0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 shutdown_connections 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f98a80a3d10 0x7f98a80a61d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c019cee0 0x7f98c019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98c0105f80 0x7f98c0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 --2- 172.21.15.86:0/2873084845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c0100eb0 0x7f98c0198670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 >> 172.21.15.86:0/2873084845 conn(0x7f98c00fc770 msgr2=0x7f98c0069a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 shutdown_connections 2024-04-25T19:11:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:11:47.963+0000 7f98c5e00640 1 -- 172.21.15.86:0/2873084845 wait complete. 2024-04-25T19:11:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:48 smithi086 ceph-mon[27783]: pgmap v274: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:11:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:48 smithi183 ceph-mon[32344]: pgmap v274: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:11:48.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:48 smithi194 ceph-mon[32740]: pgmap v274: 97 pgs: 97 active+clean; 2.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:11:50.784 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:50 smithi194 ceph-mon[32740]: pgmap v275: 97 pgs: 97 active+clean; 3.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:11:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:50 smithi086 ceph-mon[27783]: pgmap v275: 97 pgs: 97 active+clean; 3.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:11:50.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:50 smithi183 ceph-mon[32344]: pgmap v275: 97 pgs: 97 active+clean; 3.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:11:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:52 smithi086 ceph-mon[27783]: pgmap v276: 97 pgs: 97 active+clean; 3.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:52 smithi183 ceph-mon[32344]: pgmap v276: 97 pgs: 97 active+clean; 3.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:52 smithi194 ceph-mon[32740]: pgmap v276: 97 pgs: 97 active+clean; 3.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:11:54.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:54 smithi086 ceph-mon[27783]: pgmap v277: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:11:54.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:54 smithi183 ceph-mon[32344]: pgmap v277: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:11:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:54 smithi194 ceph-mon[32740]: pgmap v277: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:11:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:56 smithi086 ceph-mon[27783]: pgmap v278: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:56.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:56 smithi183 ceph-mon[32344]: pgmap v278: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:56.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:56 smithi194 ceph-mon[32740]: pgmap v278: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:11:58 smithi086 ceph-mon[27783]: pgmap v279: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:11:58 smithi183 ceph-mon[32344]: pgmap v279: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:11:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:11:58 smithi194 ceph-mon[32740]: pgmap v279: 97 pgs: 97 active+clean; 3.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:12:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:00 smithi086 ceph-mon[27783]: pgmap v280: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:12:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:00 smithi183 ceph-mon[32344]: pgmap v280: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:12:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:00 smithi194 ceph-mon[32740]: pgmap v280: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:12:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 -- 172.21.15.86:0/332621560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074ca0 msgr2=0x7f235010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 --2- 172.21.15.86:0/332621560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074ca0 0x7f235010e050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2338009900 tx=0x7f233802a480 comp rx=0 tx=0).stop 2024-04-25T19:12:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 -- 172.21.15.86:0/332621560 shutdown_connections 2024-04-25T19:12:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 --2- 172.21.15.86:0/332621560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 0x7f23501109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 --2- 172.21.15.86:0/332621560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074ca0 0x7f235010e050 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 --2- 172.21.15.86:0/332621560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350074380 0x7f2350074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 -- 172.21.15.86:0/332621560 >> 172.21.15.86:0/332621560 conn(0x7f23500fd8a0 msgr2=0x7f23500ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 -- 172.21.15.86:0/332621560 shutdown_connections 2024-04-25T19:12:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.108+0000 7f234f200640 1 -- 172.21.15.86:0/332621560 wait complete. 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.109+0000 7f234f200640 1 Processor -- start 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 -- start start 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 0x7f23501a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350074ca0 0x7f23501a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 0x7f235019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23501141b0 con 0x7f2350074380 2024-04-25T19:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2350114030 con 0x7f2350074ca0 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2350114330 con 0x7f235010e590 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 0x7f23501a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 0x7f23501a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59094/0 (socket says 172.21.15.86:59094) 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234de00640 1 -- 172.21.15.86:0/454111459 learned_addr learned my addr 172.21.15.86:0/454111459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.110+0000 7f234e800640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 0x7f235019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f2345400640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350074ca0 0x7f23501a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234e800640 1 -- 172.21.15.86:0/454111459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350074ca0 msgr2=0x7f23501a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234e800640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350074ca0 0x7f23501a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234e800640 1 -- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 msgr2=0x7f23501a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234e800640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 0x7f23501a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234e800640 1 -- 172.21.15.86:0/454111459 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235019ad10 con 0x7f235010e590 2024-04-25T19:12:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234de00640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 0x7f23501a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f234e800640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 0x7f235019a590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2348009fd0 tx=0x7f234800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.111+0000 7f2347400640 1 -- 172.21.15.86:0/454111459 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234800ce20 con 0x7f235010e590 2024-04-25T19:12:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.112+0000 7f2347400640 1 -- 172.21.15.86:0/454111459 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2348005020 con 0x7f235010e590 2024-04-25T19:12:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.112+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235019b000 con 0x7f235010e590 2024-04-25T19:12:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.112+0000 7f2347400640 1 -- 172.21.15.86:0/454111459 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348010690 con 0x7f235010e590 2024-04-25T19:12:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.112+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501a72a0 con 0x7f235010e590 2024-04-25T19:12:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.115+0000 7f23577fd740 1 Processor -- start 2024-04-25T19:12:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.115+0000 7f2347400640 1 -- 172.21.15.86:0/454111459 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23480054d0 con 0x7f235010e590 2024-04-25T19:12:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.115+0000 7f23577fd740 1 -- start start 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f23577fd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d2cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f23577fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d37650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f23577fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d39f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f23577fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561d3c92900 con 0x5561d3d2d2d0 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f23577fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5561d3c38ef0 con 0x5561d3d37b90 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f23577fd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561d3c8e8c0 con 0x5561d3d2c9b0 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f234de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d37650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f234de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d37650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59096/0 (socket says 172.21.15.86:59096) 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f234de00640 1 -- 172.21.15.86:0/2190662058 learned_addr learned my addr 172.21.15.86:0/2190662058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f2345400640 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d2cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.116+0000 7f2345400640 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d39f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f234de00640 1 -- 172.21.15.86:0/2190662058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 msgr2=0x5561d3d2cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f234de00640 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d2cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f234de00640 1 -- 172.21.15.86:0/2190662058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 msgr2=0x5561d3d39f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f234de00640 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d39f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f234de00640 1 -- 172.21.15.86:0/2190662058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d39a5210 con 0x5561d3d2d2d0 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f2347400640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f23340a3d10 0x7f23340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f2347400640 1 -- 172.21.15.86:0/454111459 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f23480d0f50 con 0x7f235010e590 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f234de00640 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d37650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2340011650 tx=0x7f2340009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c9cf7dd1842ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f2345400640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f23340a3d10 0x7f23340a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f233f400640 1 -- 172.21.15.86:0/2190662058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234001e070 con 0x5561d3d2d2d0 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f233f400640 1 -- 172.21.15.86:0/2190662058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234000fb10 con 0x5561d3d2d2d0 2024-04-25T19:12:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.117+0000 7f233f400640 1 -- 172.21.15.86:0/2190662058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340019b80 con 0x5561d3d2d2d0 2024-04-25T19:12:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 -- 172.21.15.86:0/2190662058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 msgr2=0x5561d3d37650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f2345400640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f23340a3d10 0x7f23340a61d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2338031440 tx=0x7f2338005780 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d37650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2340011650 tx=0x7f2340009f90 comp rx=0 tx=0).stop 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 -- 172.21.15.86:0/2190662058 shutdown_connections 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d39f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d37650 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 --2- 172.21.15.86:0/2190662058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d2cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 -- 172.21.15.86:0/2190662058 >> 172.21.15.86:0/2190662058 conn(0x5561d3d28ec0 msgr2=0x5561d3d292a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 -- 172.21.15.86:0/2190662058 shutdown_connections 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.118+0000 7f23577fd740 1 -- 172.21.15.86:0/2190662058 wait complete. 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.119+0000 7f23577fd740 1 -- start start 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.119+0000 7f23577fd740 1 Processor -- start 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.119+0000 7f23577fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 0x5561d3d34ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.119+0000 7f23577fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2d2d0 0x5561d3d34ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.119+0000 7f23577fd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d35530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f23577fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561d3d3a7c0 con 0x5561d3d2c9b0 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f23577fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5561d3c4c310 con 0x5561d3d2d2d0 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f23577fd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561d3d3a940 con 0x5561d3d37b90 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f23577fd740 10 client.0 _ll_register_callbacks cb 0x5561d3c1acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 0x5561d3d34ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d35530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 0x5561d3d34ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59106/0 (socket says 172.21.15.86:59106) 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 -- 172.21.15.86:0/1764039147 learned_addr learned my addr 172.21.15.86:0/1764039147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f2345400640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2d2d0 0x5561d3d34ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 -- 172.21.15.86:0/1764039147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 msgr2=0x5561d3d35530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d35530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 -- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2d2d0 msgr2=0x5561d3d34ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2d2d0 0x5561d3d34ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234de00640 1 -- 172.21.15.86:0/1764039147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d3a71620 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.120+0000 7f234e800640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d35530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f234de00640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 0x5561d3d34ab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f23400196e0 tx=0x7f2340019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d47fc064f819b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f2333e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f2333e00640 1 -- 172.21.15.86:0/1764039147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234001e040 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f2333e00640 1 -- 172.21.15.86:0/1764039147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f2340016040 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234000fb10 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f2333e00640 1 -- 172.21.15.86:0/1764039147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f234002f050 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f23577fd740 10 client.16236 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561d3d469c0 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.121+0000 7f2333e00640 1 -- 172.21.15.86:0/1764039147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234000fb10 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f2333e00640 1 -- 172.21.15.86:0/1764039147 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2340021070 con 0x5561d3d2c9b0 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f23577fd740 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f23577fd740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f23577fd740 20 client.16236 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f23577fd740 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3d47d60 0x5561d3d4a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f23577fd740 4 client.16236 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561d3ca68b0 con 0x5561d3d47d60 2024-04-25T19:12:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.122+0000 7f2345400640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3d47d60 0x5561d3d4a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.123+0000 7f2345400640 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3d47d60 0x5561d3d4a150 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974b2e69347dcdbb server_cookie=ed6bf8cbe18a14a4 in_seq=0 out_seq=0 2024-04-25T19:12:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.123+0000 7f2333e00640 10 client.16236 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f2333e00640 1 -- 172.21.15.86:0/1764039147 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561d3ca68b0 con 0x5561d3d47d60 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f2333e00640 10 client.16236 handle_command_reply: tid=0 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 client.16236 shutdown 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 msgr2=0x5561d3d34ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 0x5561d3d34ab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f23400196e0 tx=0x7f2340019520 comp rx=0 tx=0).stop 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 shutdown_connections 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3d47d60 0x5561d3d4a150 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d35530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2d2d0 0x5561d3d34ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f2333e00640 0 client.16236 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 --2- 172.21.15.86:0/1764039147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2c9b0 0x5561d3d34ab0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 >> 172.21.15.86:0/1764039147 conn(0x5561d3d28ec0 msgr2=0x5561d3d2ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 shutdown_connections 2024-04-25T19:12:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 1 -- 172.21.15.86:0/1764039147 wait complete. 2024-04-25T19:12:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.124+0000 7f23577fd740 20 client.16236 trim_cache size 0 max 16384 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.141+0000 7f23577fd740 1 Processor -- start 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.141+0000 7f23577fd740 1 -- start start 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.141+0000 7f23577fd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d43680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.141+0000 7f23577fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d4f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.141+0000 7f23577fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d4fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f23577fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23501afe30 con 0x5561d3d2d2d0 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f23577fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b1970 con 0x5561d3d37b90 2024-04-25T19:12:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f23577fd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b0500 con 0x5561d3d2c9b0 2024-04-25T19:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d43680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f2345400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d4f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d43680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36000/0 (socket says 172.21.15.86:36000) 2024-04-25T19:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 -- 172.21.15.86:0/3124829381 learned_addr learned my addr 172.21.15.86:0/3124829381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234e800640 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d4fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 -- 172.21.15.86:0/3124829381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 msgr2=0x5561d3d4fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d4fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 -- 172.21.15.86:0/3124829381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 msgr2=0x5561d3d4f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d4f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234de00640 1 -- 172.21.15.86:0/3124829381 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d3d4a4c0 con 0x5561d3d2c9b0 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.142+0000 7f234e800640 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d4fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.143+0000 7f234de00640 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d43680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2340009380 tx=0x7f2340019410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918407129fcae3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.143+0000 7f233ea00640 1 -- 172.21.15.86:0/3124829381 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340016070 con 0x5561d3d2c9b0 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.143+0000 7f233ea00640 1 -- 172.21.15.86:0/3124829381 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2340005070 con 0x5561d3d2c9b0 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.143+0000 7f233ea00640 1 -- 172.21.15.86:0/3124829381 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234001e030 con 0x5561d3d2c9b0 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.143+0000 7f23577fd740 1 -- 172.21.15.86:0/3124829381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 msgr2=0x5561d3d43680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.143+0000 7f23577fd740 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d43680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2340009380 tx=0x7f2340019410 comp rx=0 tx=0).stop 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 -- 172.21.15.86:0/3124829381 shutdown_connections 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d37b90 0x5561d3d4fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d4f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 --2- 172.21.15.86:0/3124829381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d2c9b0 0x5561d3d43680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 -- 172.21.15.86:0/3124829381 >> 172.21.15.86:0/3124829381 conn(0x5561d3d28ec0 msgr2=0x5561d3d2fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 -- 172.21.15.86:0/3124829381 shutdown_connections 2024-04-25T19:12:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 -- 172.21.15.86:0/3124829381 wait complete. 2024-04-25T19:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 -- start start 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.144+0000 7f23577fd740 1 Processor -- start 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 0x5561d3d3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d3cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b12a0 con 0x5561d3d2d2d0 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b0bd0 con 0x5561d3d2c9b0 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23501ade30 con 0x5561d3d37b90 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f23577fd740 10 client.0 _ll_register_callbacks cb 0x7f23480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 0x5561d3d3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f2345400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d3cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 0x5561d3d3fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:36920/0 (socket says 172.21.15.86:36920) 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 -- 172.21.15.86:0/2604017108 learned_addr learned my addr 172.21.15.86:0/2604017108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234e800640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 -- 172.21.15.86:0/2604017108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 msgr2=0x5561d3d3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d3d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 -- 172.21.15.86:0/2604017108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 msgr2=0x5561d3d3cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d3cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f234de00640 1 -- 172.21.15.86:0/2604017108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d3d3d8f0 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.145+0000 7f2345400640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d3cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f234de00640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 0x5561d3d3fff0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2340019560 tx=0x7f23400198f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8ef1a524f99a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f2333e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f2333e00640 1 -- 172.21.15.86:0/2604017108 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340016040 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561d3ca0320 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f2333e00640 1 -- 172.21.15.86:0/2604017108 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8637+0+0 (secure 0 0 0) 0x7f2340019c30 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f2333e00640 1 -- 172.21.15.86:0/2604017108 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2340040050 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f23577fd740 10 client.26339 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.146+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561d3dad240 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f2333e00640 1 -- 172.21.15.86:0/2604017108 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5561d3ca0320 con 0x5561d3d2c9b0 2024-04-25T19:12:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f2333e00640 1 -- 172.21.15.86:0/2604017108 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f234003b030 con 0x5561d3d2c9b0 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f23577fd740 10 client.26339 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f23577fd740 10 client.26339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f23577fd740 20 client.26339 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f23577fd740 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3dad950 0x5561d3dafd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f23577fd740 4 client.26339 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561d3c9a190 con 0x5561d3dad950 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.147+0000 7f2345400640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3dad950 0x5561d3dafd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.148+0000 7f2345400640 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3dad950 0x5561d3dafd40 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c97c5c24e140d7b server_cookie=ecdad725f35e6813 in_seq=0 out_seq=0 2024-04-25T19:12:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.148+0000 7f2333e00640 10 client.26339 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.162+0000 7f2333e00640 1 -- 172.21.15.86:0/2604017108 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561d3c9a190 con 0x5561d3dad950 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.162+0000 7f2333e00640 10 client.26339 handle_command_reply: tid=0 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 client.26339 shutdown 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 msgr2=0x5561d3d3fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 0x5561d3d3fff0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2340019560 tx=0x7f23400198f0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 shutdown_connections 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5561d3dad950 0x5561d3dafd40 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561d3d37b90 0x5561d3d3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561d3d2d2d0 0x5561d3d3cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f2333e00640 0 client.26339 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 --2- 172.21.15.86:0/2604017108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5561d3d2c9b0 0x5561d3d3fff0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 >> 172.21.15.86:0/2604017108 conn(0x5561d3d28ec0 msgr2=0x5561d3d2e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 shutdown_connections 2024-04-25T19:12:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 1 -- 172.21.15.86:0/2604017108 wait complete. 2024-04-25T19:12:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.163+0000 7f23577fd740 20 client.26339 trim_cache size 0 max 16384 2024-04-25T19:12:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:12:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:12:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:12:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f23340a3d10 msgr2=0x7f23340a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f23340a3d10 0x7f23340a61d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2338031440 tx=0x7f2338005780 comp rx=0 tx=0).stop 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 msgr2=0x7f235019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 0x7f235019a590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2348009fd0 tx=0x7f234800ef10 comp rx=0 tx=0).stop 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 shutdown_connections 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f23340a3d10 0x7f23340a61d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f235010e590 0x7f235019a590 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2350074ca0 0x7f23501a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 --2- 172.21.15.86:0/454111459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2350074380 0x7f23501a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 >> 172.21.15.86:0/454111459 conn(0x7f23500fd8a0 msgr2=0x7f23500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 shutdown_connections 2024-04-25T19:12:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:01.164+0000 7f234f200640 1 -- 172.21.15.86:0/454111459 wait complete. 2024-04-25T19:12:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:02 smithi086 ceph-mon[27783]: pgmap v281: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:12:02.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:02 smithi183 ceph-mon[32344]: pgmap v281: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:12:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:02 smithi194 ceph-mon[32740]: pgmap v281: 97 pgs: 97 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:12:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:04 smithi086 ceph-mon[27783]: pgmap v282: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 111 KiB/s wr, 41 op/s 2024-04-25T19:12:04.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:04 smithi183 ceph-mon[32344]: pgmap v282: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 111 KiB/s wr, 41 op/s 2024-04-25T19:12:04.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:04 smithi194 ceph-mon[32740]: pgmap v282: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 111 KiB/s wr, 41 op/s 2024-04-25T19:12:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:06 smithi086 ceph-mon[27783]: pgmap v283: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-25T19:12:06.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:06 smithi183 ceph-mon[32344]: pgmap v283: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-25T19:12:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:06 smithi194 ceph-mon[32740]: pgmap v283: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-25T19:12:07.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:07 smithi183 ceph-mon[32344]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T19:12:07.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:07 smithi194 ceph-mon[32740]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T19:12:08.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:07 smithi086 ceph-mon[27783]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T19:12:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:08 smithi183 ceph-mon[32344]: pgmap v285: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-25T19:12:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:08 smithi194 ceph-mon[32740]: pgmap v285: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-25T19:12:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:08 smithi086 ceph-mon[27783]: pgmap v285: 97 pgs: 97 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-25T19:12:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:10 smithi183 ceph-mon[32344]: pgmap v286: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-25T19:12:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:10 smithi194 ceph-mon[32740]: pgmap v286: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-25T19:12:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:10 smithi086 ceph-mon[27783]: pgmap v286: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-25T19:12:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:12 smithi183 ceph-mon[32344]: pgmap v287: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-25T19:12:12.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:12 smithi194 ceph-mon[32740]: pgmap v287: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-25T19:12:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:12 smithi086 ceph-mon[27783]: pgmap v287: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-25T19:12:13.988 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-25T19:12:13.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:12:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 -- 172.21.15.86:0/1053282876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 msgr2=0x7fbc0810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 --2- 172.21.15.86:0/1053282876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 0x7fbc0810ea00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009900 tx=0x7fbbfc02a410 comp rx=0 tx=0).stop 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 -- 172.21.15.86:0/1053282876 shutdown_connections 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 --2- 172.21.15.86:0/1053282876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc08108f20 0x7fbc081153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 --2- 172.21.15.86:0/1053282876 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108560 0x7fbc081089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 --2- 172.21.15.86:0/1053282876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 0x7fbc0810ea00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 -- 172.21.15.86:0/1053282876 >> 172.21.15.86:0/1053282876 conn(0x7fbc080764e0 msgr2=0x7fbc08078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 -- 172.21.15.86:0/1053282876 shutdown_connections 2024-04-25T19:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.299+0000 7fbc10000640 1 -- 172.21.15.86:0/1053282876 wait complete. 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 Processor -- start 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 -- start start 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc08108560 0x7fbc081a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 0x7fbc081a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 0x7fbc0819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08118800 con 0x7fbc0810e620 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08118680 con 0x7fbc08108560 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.300+0000 7fbc10000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08118980 con 0x7fbc08108f20 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 0x7fbc0819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 0x7fbc081a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 0x7fbc081a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47636/0 (socket says 172.21.15.86:47636) 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 -- 172.21.15.86:0/2568957425 learned_addr learned my addr 172.21.15.86:0/2568957425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0d600640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc08108560 0x7fbc081a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 -- 172.21.15.86:0/2568957425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc08108560 msgr2=0x7fbc081a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc08108560 0x7fbc081a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 -- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 msgr2=0x7fbc0819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 0x7fbc0819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 -- 172.21.15.86:0/2568957425 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0819f1e0 con 0x7fbc08108f20 2024-04-25T19:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.301+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 0x7fbc081a4df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e9f0 tx=0x7fbbf400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.302+0000 7fbc06000640 1 -- 172.21.15.86:0/2568957425 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400cd80 con 0x7fbc08108f20 2024-04-25T19:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.302+0000 7fbc06000640 1 -- 172.21.15.86:0/2568957425 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf40054d0 con 0x7fbc08108f20 2024-04-25T19:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.302+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0819f4d0 con 0x7fbc08108f20 2024-04-25T19:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.302+0000 7fbc06000640 1 -- 172.21.15.86:0/2568957425 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4010690 con 0x7fbc08108f20 2024-04-25T19:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.302+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc081ab770 con 0x7fbc08108f20 2024-04-25T19:12:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.306+0000 7fbc12d86740 1 Processor -- start 2024-04-25T19:12:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.306+0000 7fbc12d86740 1 -- start start 2024-04-25T19:12:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.306+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368234c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559368235140 0x55936823f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55936823fa00 0x559368241df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc12d86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55936819a900 con 0x559368235140 2024-04-25T19:12:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc12d86740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559368140ef0 con 0x559368234820 2024-04-25T19:12:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc12d86740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5593681968c0 con 0x55936823fa00 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc06000640 1 -- 172.21.15.86:0/2568957425 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbf4004e90 con 0x7fbc08108f20 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368234c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368234c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48070/0 (socket says 172.21.15.86:48070) 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 -- 172.21.15.86:0/1230507304 learned_addr learned my addr 172.21.15.86:0/1230507304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0d600640 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559368235140 0x55936823f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55936823fa00 0x559368241df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 -- 172.21.15.86:0/1230507304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55936823fa00 msgr2=0x559368241df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55936823fa00 0x559368241df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 -- 172.21.15.86:0/1230507304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559368235140 msgr2=0x55936823f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559368235140 0x55936823f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0e000640 1 -- 172.21.15.86:0/1230507304 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559367ead210 con 0x559368234820 2024-04-25T19:12:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.307+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55936823fa00 0x559368241df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:12:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.308+0000 7fbc0e000640 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368234c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009fd0 tx=0x7fbc000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b03e953af88802e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.308+0000 7fbbfa600640 1 -- 172.21.15.86:0/1230507304 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00004d20 con 0x559368234820 2024-04-25T19:12:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.308+0000 7fbc06000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbbf00a3dc0 0x7fbbf00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.308+0000 7fbbfa600640 1 -- 172.21.15.86:0/1230507304 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc00004050 con 0x559368234820 2024-04-25T19:12:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.308+0000 7fbbfa600640 1 -- 172.21.15.86:0/1230507304 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00005260 con 0x559368234820 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.308+0000 7fbc06000640 1 -- 172.21.15.86:0/2568957425 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fbbf40d0c60 con 0x7fbc08108f20 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc0d600640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbbf00a3dc0 0x7fbbf00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 -- 172.21.15.86:0/1230507304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 msgr2=0x559368234c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368234c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009fd0 tx=0x7fbc000118d0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 -- 172.21.15.86:0/1230507304 shutdown_connections 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55936823fa00 0x559368241df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559368235140 0x55936823f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 --2- 172.21.15.86:0/1230507304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368234c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc12d86740 1 -- 172.21.15.86:0/1230507304 >> 172.21.15.86:0/1230507304 conn(0x559368230d30 msgr2=0x559368231110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.309+0000 7fbc0d600640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbbf00a3dc0 0x7fbbf00a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006010 tx=0x7fbbfc005e80 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.310+0000 7fbc12d86740 1 -- 172.21.15.86:0/1230507304 shutdown_connections 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.310+0000 7fbc12d86740 1 -- 172.21.15.86:0/1230507304 wait complete. 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.310+0000 7fbc12d86740 1 -- start start 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.310+0000 7fbc12d86740 1 Processor -- start 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936823c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x55936823cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559368242630 con 0x55936823fa00 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559368154310 con 0x559368235140 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5593682427b0 con 0x559368234820 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc12d86740 10 client.0 _ll_register_callbacks cb 0x559368122cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x55936823cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936823c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x55936823cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48082/0 (socket says 172.21.15.86:48082) 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 -- 172.21.15.86:0/3463943617 learned_addr learned my addr 172.21.15.86:0/3463943617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 -- 172.21.15.86:0/3463943617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 msgr2=0x55936823c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936823c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 -- 172.21.15.86:0/3463943617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 msgr2=0x55936823d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0cc00640 1 -- 172.21.15.86:0/3463943617 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559367f79620 con 0x559368235140 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.311+0000 7fbc0e000640 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x55936823cef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40ce3a0 tx=0x7fbbf409c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374fee3363fff16c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbbeea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbbeea00640 1 -- 172.21.15.86:0/3463943617 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf409d290 con 0x559368235140 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbbeea00640 1 -- 172.21.15.86:0/3463943617 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fbbf40ce5c0 con 0x559368235140 2024-04-25T19:12:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc00004050 con 0x559368235140 2024-04-25T19:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbbeea00640 1 -- 172.21.15.86:0/3463943617 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbf40ce7c0 con 0x559368235140 2024-04-25T19:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbc12d86740 10 client.26351 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55936824e900 con 0x559368235140 2024-04-25T19:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.312+0000 7fbbeea00640 1 -- 172.21.15.86:0/3463943617 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc00004050 con 0x559368235140 2024-04-25T19:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbbeea00640 1 -- 172.21.15.86:0/3463943617 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbbf40a5370 con 0x559368235140 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc12d86740 10 client.26351 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc12d86740 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc12d86740 20 client.26351 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55936824fdc0 0x5593682521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc12d86740 4 client.26351 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593681ae8b0 con 0x55936824fdc0 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.313+0000 7fbc0d600640 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55936824fdc0 0x5593682521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.314+0000 7fbc0d600640 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55936824fdc0 0x5593682521b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33507af70430a855 server_cookie=21221ff78a398954 in_seq=0 out_seq=0 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.314+0000 7fbbeea00640 10 client.26351 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbbeea00640 1 -- 172.21.15.86:0/3463943617 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593681ae8b0 con 0x55936824fdc0 2024-04-25T19:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbbeea00640 10 client.26351 handle_command_reply: tid=0 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 client.26351 shutdown 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 msgr2=0x55936823cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x55936823cef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40ce3a0 tx=0x7fbbf409c790 comp rx=0 tx=0).stop 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 shutdown_connections 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55936824fdc0 0x5593682521b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x55936823cef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3463943617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936823c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbbeea00640 0 client.26351 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 >> 172.21.15.86:0/3463943617 conn(0x559368230d30 msgr2=0x559368236ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 shutdown_connections 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 1 -- 172.21.15.86:0/3463943617 wait complete. 2024-04-25T19:12:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.315+0000 7fbc12d86740 20 client.26351 trim_cache size 0 max 16384 2024-04-25T19:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.332+0000 7fbc12d86740 1 Processor -- start 2024-04-25T19:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.332+0000 7fbc12d86740 1 -- start start 2024-04-25T19:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936824b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x559368230610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc12d86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b42e0 con 0x55936823fa00 2024-04-25T19:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc12d86740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b5e20 con 0x559368235140 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc12d86740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b49b0 con 0x559368234820 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x559368230610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc0d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936824b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x559368230610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48088/0 (socket says 172.21.15.86:48088) 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.333+0000 7fbc0cc00640 1 -- 172.21.15.86:0/3268250675 learned_addr learned my addr 172.21.15.86:0/3268250675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0d600640 1 -- 172.21.15.86:0/3268250675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 msgr2=0x559368230610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0d600640 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x559368230610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0d600640 1 -- 172.21.15.86:0/3268250675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 msgr2=0x55936823d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0d600640 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0d600640 1 -- 172.21.15.86:0/3268250675 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936823b9c0 con 0x559368234820 2024-04-25T19:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x559368230610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbc0d600640 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936824b4f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc02aa00 tx=0x7fbbfc03f070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca39c1c2030325e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.334+0000 7fbbf9c00640 1 -- 172.21.15.86:0/3268250675 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc035040 con 0x559368234820 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.335+0000 7fbbf9c00640 1 -- 172.21.15.86:0/3268250675 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbfc0046c0 con 0x559368234820 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.335+0000 7fbbf9c00640 1 -- 172.21.15.86:0/3268250675 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc003850 con 0x559368234820 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 -- 172.21.15.86:0/3268250675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 msgr2=0x55936824b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936824b4f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc02aa00 tx=0x7fbbfc03f070 comp rx=0 tx=0).stop 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 -- 172.21.15.86:0/3268250675 shutdown_connections 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x55936823d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368235140 0x559368230610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 --2- 172.21.15.86:0/3268250675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368234820 0x55936824b4f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 -- 172.21.15.86:0/3268250675 >> 172.21.15.86:0/3268250675 conn(0x559368230d30 msgr2=0x559368231110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 -- 172.21.15.86:0/3268250675 shutdown_connections 2024-04-25T19:12:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 -- 172.21.15.86:0/3268250675 wait complete. 2024-04-25T19:12:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.336+0000 7fbc12d86740 1 -- start start 2024-04-25T19:12:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 Processor -- start 2024-04-25T19:12:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368245560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368235140 0x559368245aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x559368245fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b5750 con 0x55936823fa00 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b5080 con 0x559368234820 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081b22e0 con 0x559368235140 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.337+0000 7fbc12d86740 10 client.0 _ll_register_callbacks cb 0x7fbbf4098ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x559368245fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x559368245fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56936/0 (socket says 172.21.15.86:56936) 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0e000640 1 -- 172.21.15.86:0/446300096 learned_addr learned my addr 172.21.15.86:0/446300096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0d600640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368245560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368235140 0x559368245aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0d600640 1 -- 172.21.15.86:0/446300096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368235140 msgr2=0x559368245aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0d600640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368235140 0x559368245aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0d600640 1 -- 172.21.15.86:0/446300096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 msgr2=0x559368245fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0d600640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x559368245fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0d600640 1 -- 172.21.15.86:0/446300096 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593682b5430 con 0x559368234820 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368235140 0x559368245aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.338+0000 7fbc0e000640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x559368245fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbc0d600640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368245560 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc02aa00 tx=0x7fbbfc009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=815241e2f980432a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbbeea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbbeea00640 1 -- 172.21.15.86:0/446300096 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc054800 con 0x559368234820 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbbeea00640 1 -- 172.21.15.86:0/446300096 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fbbfc004870 con 0x559368234820 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbfc0046c0 con 0x559368234820 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbbeea00640 1 -- 172.21.15.86:0/446300096 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbfc004a90 con 0x559368234820 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbc12d86740 10 client.26357 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.339+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5593682b6720 con 0x559368234820 2024-04-25T19:12:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbbeea00640 1 -- 172.21.15.86:0/446300096 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbfc0046c0 con 0x559368234820 2024-04-25T19:12:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbbeea00640 1 -- 172.21.15.86:0/446300096 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbbfc03d450 con 0x559368234820 2024-04-25T19:12:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbc12d86740 10 client.26357 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbc12d86740 10 client.26357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbc12d86740 20 client.26357 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbc12d86740 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5593682b6e00 0x5593682b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbc12d86740 4 client.26357 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.340+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593681a8320 con 0x5593682b6e00 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.341+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5593682b6e00 0x5593682b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.341+0000 7fbc0cc00640 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5593682b6e00 0x5593682b9220 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0f457102f43941 server_cookie=e714b31fe3a8a4c5 in_seq=0 out_seq=0 2024-04-25T19:12:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.341+0000 7fbbeea00640 10 client.26357 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbbeea00640 1 -- 172.21.15.86:0/446300096 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593681a8320 con 0x5593682b6e00 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbbeea00640 10 client.26357 handle_command_reply: tid=0 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 client.26357 shutdown 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 msgr2=0x559368245560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368245560 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc02aa00 tx=0x7fbbfc009880 comp rx=0 tx=0).stop 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 shutdown_connections 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5593682b6e00 0x5593682b9220 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936823fa00 0x559368245fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559368235140 0x559368245aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 --2- 172.21.15.86:0/446300096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559368234820 0x559368245560 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbbeea00640 0 client.26357 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 >> 172.21.15.86:0/446300096 conn(0x559368230d30 msgr2=0x559368236a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 shutdown_connections 2024-04-25T19:12:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 1 -- 172.21.15.86:0/446300096 wait complete. 2024-04-25T19:12:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.357+0000 7fbc12d86740 20 client.26357 trim_cache size 0 max 16384 2024-04-25T19:12:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:12:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:12:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:12:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbbf00a3dc0 msgr2=0x7fbbf00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbbf00a3dc0 0x7fbbf00a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc006010 tx=0x7fbbfc005e80 comp rx=0 tx=0).stop 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 msgr2=0x7fbc081a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 0x7fbc081a4df0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e9f0 tx=0x7fbbf400edf0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 shutdown_connections 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbbf00a3dc0 0x7fbbf00a6280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc0810e620 0x7fbc0819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc08108f20 0x7fbc081a4df0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 --2- 172.21.15.86:0/2568957425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc08108560 0x7fbc081a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 >> 172.21.15.86:0/2568957425 conn(0x7fbc080764e0 msgr2=0x7fbc08077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 shutdown_connections 2024-04-25T19:12:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:14.358+0000 7fbc10000640 1 -- 172.21.15.86:0/2568957425 wait complete. 2024-04-25T19:12:14.440 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:12:14.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:14 smithi194 ceph-mon[32740]: pgmap v288: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-25T19:12:14.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:14 smithi194 ceph-mon[32740]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T19:12:14.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:14 smithi086 ceph-mon[27783]: pgmap v288: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-25T19:12:14.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:14 smithi086 ceph-mon[27783]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T19:12:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:14 smithi183 ceph-mon[32344]: pgmap v288: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-25T19:12:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:14 smithi183 ceph-mon[32344]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T19:12:16.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 -- 172.21.15.86:0/1841068925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18103450 msgr2=0x7f6b1810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:16.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 --2- 172.21.15.86:0/1841068925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18103450 0x7f6b1810f8b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000b0a0 tx=0x7f6b1000b3b0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.634 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:16 smithi086 ceph-mon[27783]: pgmap v290: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:16.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 -- 172.21.15.86:0/1841068925 shutdown_connections 2024-04-25T19:12:16.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 --2- 172.21.15.86:0/1841068925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18103450 0x7f6b1810f8b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 --2- 172.21.15.86:0/1841068925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18108a50 0x7f6b18108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 --2- 172.21.15.86:0/1841068925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18102a90 0x7f6b18102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 -- 172.21.15.86:0/1841068925 >> 172.21.15.86:0/1841068925 conn(0x7f6b18079ba0 msgr2=0x7f6b18079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:16.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 -- 172.21.15.86:0/1841068925 shutdown_connections 2024-04-25T19:12:16.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.630+0000 7f6b20600700 1 -- 172.21.15.86:0/1841068925 wait complete. 2024-04-25T19:12:16.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.631+0000 7f6b20600700 1 Processor -- start 2024-04-25T19:12:16.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 -- start start 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18102a90 0x7f6b18199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 0x7f6b18199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18108a50 0x7f6b1819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18112c20 con 0x7f6b18102a90 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18112aa0 con 0x7f6b18108a50 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.632+0000 7f6b20600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18112da0 con 0x7f6b18103450 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 0x7f6b18199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:16.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 0x7f6b18199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47694/0 (socket says 172.21.15.86:47694) 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 -- 172.21.15.86:0/1459030357 learned_addr learned my addr 172.21.15.86:0/1459030357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1dc00700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18102a90 0x7f6b18199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1e600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18108a50 0x7f6b1819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 -- 172.21.15.86:0/1459030357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18108a50 msgr2=0x7f6b1819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18108a50 0x7f6b1819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 -- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18102a90 msgr2=0x7f6b18199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18102a90 0x7f6b18199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1d200700 1 -- 172.21.15.86:0/1459030357 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1819e140 con 0x7f6b18103450 2024-04-25T19:12:16.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1e600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18108a50 0x7f6b1819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.633+0000 7f6b1dc00700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18102a90 0x7f6b18199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.634+0000 7f6b1d200700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 0x7f6b18199670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400e9c0 tx=0x7f6b0400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.634+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400cbd0 con 0x7f6b18103450 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.634+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0400ee60 con 0x7f6b18103450 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.634+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1819e430 con 0x7f6b18103450 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.634+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b040055c0 con 0x7f6b18103450 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.634+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b18074c70 con 0x7f6b18103450 2024-04-25T19:12:16.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.636+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b18069730 con 0x7f6b18103450 2024-04-25T19:12:16.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.638+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b04004750 con 0x7f6b18103450 2024-04-25T19:12:16.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.640+0000 7f6b16000700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6b000a3db0 0x7f6b000a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:16.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.640+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f6b040d0b90 con 0x7f6b18103450 2024-04-25T19:12:16.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.640+0000 7f6b1dc00700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6b000a3db0 0x7f6b000a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:16.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.641+0000 7f6b1dc00700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6b000a3db0 0x7f6b000a6270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009fd0 tx=0x7f6b0c009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:16.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.643+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b0409d160 con 0x7f6b18103450 2024-04-25T19:12:16.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:16 smithi194 ceph-mon[32740]: pgmap v290: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:16 smithi183 ceph-mon[32344]: pgmap v290: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.917+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1819a490 con 0x7f6b18103450 2024-04-25T19:12:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.918+0000 7f6b16000700 1 -- 172.21.15.86:0/1459030357 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f6b0409ca60 con 0x7f6b18103450 2024-04-25T19:12:16.921 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:12:16.921 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:12:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.921+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6b000a3db0 msgr2=0x7f6b000a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.921+0000 7f6b20600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6b000a3db0 0x7f6b000a6270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009fd0 tx=0x7f6b0c009260 comp rx=0 tx=0).stop 2024-04-25T19:12:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.921+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 msgr2=0x7f6b18199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.921+0000 7f6b20600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 0x7f6b18199670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400e9c0 tx=0x7f6b0400ecd0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 shutdown_connections 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b18108a50 0x7f6b1819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b18102a90 0x7f6b18199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6b000a3db0 0x7f6b000a6270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 --2- 172.21.15.86:0/1459030357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b18103450 0x7f6b18199670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 >> 172.21.15.86:0/1459030357 conn(0x7f6b18079ba0 msgr2=0x7f6b18101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 shutdown_connections 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:16.922+0000 7f6b20600700 1 -- 172.21.15.86:0/1459030357 wait complete. 2024-04-25T19:12:16.925 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:12:17.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:17 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1459030357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:12:17.723 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-25T19:12:17.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-25T19:12:17.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:17 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1459030357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:12:17.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:17 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1459030357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:12:18.165 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:12:18.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:18 smithi086 ceph-mon[27783]: pgmap v291: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-25T19:12:18.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:18 smithi194 ceph-mon[32740]: pgmap v291: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-25T19:12:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:18 smithi183 ceph-mon[32344]: pgmap v291: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-25T19:12:19.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:19 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:12:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:19 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:12:19.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:19 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:12:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.434+0000 7fb2a4800700 1 -- 172.21.15.86:0/3197097703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 msgr2=0x7fb29c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.434+0000 7fb2a4800700 1 --2- 172.21.15.86:0/3197097703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c108e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb2900098f0 tx=0x7fb290009c00 comp rx=0 tx=0).stop 2024-04-25T19:12:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 -- 172.21.15.86:0/3197097703 shutdown_connections 2024-04-25T19:12:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 --2- 172.21.15.86:0/3197097703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 --2- 172.21.15.86:0/3197097703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c108e60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 --2- 172.21.15.86:0/3197097703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 0x7fb29c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 -- 172.21.15.86:0/3197097703 >> 172.21.15.86:0/3197097703 conn(0x7fb29c079b20 msgr2=0x7fb29c079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:20.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 -- 172.21.15.86:0/3197097703 shutdown_connections 2024-04-25T19:12:20.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.435+0000 7fb2a4800700 1 -- 172.21.15.86:0/3197097703 wait complete. 2024-04-25T19:12:20.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.436+0000 7fb2a4800700 1 Processor -- start 2024-04-25T19:12:20.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 -- start start 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 0x7fb29c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112c60 con 0x7fb29c102af0 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112ae0 con 0x7fb29c1034b0 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a4800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112de0 con 0x7fb29c108a80 2024-04-25T19:12:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.437+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 0x7fb29c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50452/0 (socket says 172.21.15.86:50452) 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 -- 172.21.15.86:0/828587544 learned_addr learned my addr 172.21.15.86:0/828587544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 -- 172.21.15.86:0/828587544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 msgr2=0x7fb29c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 -- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 msgr2=0x7fb29c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 0x7fb29c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1400700 1 -- 172.21.15.86:0/828587544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c19e180 con 0x7fb29c1034b0 2024-04-25T19:12:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a2800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.438+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 0x7fb29c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.439+0000 7fb2a1400700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c1996b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e9c0 tx=0x7fb28c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.439+0000 7fb296a00700 1 -- 172.21.15.86:0/828587544 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c004a30 con 0x7fb29c1034b0 2024-04-25T19:12:20.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.439+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c19e470 con 0x7fb29c1034b0 2024-04-25T19:12:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.439+0000 7fb296a00700 1 -- 172.21.15.86:0/828587544 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c00cc70 con 0x7fb29c1034b0 2024-04-25T19:12:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.439+0000 7fb296a00700 1 -- 172.21.15.86:0/828587544 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0054a0 con 0x7fb29c1034b0 2024-04-25T19:12:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.439+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1a5cf0 con 0x7fb29c1034b0 2024-04-25T19:12:20.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.442+0000 7fb296a00700 1 -- 172.21.15.86:0/828587544 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb28c021440 con 0x7fb29c1034b0 2024-04-25T19:12:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 Processor -- start 2024-04-25T19:12:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb296a00700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2840a3e50 0x7fb2840a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 -- start start 2024-04-25T19:12:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb296a00700 1 -- 172.21.15.86:0/828587544 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fb28c0d09b0 con 0x7fb29c1034b0 2024-04-25T19:12:20.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2a2800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2840a3e50 0x7fb2840a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 0x563a68f5ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f6e590 0x563a68f70980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a68eb2020 con 0x563a68f6e590 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.445+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a68e588c0 con 0x563a68f5e9b0 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a68eadfe0 con 0x563a68f5f2d0 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 0x563a68f5ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 0x563a68f5ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50468/0 (socket says 172.21.15.86:50468) 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 -- 172.21.15.86:0/3708932765 learned_addr learned my addr 172.21.15.86:0/3708932765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a2800700 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f6e590 0x563a68f70980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 -- 172.21.15.86:0/3708932765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 msgr2=0x563a68f6e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 -- 172.21.15.86:0/3708932765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f6e590 msgr2=0x563a68f70980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f6e590 0x563a68f70980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a2800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2840a3e50 0x7fb2840a6310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb298003e80 tx=0x7fb298009920 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1e00700 1 -- 172.21.15.86:0/3708932765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a68e4e810 con 0x563a68f5e9b0 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.446+0000 7fb2a1400700 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.447+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 0x563a68f5ed90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb290009230 tx=0x7fb29000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e82e4252ccdde9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.447+0000 7fb28b400700 1 -- 172.21.15.86:0/3708932765 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29000f6e0 con 0x563a68f5e9b0 2024-04-25T19:12:20.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.447+0000 7fb28b400700 1 -- 172.21.15.86:0/3708932765 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb29000f880 con 0x563a68f5e9b0 2024-04-25T19:12:20.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.447+0000 7fb28b400700 1 -- 172.21.15.86:0/3708932765 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290004c70 con 0x563a68f5e9b0 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 -- 172.21.15.86:0/3708932765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 msgr2=0x563a68f5ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 0x563a68f5ed90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb290009230 tx=0x7fb29000bf60 comp rx=0 tx=0).stop 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 -- 172.21.15.86:0/3708932765 shutdown_connections 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f6e590 0x563a68f70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f5e9b0 0x563a68f5ed90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/3708932765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 -- 172.21.15.86:0/3708932765 >> 172.21.15.86:0/3708932765 conn(0x563a68e8d730 msgr2=0x563a68f5ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 -- 172.21.15.86:0/3708932765 shutdown_connections 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.448+0000 7fb2af1efb80 1 -- 172.21.15.86:0/3708932765 wait complete. 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.449+0000 7fb2af1efb80 1 -- start start 2024-04-25T19:12:20.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.449+0000 7fb2af1efb80 1 Processor -- start 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f6dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f6b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a68f71050 con 0x563a68f5e9b0 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a68e6b9f0 con 0x563a68f6e590 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a68f711d0 con 0x563a68f5f2d0 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2af1efb80 10 client.0 _ll_register_callbacks cb 0x563a68e4a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f6dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f6b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f6dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60368/0 (socket says 172.21.15.86:60368) 2024-04-25T19:12:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a1e00700 1 -- 172.21.15.86:0/2502775267 learned_addr learned my addr 172.21.15.86:0/2502775267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a1400700 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a2800700 1 -- 172.21.15.86:0/2502775267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 msgr2=0x563a68f6afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a2800700 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a2800700 1 -- 172.21.15.86:0/2502775267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 msgr2=0x563a68f6dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a2800700 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f6dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.450+0000 7fb2a2800700 1 -- 172.21.15.86:0/2502775267 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a68dffa60 con 0x563a68f6e590 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb2a2800700 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f6b500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb298009fd0 tx=0x7fb298004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e971e22d41ac2999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb27fe00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb27fe00700 1 -- 172.21.15.86:0/2502775267 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298026020 con 0x563a68f6e590 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29000f880 con 0x563a68f6e590 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb27fe00700 1 -- 172.21.15.86:0/2502775267 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fb298019ac0 con 0x563a68f6e590 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb27fe00700 1 -- 172.21.15.86:0/2502775267 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb298031050 con 0x563a68f6e590 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb2af1efb80 10 client.26375 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.451+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a68e48c80 con 0x563a68f6e590 2024-04-25T19:12:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb27fe00700 1 -- 172.21.15.86:0/2502775267 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb29000f880 con 0x563a68f6e590 2024-04-25T19:12:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb27fe00700 1 -- 172.21.15.86:0/2502775267 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb298022cd0 con 0x563a68f6e590 2024-04-25T19:12:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb2af1efb80 10 client.26375 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb2af1efb80 10 client.26375 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:12:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb2af1efb80 20 client.26375 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68f7dd20 0x563a68f80130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb2af1efb80 4 client.26375 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.452+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a68ebf810 con 0x563a68f7dd20 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.453+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68f7dd20 0x563a68f80130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.453+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68f7dd20 0x563a68f80130 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8e048c23635fa server_cookie=e6f6d4569205638d in_seq=0 out_seq=0 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.453+0000 7fb27fe00700 10 client.26375 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb27fe00700 1 -- 172.21.15.86:0/2502775267 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a68ebf810 con 0x563a68f7dd20 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb27fe00700 10 client.26375 handle_command_reply: tid=0 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 client.26375 shutdown 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 msgr2=0x563a68f6b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f6b500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb298009fd0 tx=0x7fb298004b40 comp rx=0 tx=0).stop 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 shutdown_connections 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68f7dd20 0x563a68f80130 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f6b500 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f6dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/2502775267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb27fe00700 0 client.26375 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 >> 172.21.15.86:0/2502775267 conn(0x563a68e8d730 msgr2=0x563a68f5aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 shutdown_connections 2024-04-25T19:12:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.454+0000 7fb2af1efb80 1 -- 172.21.15.86:0/2502775267 wait complete. 2024-04-25T19:12:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.455+0000 7fb2af1efb80 20 client.26375 trim_cache size 0 max 16384 2024-04-25T19:12:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 Processor -- start 2024-04-25T19:12:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 -- start start 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f85a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ab360 con 0x563a68f5e9b0 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1acea0 con 0x563a68f6e590 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.477+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1aba30 con 0x563a68f5f2d0 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f85a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f85a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:50474/0 (socket says 172.21.15.86:50474) 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 -- 172.21.15.86:0/912295724 learned_addr learned my addr 172.21.15.86:0/912295724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a1400700 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 -- 172.21.15.86:0/912295724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 msgr2=0x563a68f6d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 -- 172.21.15.86:0/912295724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 msgr2=0x563a68f79f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f79f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 -- 172.21.15.86:0/912295724 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a68f807b0 con 0x563a68f6e590 2024-04-25T19:12:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f79f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:12:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.478+0000 7fb2a2800700 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f85a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb298003d40 tx=0x7fb298009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e42241897d2547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.479+0000 7fb28aa00700 1 -- 172.21.15.86:0/912295724 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298026050 con 0x563a68f6e590 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.479+0000 7fb28aa00700 1 -- 172.21.15.86:0/912295724 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298019df0 con 0x563a68f6e590 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.479+0000 7fb28aa00700 1 -- 172.21.15.86:0/912295724 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298017e30 con 0x563a68f6e590 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.479+0000 7fb2af1efb80 1 -- 172.21.15.86:0/912295724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 msgr2=0x563a68f85a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f85a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb298003d40 tx=0x7fb298009f10 comp rx=0 tx=0).stop 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 -- 172.21.15.86:0/912295724 shutdown_connections 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f85a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f79f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/912295724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f6d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 -- 172.21.15.86:0/912295724 >> 172.21.15.86:0/912295724 conn(0x563a68e8d730 msgr2=0x563a68f66660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 -- 172.21.15.86:0/912295724 shutdown_connections 2024-04-25T19:12:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 -- 172.21.15.86:0/912295724 wait complete. 2024-04-25T19:12:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 -- start start 2024-04-25T19:12:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.480+0000 7fb2af1efb80 1 Processor -- start 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f73a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ac7d0 con 0x563a68f5e9b0 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ac100 con 0x563a68f6e590 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1a9320 con 0x563a68f5f2d0 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2af1efb80 10 client.0 _ll_register_callbacks cb 0x563a68e4a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f72fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60386/0 (socket says 172.21.15.86:60386) 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2a1e00700 1 -- 172.21.15.86:0/4206835562 learned_addr learned my addr 172.21.15.86:0/4206835562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.481+0000 7fb2a2800700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f73a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a1400700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a2800700 1 -- 172.21.15.86:0/4206835562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 msgr2=0x563a68f734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a2800700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a2800700 1 -- 172.21.15.86:0/4206835562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 msgr2=0x563a68f72fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a2800700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f72fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a2800700 1 -- 172.21.15.86:0/4206835562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a68fe43f0 con 0x563a68f6e590 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2a2800700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f73a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb298000f80 tx=0x7fb298017830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28b3dbc7cdaa608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb27fe00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb27fe00700 1 -- 172.21.15.86:0/4206835562 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298042030 con 0x563a68f6e590 2024-04-25T19:12:20.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb298019df0 con 0x563a68f6e590 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.482+0000 7fb27fe00700 1 -- 172.21.15.86:0/4206835562 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fb298017d00 con 0x563a68f6e590 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb27fe00700 1 -- 172.21.15.86:0/4206835562 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb298026040 con 0x563a68f6e590 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 10 client.26381 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a68fe5600 con 0x563a68f6e590 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb27fe00700 1 -- 172.21.15.86:0/4206835562 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298019df0 con 0x563a68f6e590 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb27fe00700 1 -- 172.21.15.86:0/4206835562 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb298005040 con 0x563a68f6e590 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 10 client.26381 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 10 client.26381 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 20 client.26381 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68fe5c60 0x563a68fe8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 4 client.26381 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.483+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x563a68eb98b0 con 0x563a68fe5c60 2024-04-25T19:12:20.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.484+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68fe5c60 0x563a68fe8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:20.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.484+0000 7fb2a1e00700 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68fe5c60 0x563a68fe8050 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6642285569ec14f server_cookie=8bce71220d9374c7 in_seq=0 out_seq=0 2024-04-25T19:12:20.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.484+0000 7fb27fe00700 10 client.26381 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb27fe00700 1 -- 172.21.15.86:0/4206835562 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563a68eb98b0 con 0x563a68fe5c60 2024-04-25T19:12:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb27fe00700 10 client.26381 handle_command_reply: tid=0 2024-04-25T19:12:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 client.26381 shutdown 2024-04-25T19:12:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 msgr2=0x563a68f73a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f73a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb298000f80 tx=0x7fb298017830 comp rx=0 tx=0).stop 2024-04-25T19:12:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 shutdown_connections 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a68fe5c60 0x563a68fe8050 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a68f6e590 0x563a68f73a30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a68f5e9b0 0x563a68f72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 --2- 172.21.15.86:0/4206835562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a68f5f2d0 0x563a68f734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 >> 172.21.15.86:0/4206835562 conn(0x563a68e8d730 msgr2=0x563a68f6d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.501+0000 7fb27fe00700 0 client.26381 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.502+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 shutdown_connections 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.502+0000 7fb2af1efb80 1 -- 172.21.15.86:0/4206835562 wait complete. 2024-04-25T19:12:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.502+0000 7fb2af1efb80 20 client.26381 trim_cache size 0 max 16384 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 26, 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "6f56fff9": { 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 26, 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:12:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 70.0, 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:20.507 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2840a3e50 msgr2=0x7fb2840a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2840a3e50 0x7fb2840a6310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb298003e80 tx=0x7fb298009920 comp rx=0 tx=0).stop 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 msgr2=0x7fb29c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c1996b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e9c0 tx=0x7fb28c00ecd0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 shutdown_connections 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2840a3e50 0x7fb2840a6310 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb29c108a80 0x7fb29c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29c102af0 0x7fb29c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 --2- 172.21.15.86:0/828587544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb29c1034b0 0x7fb29c1996b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:20.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 >> 172.21.15.86:0/828587544 conn(0x7fb29c079b20 msgr2=0x7fb29c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:20.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 shutdown_connections 2024-04-25T19:12:20.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:20.503+0000 7fb2a4800700 1 -- 172.21.15.86:0/828587544 wait complete. 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: pgmap v292: 97 pgs: 97 active+clean; 4.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:12:21.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:20 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: pgmap v292: 97 pgs: 97 active+clean; 4.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:12:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:20 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: pgmap v292: 97 pgs: 97 active+clean; 4.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:12:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:20 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:12:21.292 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6f56fff9', quiesce took 7.3 seconds. Will release after: 10 2024-04-25T19:12:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:22 smithi086 ceph-mon[27783]: pgmap v293: 97 pgs: 97 active+clean; 4.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:23.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:22 smithi183 ceph-mon[32344]: pgmap v293: 97 pgs: 97 active+clean; 4.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:23.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:22 smithi194 ceph-mon[32740]: pgmap v293: 97 pgs: 97 active+clean; 4.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 18 op/s 2024-04-25T19:12:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:24 smithi086 ceph-mon[27783]: pgmap v294: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-25T19:12:25.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:24 smithi183 ceph-mon[32344]: pgmap v294: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-25T19:12:25.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:24 smithi194 ceph-mon[32740]: pgmap v294: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-25T19:12:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:26 smithi086 ceph-mon[27783]: pgmap v295: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:12:27.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:26 smithi183 ceph-mon[32344]: pgmap v295: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:12:27.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:26 smithi194 ceph-mon[32740]: pgmap v295: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 -- 172.21.15.86:0/62650641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e20104d70 msgr2=0x7f7e20108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 --2- 172.21.15.86:0/62650641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e20104d70 0x7f7e20108db0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009900 tx=0x7f7e0802a410 comp rx=0 tx=0).stop 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 -- 172.21.15.86:0/62650641 shutdown_connections 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 --2- 172.21.15.86:0/62650641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e20109490 0x7f7e20111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 --2- 172.21.15.86:0/62650641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e20104d70 0x7f7e20108db0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 --2- 172.21.15.86:0/62650641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e201043c0 0x7f7e201047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 -- 172.21.15.86:0/62650641 >> 172.21.15.86:0/62650641 conn(0x7f7e200ffc20 msgr2=0x7f7e20102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 -- 172.21.15.86:0/62650641 shutdown_connections 2024-04-25T19:12:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.500+0000 7f7e25a00640 1 -- 172.21.15.86:0/62650641 wait complete. 2024-04-25T19:12:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.501+0000 7f7e25a00640 1 Processor -- start 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.501+0000 7f7e25a00640 1 -- start start 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e25a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 0x7f7e201a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e25a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e20109490 0x7f7e201a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e25a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e2019a700 0x7f7e2019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e25a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20114400 con 0x7f7e201043c0 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e25a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20114280 con 0x7f7e2019a700 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e25a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20114580 con 0x7f7e20109490 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 0x7f7e201a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 0x7f7e201a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60412/0 (socket says 172.21.15.86:60412) 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 -- 172.21.15.86:0/2673399018 learned_addr learned my addr 172.21.15.86:0/2673399018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e16000640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e20109490 0x7f7e201a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1f400640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e2019a700 0x7f7e2019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 -- 172.21.15.86:0/2673399018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e20109490 msgr2=0x7f7e201a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e20109490 0x7f7e201a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 -- 172.21.15.86:0/2673399018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e2019a700 msgr2=0x7f7e2019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e2019a700 0x7f7e2019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.502+0000 7f7e1ea00640 1 -- 172.21.15.86:0/2673399018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2019b300 con 0x7f7e201043c0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.503+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 0x7f7e201a0640 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000ea60 tx=0x7f7e1000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.503+0000 7f7e17e00640 1 -- 172.21.15.86:0/2673399018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1000cde0 con 0x7f7e201043c0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.503+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e201a73e0 con 0x7f7e201043c0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.503+0000 7f7e17e00640 1 -- 172.21.15.86:0/2673399018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e100035f0 con 0x7f7e201043c0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.503+0000 7f7e17e00640 1 -- 172.21.15.86:0/2673399018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10010690 con 0x7f7e201043c0 2024-04-25T19:12:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.503+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e201a7790 con 0x7f7e201043c0 2024-04-25T19:12:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e17e00640 1 -- 172.21.15.86:0/2673399018 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e100037a0 con 0x7f7e201043c0 2024-04-25T19:12:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 Processor -- start 2024-04-25T19:12:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 -- start start 2024-04-25T19:12:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd5f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5ffa00 0x55f7bd601df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f7bd55a900 con 0x55f7bd5f5140 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7bd500ef0 con 0x55f7bd5f4820 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e289d2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f7bd5568c0 con 0x55f7bd5ffa00 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.507+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60414/0 (socket says 172.21.15.86:60414) 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd5f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 -- 172.21.15.86:0/3323843359 learned_addr learned my addr 172.21.15.86:0/3323843359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e16000640 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5ffa00 0x55f7bd601df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 -- 172.21.15.86:0/3323843359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5ffa00 msgr2=0x55f7bd601df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5ffa00 0x55f7bd601df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 -- 172.21.15.86:0/3323843359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 msgr2=0x55f7bd5f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd5f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 -- 172.21.15.86:0/3323843359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7bd26d210 con 0x55f7bd5f5140 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5ff4c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7e100ce1e0 tx=0x7f7e100d96e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cc3d619a7508e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.508+0000 7f7e17e00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7e040a3dc0 0x7f7e040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e0fe00640 1 -- 172.21.15.86:0/3323843359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10098d90 con 0x55f7bd5f5140 2024-04-25T19:12:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e0fe00640 1 -- 172.21.15.86:0/3323843359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1001bb40 con 0x55f7bd5f5140 2024-04-25T19:12:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e17e00640 1 -- 172.21.15.86:0/2673399018 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f7e100d0c20 con 0x7f7e201043c0 2024-04-25T19:12:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e0fe00640 1 -- 172.21.15.86:0/3323843359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1009bf40 con 0x55f7bd5f5140 2024-04-25T19:12:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e1f400640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7e040a3dc0 0x7f7e040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e289d2740 1 -- 172.21.15.86:0/3323843359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 msgr2=0x55f7bd5ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e1f400640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7e040a3dc0 0x7f7e040a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005fd0 tx=0x7f7e18009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.509+0000 7f7e289d2740 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5ff4c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7e100ce1e0 tx=0x7f7e100d96e0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 -- 172.21.15.86:0/3323843359 shutdown_connections 2024-04-25T19:12:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5ffa00 0x55f7bd601df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5ff4c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 --2- 172.21.15.86:0/3323843359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd5f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 -- 172.21.15.86:0/3323843359 >> 172.21.15.86:0/3323843359 conn(0x55f7bd5f0d30 msgr2=0x55f7bd5f1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 -- 172.21.15.86:0/3323843359 shutdown_connections 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 -- 172.21.15.86:0/3323843359 wait complete. 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 -- start start 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.510+0000 7f7e289d2740 1 Processor -- start 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f4820 0x55f7bd5fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f7bd602630 con 0x55f7bd5f5140 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7bd514310 con 0x55f7bd5ffa00 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f7bd6027b0 con 0x55f7bd5f4820 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e289d2740 10 client.0 _ll_register_callbacks cb 0x55f7bd4e2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5fce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60416/0 (socket says 172.21.15.86:60416) 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e16000640 1 -- 172.21.15.86:0/1866027278 learned_addr learned my addr 172.21.15.86:0/1866027278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f4820 0x55f7bd5fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.511+0000 7f7e1f400640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e16000640 1 -- 172.21.15.86:0/1866027278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f4820 msgr2=0x55f7bd5fc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e16000640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f4820 0x55f7bd5fc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e16000640 1 -- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 msgr2=0x55f7bd5fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e16000640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e16000640 1 -- 172.21.15.86:0/1866027278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7bd339620 con 0x55f7bd5f5140 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e1f400640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e16000640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5fce90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000c00 tx=0x7f7e08004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36172ecdffa7ac54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e0cc00640 1 -- 172.21.15.86:0/1866027278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08035040 con 0x55f7bd5f5140 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e0cc00640 1 -- 172.21.15.86:0/1866027278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f7e08005450 con 0x55f7bd5f5140 2024-04-25T19:12:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.512+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1001bb40 con 0x55f7bd5f5140 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e0cc00640 1 -- 172.21.15.86:0/1866027278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e08005650 con 0x55f7bd5f5140 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 10 client.16290 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7bd60e840 con 0x55f7bd5f5140 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e0cc00640 1 -- 172.21.15.86:0/1866027278 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1001bb40 con 0x55f7bd5f5140 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e0cc00640 1 -- 172.21.15.86:0/1866027278 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7e08003b50 con 0x55f7bd5f5140 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 10 client.16290 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 20 client.16290 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd60fd00 0x55f7bd6120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 4 client.16290 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.513+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7bd56e8b0 con 0x55f7bd60fd00 2024-04-25T19:12:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.514+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd60fd00 0x55f7bd6120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.514+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd60fd00 0x55f7bd6120f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c85859d71fa93b6 server_cookie=91750d91f1ca02f3 in_seq=0 out_seq=0 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.514+0000 7f7e0cc00640 10 client.16290 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e0cc00640 1 -- 172.21.15.86:0/1866027278 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7bd56e8b0 con 0x55f7bd60fd00 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e0cc00640 10 client.16290 handle_command_reply: tid=0 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 client.16290 shutdown 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 msgr2=0x55f7bd5fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5fce90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000c00 tx=0x7f7e08004440 comp rx=0 tx=0).stop 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 shutdown_connections 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd60fd00 0x55f7bd6120f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f5140 0x55f7bd5fce90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 --2- 172.21.15.86:0/1866027278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f4820 0x55f7bd5fc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e0cc00640 0 client.16290 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.515+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 >> 172.21.15.86:0/1866027278 conn(0x55f7bd5f0d30 msgr2=0x55f7bd5f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.516+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 shutdown_connections 2024-04-25T19:12:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.516+0000 7f7e289d2740 1 -- 172.21.15.86:0/1866027278 wait complete. 2024-04-25T19:12:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.516+0000 7f7e289d2740 20 client.16290 trim_cache size 0 max 16384 2024-04-25T19:12:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 Processor -- start 2024-04-25T19:12:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 -- start start 2024-04-25T19:12:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd60b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd5f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201affd0 con 0x55f7bd5ffa00 2024-04-25T19:12:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201b1b10 con 0x55f7bd5f4820 2024-04-25T19:12:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e289d2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201b06a0 con 0x55f7bd5f5140 2024-04-25T19:12:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.532+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd60b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60418/0 (socket says 172.21.15.86:60418) 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1f400640 1 -- 172.21.15.86:0/180987579 learned_addr learned my addr 172.21.15.86:0/180987579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e16000640 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd5f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1ea00640 1 -- 172.21.15.86:0/180987579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 msgr2=0x55f7bd5f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd5f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1ea00640 1 -- 172.21.15.86:0/180987579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5ffa00 msgr2=0x55f7bd5fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1ea00640 1 -- 172.21.15.86:0/180987579 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7bd5f4f30 con 0x55f7bd5f4820 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd60b4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f7bd5f5dc0 tx=0x7f7e10003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a63b6db079e684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e0f400640 1 -- 172.21.15.86:0/180987579 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1009c270 con 0x55f7bd5f4820 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e0f400640 1 -- 172.21.15.86:0/180987579 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e100d44a0 con 0x55f7bd5f4820 2024-04-25T19:12:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.533+0000 7f7e0f400640 1 -- 172.21.15.86:0/180987579 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1009c450 con 0x55f7bd5f4820 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 -- 172.21.15.86:0/180987579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 msgr2=0x55f7bd60b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd60b4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f7bd5f5dc0 tx=0x7f7e10003e90 comp rx=0 tx=0).stop 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 -- 172.21.15.86:0/180987579 shutdown_connections 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5ffa00 0x55f7bd5fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd5f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 --2- 172.21.15.86:0/180987579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5f4820 0x55f7bd60b4f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 -- 172.21.15.86:0/180987579 >> 172.21.15.86:0/180987579 conn(0x55f7bd5f0d30 msgr2=0x55f7bd5fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 -- 172.21.15.86:0/180987579 shutdown_connections 2024-04-25T19:12:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.534+0000 7f7e289d2740 1 -- 172.21.15.86:0/180987579 wait complete. 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 -- start start 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 Processor -- start 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 0x55f7bd607a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd607f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd604fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201b1440 con 0x55f7bd5f4820 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201b0d70 con 0x55f7bd5ffa00 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201adfd0 con 0x55f7bd5f5140 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e289d2740 10 client.0 _ll_register_callbacks cb 0x7f7e080040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.535+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 0x55f7bd607a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 0x55f7bd607a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60430/0 (socket says 172.21.15.86:60430) 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 -- 172.21.15.86:0/307403988 learned_addr learned my addr 172.21.15.86:0/307403988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e16000640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd607f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1f400640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd604fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 -- 172.21.15.86:0/307403988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 msgr2=0x55f7bd607f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd607f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 -- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 msgr2=0x55f7bd604fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd604fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 -- 172.21.15.86:0/307403988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7bd6057a0 con 0x55f7bd5f4820 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e1ea00640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 0x55f7bd607a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e10000c00 tx=0x7f7e10003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff86154f79e69db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e0cc00640 1 -- 172.21.15.86:0/307403988 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1009c630 con 0x55f7bd5f4820 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e0cc00640 1 -- 172.21.15.86:0/307403988 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f7e1009adf0 con 0x55f7bd5f4820 2024-04-25T19:12:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.536+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e100d44a0 con 0x55f7bd5f4820 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e0cc00640 1 -- 172.21.15.86:0/307403988 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e1009c8c0 con 0x55f7bd5f4820 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7bd675ec0 con 0x55f7bd5f4820 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e0cc00640 1 -- 172.21.15.86:0/307403988 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e100d44a0 con 0x55f7bd5f4820 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e0cc00640 1 -- 172.21.15.86:0/307403988 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7e100d7070 con 0x55f7bd5f4820 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 20 client.16296 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd6760e0 0x55f7bd6784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 4 client.16296 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.537+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7bd5f8550 con 0x55f7bd6760e0 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.538+0000 7f7e16000640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd6760e0 0x55f7bd6784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.538+0000 7f7e16000640 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd6760e0 0x55f7bd6784d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4989b782bf82c server_cookie=6742f2f81bb0d5da in_seq=0 out_seq=0 2024-04-25T19:12:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.538+0000 7f7e0cc00640 10 client.16296 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e0cc00640 1 -- 172.21.15.86:0/307403988 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7bd5f8550 con 0x55f7bd6760e0 2024-04-25T19:12:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e0cc00640 10 client.16296 handle_command_reply: tid=0 2024-04-25T19:12:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 client.16296 shutdown 2024-04-25T19:12:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 msgr2=0x55f7bd607a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 0x55f7bd607a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e10000c00 tx=0x7f7e10003dd0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 shutdown_connections 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f7bd6760e0 0x55f7bd6784d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bd5ffa00 0x55f7bd604fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7bd5f5140 0x55f7bd607f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e0cc00640 0 client.16296 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 --2- 172.21.15.86:0/307403988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7bd5f4820 0x55f7bd607a10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.552+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 >> 172.21.15.86:0/307403988 conn(0x55f7bd5f0d30 msgr2=0x55f7bd5efe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.553+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 shutdown_connections 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.553+0000 7f7e289d2740 1 -- 172.21.15.86:0/307403988 wait complete. 2024-04-25T19:12:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.553+0000 7f7e289d2740 20 client.16296 trim_cache size 0 max 16384 2024-04-25T19:12:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:12:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:12:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:12:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:12:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7e040a3dc0 msgr2=0x7f7e040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7e040a3dc0 0x7f7e040a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005fd0 tx=0x7f7e18009500 comp rx=0 tx=0).stop 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 msgr2=0x7f7e201a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 0x7f7e201a0640 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000ea60 tx=0x7f7e1000ef20 comp rx=0 tx=0).stop 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 shutdown_connections 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f7e040a3dc0 0x7f7e040a6280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e2019a700 0x7f7e2019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e20109490 0x7f7e201a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 --2- 172.21.15.86:0/2673399018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e201043c0 0x7f7e201a0640 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 >> 172.21.15.86:0/2673399018 conn(0x7f7e200ffc20 msgr2=0x7f7e20101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 shutdown_connections 2024-04-25T19:12:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:27.554+0000 7f7e25a00640 1 -- 172.21.15.86:0/2673399018 wait complete. 2024-04-25T19:12:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:28 smithi086 ceph-mon[27783]: pgmap v296: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-25T19:12:29.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:28 smithi183 ceph-mon[32344]: pgmap v296: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-25T19:12:29.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:28 smithi194 ceph-mon[32740]: pgmap v296: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-25T19:12:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:30 smithi086 ceph-mon[27783]: pgmap v297: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-25T19:12:31.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:30 smithi183 ceph-mon[32344]: pgmap v297: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-25T19:12:31.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:30 smithi194 ceph-mon[32740]: pgmap v297: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-25T19:12:31.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:12:31.749 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:12:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:32 smithi086 ceph-mon[27783]: pgmap v298: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-25T19:12:33.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:32 smithi183 ceph-mon[32344]: pgmap v298: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-25T19:12:33.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:32 smithi194 ceph-mon[32740]: pgmap v298: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 -- 172.21.15.86:0/939747059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 msgr2=0x7f1ab01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 --2- 172.21.15.86:0/939747059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01096b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00099c0 tx=0x7f1aa0009cd0 comp rx=0 tx=0).stop 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 -- 172.21.15.86:0/939747059 shutdown_connections 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 --2- 172.21.15.86:0/939747059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab010e3b0 0x7f1ab0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 --2- 172.21.15.86:0/939747059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01096b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 --2- 172.21.15.86:0/939747059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab0109c80 0x7f1ab010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 -- 172.21.15.86:0/939747059 >> 172.21.15.86:0/939747059 conn(0x7f1ab0076210 msgr2=0x7f1ab0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 -- 172.21.15.86:0/939747059 shutdown_connections 2024-04-25T19:12:33.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.874+0000 7f1ab5800700 1 -- 172.21.15.86:0/939747059 wait complete. 2024-04-25T19:12:33.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.875+0000 7f1ab5800700 1 Processor -- start 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.875+0000 7f1ab5800700 1 -- start start 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1ab5800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1ab5800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0109c80 0x7f1ab01a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1ab5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab010e3b0 0x7f1ab019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1ab5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0119290 con 0x7f1ab010e3b0 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1ab5800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0119110 con 0x7f1ab0109c80 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1ab5800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0119410 con 0x7f1ab01092d0 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aaea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aaf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab010e3b0 0x7f1ab019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aaea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41926/0 (socket says 172.21.15.86:41926) 2024-04-25T19:12:33.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aae000700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0109c80 0x7f1ab01a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aaea00700 1 -- 172.21.15.86:0/1592309050 learned_addr learned my addr 172.21.15.86:0/1592309050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aae000700 1 -- 172.21.15.86:0/1592309050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 msgr2=0x7f1ab01a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aae000700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aae000700 1 -- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab010e3b0 msgr2=0x7f1ab019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aae000700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab010e3b0 0x7f1ab019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aae000700 1 -- 172.21.15.86:0/1592309050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab019fd50 con 0x7f1ab0109c80 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.876+0000 7f1aaea00700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.877+0000 7f1aae000700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0109c80 0x7f1ab01a5920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9c0 tx=0x7f1a9c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.877+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00c9c0 con 0x7f1ab0109c80 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.877+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c00cb60 con 0x7f1ab0109c80 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.877+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab01a0040 con 0x7f1ab0109c80 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.877+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c018680 con 0x7f1ab0109c80 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.877+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01ac2b0 con 0x7f1ab0109c80 2024-04-25T19:12:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.878+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab0069730 con 0x7f1ab0109c80 2024-04-25T19:12:33.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.880+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a9c010430 con 0x7f1ab0109c80 2024-04-25T19:12:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.882+0000 7f1aa7400700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1a940a3e30 0x7f1a940a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.882+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f1a9c0d0d20 con 0x7f1ab0109c80 2024-04-25T19:12:33.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.882+0000 7f1aaea00700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1a940a3e30 0x7f1a940a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:33.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.883+0000 7f1aaea00700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1a940a3e30 0x7f1a940a62f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0000c00 tx=0x7f1aa0005fd0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:33.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:33.884+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a9c09d2f0 con 0x7f1ab0109c80 2024-04-25T19:12:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.157+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab002df80 con 0x7f1ab0109c80 2024-04-25T19:12:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.158+0000 7f1aa7400700 1 -- 172.21.15.86:0/1592309050 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f1a9c09cbf0 con 0x7f1ab0109c80 2024-04-25T19:12:34.160 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:12:34.160 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1a940a3e30 msgr2=0x7f1a940a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1a940a3e30 0x7f1a940a62f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0000c00 tx=0x7f1aa0005fd0 comp rx=0 tx=0).stop 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0109c80 msgr2=0x7f1ab01a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0109c80 0x7f1ab01a5920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9c0 tx=0x7f1a9c00ecd0 comp rx=0 tx=0).stop 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 shutdown_connections 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab010e3b0 0x7f1ab019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1a940a3e30 0x7f1a940a62f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ab01092d0 0x7f1ab01a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 --2- 172.21.15.86:0/1592309050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0109c80 0x7f1ab01a5920 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 >> 172.21.15.86:0/1592309050 conn(0x7f1ab0076210 msgr2=0x7f1ab0077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 shutdown_connections 2024-04-25T19:12:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:34.161+0000 7f1ab5800700 1 -- 172.21.15.86:0/1592309050 wait complete. 2024-04-25T19:12:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:12:34.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:34 smithi086 ceph-mon[27783]: pgmap v299: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T19:12:34.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:34 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1592309050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:12:35.031 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 6f56fff9 --release --await' 2024-04-25T19:12:35.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 6f56fff9 --release --await 2024-04-25T19:12:35.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:34 smithi183 ceph-mon[32344]: pgmap v299: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T19:12:35.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:34 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1592309050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:12:35.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:34 smithi194 ceph-mon[32740]: pgmap v299: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T19:12:35.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:34 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1592309050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:12:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:12:35.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:35 smithi086 ceph-mon[27783]: pgmap v300: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-25T19:12:36.068 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:35 smithi183 ceph-mon[32344]: pgmap v300: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-25T19:12:36.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:35 smithi194 ceph-mon[32740]: pgmap v300: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 -- 172.21.15.86:0/2335957274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 msgr2=0x7f39ac10f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 --2- 172.21.15.86:0/2335957274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac10f410 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009990 tx=0x7f39a0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 -- 172.21.15.86:0/2335957274 shutdown_connections 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 --2- 172.21.15.86:0/2335957274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac109940 0x7f39ac115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 --2- 172.21.15.86:0/2335957274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac10f410 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 --2- 172.21.15.86:0/2335957274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 0x7f39ac109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 -- 172.21.15.86:0/2335957274 >> 172.21.15.86:0/2335957274 conn(0x7f39ac0787d0 msgr2=0x7f39ac107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 -- 172.21.15.86:0/2335957274 shutdown_connections 2024-04-25T19:12:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.698+0000 7f39b2e00700 1 -- 172.21.15.86:0/2335957274 wait complete. 2024-04-25T19:12:37.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 Processor -- start 2024-04-25T19:12:37.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 -- start start 2024-04-25T19:12:37.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 0x7f39ac19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac109940 0x7f39ac19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac1a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1190f0 con 0x7f39ac109940 2024-04-25T19:12:37.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac118f70 con 0x7f39ac10f030 2024-04-25T19:12:37.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.699+0000 7f39b2e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac119270 con 0x7f39ac108f80 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b1a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 0x7f39ac19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b1a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 0x7f39ac19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41944/0 (socket says 172.21.15.86:41944) 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b1a00700 1 -- 172.21.15.86:0/784613781 learned_addr learned my addr 172.21.15.86:0/784613781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b2400700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac1a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b1000700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac109940 0x7f39ac19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b2400700 1 -- 172.21.15.86:0/784613781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 msgr2=0x7f39ac19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b2400700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 0x7f39ac19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b2400700 1 -- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac109940 msgr2=0x7f39ac19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b2400700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac109940 0x7f39ac19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.700+0000 7f39b2400700 1 -- 172.21.15.86:0/784613781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac1a45d0 con 0x7f39ac10f030 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.701+0000 7f39b2400700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac1a3e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39a400efb0 tx=0x7f39a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:37.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.701+0000 7f39aa000700 1 -- 172.21.15.86:0/784613781 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4003a00 con 0x7f39ac10f030 2024-04-25T19:12:37.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.701+0000 7f39aa000700 1 -- 172.21.15.86:0/784613781 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a400ce50 con 0x7f39ac10f030 2024-04-25T19:12:37.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.701+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ac1a48c0 con 0x7f39ac10f030 2024-04-25T19:12:37.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.701+0000 7f39aa000700 1 -- 172.21.15.86:0/784613781 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40047c0 con 0x7f39ac10f030 2024-04-25T19:12:37.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.701+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ac1ac140 con 0x7f39ac10f030 2024-04-25T19:12:37.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.705+0000 7f39aa000700 1 -- 172.21.15.86:0/784613781 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39a4012050 con 0x7f39ac10f030 2024-04-25T19:12:37.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.706+0000 7f39bec99b80 1 Processor -- start 2024-04-25T19:12:37.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 -- start start 2024-04-25T19:12:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39aa000700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39940a3db0 0x7f39940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a132c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a13207f90 con 0x563a132b57c0 2024-04-25T19:12:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a131ae830 con 0x563a132b4ea0 2024-04-25T19:12:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39bec99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a13203f50 con 0x563a132c0670 2024-04-25T19:12:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39aa000700 1 -- 172.21.15.86:0/784613781 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f39a40d01b0 con 0x7f39ac10f030 2024-04-25T19:12:37.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.707+0000 7f39b1000700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39940a3db0 0x7f39940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a132c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a132c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41958/0 (socket says 172.21.15.86:41958) 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 -- 172.21.15.86:0/1590547093 learned_addr learned my addr 172.21.15.86:0/1590547093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b1a00700 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 -- 172.21.15.86:0/1590547093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 msgr2=0x563a132b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 -- 172.21.15.86:0/1590547093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 msgr2=0x563a132c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 -- 172.21.15.86:0/1590547093 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a131a4780 con 0x563a132c0670 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.708+0000 7f39b2400700 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a132c2a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a47c0 tx=0x7f39a409bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d57e7a8d71f948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.709+0000 7f39b1000700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39940a3db0 0x7f39940a6270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39ac10e020 tx=0x7f3998009bf0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.709+0000 7f399ea00700 1 -- 172.21.15.86:0/1590547093 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a404a2a0 con 0x563a132c0670 2024-04-25T19:12:37.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.709+0000 7f399ea00700 1 -- 172.21.15.86:0/1590547093 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a409c810 con 0x563a132c0670 2024-04-25T19:12:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.709+0000 7f399ea00700 1 -- 172.21.15.86:0/1590547093 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40cdab0 con 0x563a132c0670 2024-04-25T19:12:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.709+0000 7f39bec99b80 1 -- 172.21.15.86:0/1590547093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 msgr2=0x563a132c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.709+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a132c2a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a47c0 tx=0x7f39a409bf60 comp rx=0 tx=0).stop 2024-04-25T19:12:37.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 -- 172.21.15.86:0/1590547093 shutdown_connections 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a132c2a60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1590547093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 -- 172.21.15.86:0/1590547093 >> 172.21.15.86:0/1590547093 conn(0x563a131e4050 msgr2=0x563a132b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 -- 172.21.15.86:0/1590547093 shutdown_connections 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 -- 172.21.15.86:0/1590547093 wait complete. 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.710+0000 7f39bec99b80 1 -- start start 2024-04-25T19:12:37.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 Processor -- start 2024-04-25T19:12:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132bd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132c0670 0x563a132bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a132c3100 con 0x563a132b57c0 2024-04-25T19:12:37.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a131c1960 con 0x563a132c0670 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a132c3280 con 0x563a132b4ea0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.711+0000 7f39bec99b80 10 client.0 _ll_register_callbacks cb 0x563a131a0820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132bd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132bd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34142/0 (socket says 172.21.15.86:34142) 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 -- 172.21.15.86:0/2970831510 learned_addr learned my addr 172.21.15.86:0/2970831510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1a00700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b2400700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132c0670 0x563a132bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 -- 172.21.15.86:0/2970831510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 msgr2=0x563a132bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 -- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132c0670 msgr2=0x563a132bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132c0670 0x563a132bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 -- 172.21.15.86:0/2970831510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a131559d0 con 0x563a132b57c0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f39b1000700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132bd110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39980095f0 tx=0x7f399801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992876af7a7085e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:37.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.712+0000 7f3993400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f3993400700 1 -- 172.21.15.86:0/2970831510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399800fa40 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f3993400700 1 -- 172.21.15.86:0/2970831510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f399800fbe0 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a409c810 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f3993400700 1 -- 172.21.15.86:0/2970831510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f399800fde0 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f39bec99b80 10 client.16314 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a1319ebf0 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.713+0000 7f3993400700 1 -- 172.21.15.86:0/2970831510 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a409c810 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f3993400700 1 -- 172.21.15.86:0/2970831510 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f399801f600 con 0x563a132b57c0 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39bec99b80 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39bec99b80 10 client.16314 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39bec99b80 20 client.16314 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39bec99b80 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a132cfe40 0x563a132d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39bec99b80 4 client.16314 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:37.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a13215780 con 0x563a132cfe40 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.714+0000 7f39b1a00700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a132cfe40 0x563a132d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.715+0000 7f39b1a00700 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a132cfe40 0x563a132d2270 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5694c5492a7e86 server_cookie=4ea6a17380ca1d29 in_seq=0 out_seq=0 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.715+0000 7f3993400700 10 client.16314 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f3993400700 1 -- 172.21.15.86:0/2970831510 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a13215780 con 0x563a132cfe40 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f3993400700 10 client.16314 handle_command_reply: tid=0 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 client.16314 shutdown 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 msgr2=0x563a132bd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132bd110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39980095f0 tx=0x7f399801d000 comp rx=0 tx=0).stop 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 shutdown_connections 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a132cfe40 0x563a132d2270 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132c0670 0x563a132bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 --2- 172.21.15.86:0/2970831510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132bd110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 >> 172.21.15.86:0/2970831510 conn(0x563a131e4050 msgr2=0x563a132b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f3993400700 0 client.16314 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 shutdown_connections 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 1 -- 172.21.15.86:0/2970831510 wait complete. 2024-04-25T19:12:37.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.716+0000 7f39bec99b80 20 client.16314 trim_cache size 0 max 16384 2024-04-25T19:12:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.737+0000 7f39bec99b80 1 Processor -- start 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.737+0000 7f39bec99b80 1 -- start start 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b57c0 0x563a132b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 0x563a132b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39bec99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b38d0 con 0x563a132c0670 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39bec99b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b5410 con 0x563a132b57c0 2024-04-25T19:12:37.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39bec99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b3fa0 con 0x563a132b4ea0 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39b2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 0x563a132b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39b2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 0x563a132b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34152/0 (socket says 172.21.15.86:34152) 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39b2400700 1 -- 172.21.15.86:0/1647281341 learned_addr learned my addr 172.21.15.86:0/1647281341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39b1000700 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b57c0 0x563a132b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.738+0000 7f39b1a00700 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f39b2400700 1 -- 172.21.15.86:0/1647281341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 msgr2=0x563a132cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f39b2400700 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f39b2400700 1 -- 172.21.15.86:0/1647281341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b57c0 msgr2=0x563a132b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f39b2400700 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b57c0 0x563a132b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f39b2400700 1 -- 172.21.15.86:0/1647281341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a132bb680 con 0x563a132c0670 2024-04-25T19:12:37.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f39b2400700 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 0x563a132b11b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f39a409c050 tx=0x7f39a4004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dd09390d9f8ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f399e000700 1 -- 172.21.15.86:0/1647281341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40b5070 con 0x563a132c0670 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f399e000700 1 -- 172.21.15.86:0/1647281341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a409c110 con 0x563a132c0670 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.739+0000 7f399e000700 1 -- 172.21.15.86:0/1647281341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40cdab0 con 0x563a132c0670 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 -- 172.21.15.86:0/1647281341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 msgr2=0x563a132b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 0x563a132b11b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f39a409c050 tx=0x7f39a4004960 comp rx=0 tx=0).stop 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 -- 172.21.15.86:0/1647281341 shutdown_connections 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132c0670 0x563a132b11b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132b4ea0 0x563a132cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1647281341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b57c0 0x563a132b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 -- 172.21.15.86:0/1647281341 >> 172.21.15.86:0/1647281341 conn(0x563a131e4050 msgr2=0x563a132b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 -- 172.21.15.86:0/1647281341 shutdown_connections 2024-04-25T19:12:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.740+0000 7f39bec99b80 1 -- 172.21.15.86:0/1647281341 wait complete. 2024-04-25T19:12:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 -- start start 2024-04-25T19:12:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 Processor -- start 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a13330440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b4d40 con 0x563a132b57c0 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b4670 con 0x563a132b4ea0 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b1890 con 0x563a132c0670 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.741+0000 7f39bec99b80 10 client.0 _ll_register_callbacks cb 0x563a131a0820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34166/0 (socket says 172.21.15.86:34166) 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 -- 172.21.15.86:0/1790350439 learned_addr learned my addr 172.21.15.86:0/1790350439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b2400700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a13330440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1a00700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 -- 172.21.15.86:0/1790350439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 msgr2=0x563a13330440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a13330440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 -- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 msgr2=0x563a132c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 -- 172.21.15.86:0/1790350439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a13330a80 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f39b1000700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c5cb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3998009e90 tx=0x7f399801f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95518f29e605a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.742+0000 7f3993400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f3993400700 1 -- 172.21.15.86:0/1790350439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998009260 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a1320f820 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f3993400700 1 -- 172.21.15.86:0/1790350439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f3998011df0 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f3993400700 1 -- 172.21.15.86:0/1790350439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39980045f0 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f39bec99b80 10 client.16320 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a13337e50 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f3993400700 1 -- 172.21.15.86:0/1790350439 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563a1320f820 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.743+0000 7f3993400700 1 -- 172.21.15.86:0/1790350439 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f39980407f0 con 0x563a132b57c0 2024-04-25T19:12:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39bec99b80 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39bec99b80 10 client.16320 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39bec99b80 20 client.16320 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a13338580 0x563a1333a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39bec99b80 4 client.16320 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6f56fff9", "release": true, "await": true}] 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6f56fff9", "release": true, "await": true}) -- 0x563a13088350 con 0x563a13338580 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39b1a00700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a13338580 0x563a1333a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f39b1a00700 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a13338580 0x563a1333a9a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125c9ac031f54ffc server_cookie=9626241d6b83ee96 in_seq=0 out_seq=0 2024-04-25T19:12:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.744+0000 7f3993400700 10 client.16320 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f3993400700 1 -- 172.21.15.86:0/1790350439 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563a13088350 con 0x563a13338580 2024-04-25T19:12:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f3993400700 10 client.16320 handle_command_reply: tid=0 2024-04-25T19:12:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 client.16320 shutdown 2024-04-25T19:12:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 msgr2=0x563a132c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c5cb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3998009e90 tx=0x7f399801f6f0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 shutdown_connections 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a13338580 0x563a1333a9a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a132c0670 0x563a13330440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f3993400700 0 client.16320 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a132b4ea0 0x563a132c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 --2- 172.21.15.86:0/1790350439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a132b57c0 0x563a132c5cb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.748+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 >> 172.21.15.86:0/1790350439 conn(0x563a131e4050 msgr2=0x563a132b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.749+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 shutdown_connections 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.749+0000 7f39bec99b80 1 -- 172.21.15.86:0/1790350439 wait complete. 2024-04-25T19:12:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.749+0000 7f39bec99b80 20 client.16320 trim_cache size 0 max 16384 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 28, 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "6f56fff9": { 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 28, 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:12:37.752 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 70.0, 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:12:37.753 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39940a3db0 msgr2=0x7f39940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39940a3db0 0x7f39940a6270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39ac10e020 tx=0x7f3998009bf0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 msgr2=0x7f39ac1a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac1a3e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39a400efb0 tx=0x7f39a400c3a0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 shutdown_connections 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac10f030 0x7f39ac1a3e90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39940a3db0 0x7f39940a6270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac108f80 0x7f39ac19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 --2- 172.21.15.86:0/784613781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac109940 0x7f39ac19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 >> 172.21.15.86:0/784613781 conn(0x7f39ac0787d0 msgr2=0x7f39ac079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 shutdown_connections 2024-04-25T19:12:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:37.750+0000 7f39b2e00700 1 -- 172.21.15.86:0/784613781 wait complete. 2024-04-25T19:12:38.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:38 smithi183 ceph-mon[32344]: pgmap v301: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-25T19:12:38.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:38 smithi194 ceph-mon[32740]: pgmap v301: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-25T19:12:38.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:38 smithi086 ceph-mon[27783]: pgmap v301: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-25T19:12:38.602 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6f56fff9', seconds elapsed: 24.6 2024-04-25T19:12:40.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:40 smithi183 ceph-mon[32344]: pgmap v302: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T19:12:40.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:40 smithi194 ceph-mon[32740]: pgmap v302: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T19:12:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:40 smithi086 ceph-mon[27783]: pgmap v302: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T19:12:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.703+0000 7f4ec1200640 1 -- 172.21.15.86:0/284162850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc108560 msgr2=0x7f4ebc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.703+0000 7f4ec1200640 1 --2- 172.21.15.86:0/284162850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc108560 0x7f4ebc1089e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009930 tx=0x7f4ea802a420 comp rx=0 tx=0).stop 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 -- 172.21.15.86:0/284162850 shutdown_connections 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 --2- 172.21.15.86:0/284162850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 0x7f4ebc1153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 --2- 172.21.15.86:0/284162850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc108560 0x7f4ebc1089e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 --2- 172.21.15.86:0/284162850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc10e620 0x7f4ebc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 -- 172.21.15.86:0/284162850 >> 172.21.15.86:0/284162850 conn(0x7f4ebc0764e0 msgr2=0x7f4ebc078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 -- 172.21.15.86:0/284162850 shutdown_connections 2024-04-25T19:12:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.704+0000 7f4ec1200640 1 -- 172.21.15.86:0/284162850 wait complete. 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.705+0000 7f4ec1200640 1 Processor -- start 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.705+0000 7f4ec1200640 1 -- start start 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.705+0000 7f4ec1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc108560 0x7f4ebc1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.705+0000 7f4ec1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 0x7f4ebc1a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4ec1200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc10e620 0x7f4ebc19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4ec1200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc118790 con 0x7f4ebc108f20 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4ec1200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc118610 con 0x7f4ebc108560 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4ec1200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc118910 con 0x7f4ebc10e620 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 0x7f4ebc1a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc108560 0x7f4ebc1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 0x7f4ebc1a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40696/0 (socket says 172.21.15.86:40696) 2024-04-25T19:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9400640 1 -- 172.21.15.86:0/15547170 learned_addr learned my addr 172.21.15.86:0/15547170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eba800640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc10e620 0x7f4ebc19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9e00640 1 -- 172.21.15.86:0/15547170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc10e620 msgr2=0x7f4ebc19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9e00640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc10e620 0x7f4ebc19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9e00640 1 -- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 msgr2=0x7f4ebc1a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9e00640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 0x7f4ebc1a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.706+0000 7f4eb9e00640 1 -- 172.21.15.86:0/15547170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc19f1f0 con 0x7f4ebc108560 2024-04-25T19:12:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.707+0000 7f4eb9e00640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc108560 0x7f4ebc1a48c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e9f0 tx=0x7f4eac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.707+0000 7f4eb2a00640 1 -- 172.21.15.86:0/15547170 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00ccc0 con 0x7f4ebc108560 2024-04-25T19:12:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.707+0000 7f4eb2a00640 1 -- 172.21.15.86:0/15547170 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eac0108e0 con 0x7f4ebc108560 2024-04-25T19:12:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.707+0000 7f4eb2a00640 1 -- 172.21.15.86:0/15547170 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac018880 con 0x7f4ebc108560 2024-04-25T19:12:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.707+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc19f4e0 con 0x7f4ebc108560 2024-04-25T19:12:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.707+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc1ab780 con 0x7f4ebc108560 2024-04-25T19:12:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.710+0000 7f4ec402d740 1 Processor -- start 2024-04-25T19:12:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.710+0000 7f4eb2a00640 1 -- 172.21.15.86:0/15547170 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4eac010430 con 0x7f4ebc108560 2024-04-25T19:12:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.710+0000 7f4ec402d740 1 -- start start 2024-04-25T19:12:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca65ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca6654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca667df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4ec402d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558aca5c0900 con 0x558aca665a00 2024-04-25T19:12:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4ec402d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558aca566ef0 con 0x558aca65a820 2024-04-25T19:12:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4ec402d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558aca5bc8c0 con 0x558aca65b140 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca667df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4eba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca65ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4eba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca65ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:45964/0 (socket says 172.21.15.86:45964) 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4eba800640 1 -- 172.21.15.86:0/3857455702 learned_addr learned my addr 172.21.15.86:0/3857455702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.711+0000 7f4eb9e00640 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca6654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eba800640 1 -- 172.21.15.86:0/3857455702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 msgr2=0x558aca6654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eba800640 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca6654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eba800640 1 -- 172.21.15.86:0/3857455702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 msgr2=0x558aca667df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eba800640 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca667df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eba800640 1 -- 172.21.15.86:0/3857455702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aca2d3210 con 0x558aca65a820 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eba800640 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca65ac00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fd0 tx=0x7f4eb40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ed6068d6f5b0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4eb2a00640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4ea00a3e40 0x7f4ea00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4ea7400640 1 -- 172.21.15.86:0/3857455702 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4004d20 con 0x558aca65a820 2024-04-25T19:12:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4ea7400640 1 -- 172.21.15.86:0/3857455702 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb4004050 con 0x558aca65a820 2024-04-25T19:12:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.712+0000 7f4ea7400640 1 -- 172.21.15.86:0/3857455702 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4005260 con 0x558aca65a820 2024-04-25T19:12:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4eb9e00640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4ea00a3e40 0x7f4ea00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4eb2a00640 1 -- 172.21.15.86:0/15547170 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f4eac0d0b50 con 0x7f4ebc108560 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 -- 172.21.15.86:0/3857455702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 msgr2=0x558aca65ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4eb9e00640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4ea00a3e40 0x7f4ea00a6300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558aca65bdc0 tx=0x7f4eac0d4600 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca65ac00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fd0 tx=0x7f4eb40118d0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 -- 172.21.15.86:0/3857455702 shutdown_connections 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca667df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca6654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3857455702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca65ac00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 -- 172.21.15.86:0/3857455702 >> 172.21.15.86:0/3857455702 conn(0x558aca656d30 msgr2=0x558aca657110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 -- 172.21.15.86:0/3857455702 shutdown_connections 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.713+0000 7f4ec402d740 1 -- 172.21.15.86:0/3857455702 wait complete. 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 -- start start 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 Processor -- start 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65a820 0x558aca662920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca662e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca665a00 0x558aca6633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558aca668630 con 0x558aca65a820 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558aca57a310 con 0x558aca665a00 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.714+0000 7f4ec402d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558aca6687b0 con 0x558aca65b140 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4ec402d740 10 client.0 _ll_register_callbacks cb 0x558aca548cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65a820 0x558aca662920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca662e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65a820 0x558aca662920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40716/0 (socket says 172.21.15.86:40716) 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eba800640 1 -- 172.21.15.86:0/2727484814 learned_addr learned my addr 172.21.15.86:0/2727484814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eba800640 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca665a00 0x558aca6633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eb9400640 1 -- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca665a00 msgr2=0x558aca6633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eb9400640 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca665a00 0x558aca6633a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eb9400640 1 -- 172.21.15.86:0/2727484814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65a820 msgr2=0x558aca662920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eb9400640 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65a820 0x558aca662920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.715+0000 7f4eb9400640 1 -- 172.21.15.86:0/2727484814 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aca39f620 con 0x558aca65b140 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4eb9400640 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca662e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8004770 tx=0x7f4ea8031e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b87003f2a5150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4e9be00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:12:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4e9be00640 1 -- 172.21.15.86:0/2727484814 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8035040 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb4004050 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4e9be00640 1 -- 172.21.15.86:0/2727484814 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f4ea8031760 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4e9be00640 1 -- 172.21.15.86:0/2727484814 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ea8004040 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4ec402d740 10 client.26428 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.716+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558aca674880 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4e9be00640 1 -- 172.21.15.86:0/2727484814 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb4004050 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4e9be00640 1 -- 172.21.15.86:0/2727484814 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ea8003cd0 con 0x558aca65b140 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4ec402d740 10 client.26428 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4ec402d740 10 client.26428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4ec402d740 20 client.26428 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4ec402d740 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca675cf0 0x558aca678160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4ec402d740 4 client.26428 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aca5d48b0 con 0x558aca675cf0 2024-04-25T19:12:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.717+0000 7f4eba800640 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca675cf0 0x558aca678160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.718+0000 7f4eba800640 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca675cf0 0x558aca678160 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a09989cefdfd756 server_cookie=4ba5dc59bdb4d290 in_seq=0 out_seq=0 2024-04-25T19:12:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.718+0000 7f4e9be00640 10 client.26428 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4e9be00640 1 -- 172.21.15.86:0/2727484814 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558aca5d48b0 con 0x558aca675cf0 2024-04-25T19:12:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4e9be00640 10 client.26428 handle_command_reply: tid=0 2024-04-25T19:12:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 client.26428 shutdown 2024-04-25T19:12:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 msgr2=0x558aca662e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca662e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8004770 tx=0x7f4ea8031e60 comp rx=0 tx=0).stop 2024-04-25T19:12:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 shutdown_connections 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca675cf0 0x558aca678160 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca665a00 0x558aca6633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca662e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 --2- 172.21.15.86:0/2727484814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65a820 0x558aca662920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4e9be00640 0 client.26428 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 >> 172.21.15.86:0/2727484814 conn(0x558aca656d30 msgr2=0x558aca65ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 shutdown_connections 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.719+0000 7f4ec402d740 1 -- 172.21.15.86:0/2727484814 wait complete. 2024-04-25T19:12:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.720+0000 7f4ec402d740 20 client.26428 trim_cache size 0 max 16384 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.735+0000 7f4ec402d740 1 Processor -- start 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 -- start start 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca6714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 0x558aca65d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca665a00 0x558aca67da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc1b42a0 con 0x558aca65b140 2024-04-25T19:12:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc1b5de0 con 0x558aca65a820 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4ec402d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc1b4970 con 0x558aca665a00 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 0x558aca65d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4eb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 0x558aca65d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40722/0 (socket says 172.21.15.86:40722) 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4eb9400640 1 -- 172.21.15.86:0/232952700 learned_addr learned my addr 172.21.15.86:0/232952700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4eba800640 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca665a00 0x558aca67da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.736+0000 7f4eba800640 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca6714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4eb9400640 1 -- 172.21.15.86:0/232952700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca665a00 msgr2=0x558aca67da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4eb9400640 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca665a00 0x558aca67da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4eb9400640 1 -- 172.21.15.86:0/232952700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 msgr2=0x558aca6714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4eb9400640 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca6714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4eb9400640 1 -- 172.21.15.86:0/232952700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aca6787e0 con 0x558aca65b140 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4eb9400640 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 0x558aca65d1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009a50 tx=0x7f4ea802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70aec0b5ff494b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4ea6a00640 1 -- 172.21.15.86:0/232952700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8035030 con 0x558aca65b140 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4ea6a00640 1 -- 172.21.15.86:0/232952700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea803d990 con 0x558aca65b140 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.737+0000 7f4ea6a00640 1 -- 172.21.15.86:0/232952700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80035f0 con 0x558aca65b140 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 -- 172.21.15.86:0/232952700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 msgr2=0x558aca65d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 0x558aca65d1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009a50 tx=0x7f4ea802a420 comp rx=0 tx=0).stop 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 -- 172.21.15.86:0/232952700 shutdown_connections 2024-04-25T19:12:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca665a00 0x558aca67da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca65b140 0x558aca65d1e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 --2- 172.21.15.86:0/232952700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca6714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 -- 172.21.15.86:0/232952700 >> 172.21.15.86:0/232952700 conn(0x558aca656d30 msgr2=0x558aca663170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 -- 172.21.15.86:0/232952700 shutdown_connections 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 -- 172.21.15.86:0/232952700 wait complete. 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.738+0000 7f4ec402d740 1 -- start start 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 Processor -- start 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca66de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca66b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca66b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc1b5710 con 0x558aca665a00 2024-04-25T19:12:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc1b5040 con 0x558aca65a820 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc1b22a0 con 0x558aca65b140 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4ec402d740 10 client.0 _ll_register_callbacks cb 0x7f4ea8031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4eba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca66b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4eba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca66b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40726/0 (socket says 172.21.15.86:40726) 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4eba800640 1 -- 172.21.15.86:0/3019514456 learned_addr learned my addr 172.21.15.86:0/3019514456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4eba800640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca66de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.739+0000 7f4eb9400640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca66b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4eba800640 1 -- 172.21.15.86:0/3019514456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 msgr2=0x558aca66b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4eba800640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca66b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4eba800640 1 -- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 msgr2=0x558aca66de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4eba800640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca66de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4eba800640 1 -- 172.21.15.86:0/3019514456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aca66bf50 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4eba800640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca66b720 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009730 tx=0x7f4eb4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6798f2353628d4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4e9be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4e9be00640 1 -- 172.21.15.86:0/3019514456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40045f0 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea803d990 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4e9be00640 1 -- 172.21.15.86:0/3019514456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f4eb40049e0 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.740+0000 7f4e9be00640 1 -- 172.21.15.86:0/3019514456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4eb4004be0 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558aca6dc390 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4e9be00640 1 -- 172.21.15.86:0/3019514456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea803d990 con 0x558aca665a00 2024-04-25T19:12:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4e9be00640 1 -- 172.21.15.86:0/3019514456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4eb4025040 con 0x558aca665a00 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 20 client.16338 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca6dc910 0x558aca6ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 4 client.16338 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558aca5ce320 con 0x558aca6dc910 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.741+0000 7f4eb9400640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca6dc910 0x558aca6ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.742+0000 7f4eb9400640 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca6dc910 0x558aca6ded00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7a875d82b267a1 server_cookie=1e36b61bbb0158bd in_seq=0 out_seq=0 2024-04-25T19:12:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.742+0000 7f4e9be00640 10 client.16338 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4e9be00640 1 -- 172.21.15.86:0/3019514456 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558aca5ce320 con 0x558aca6dc910 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4e9be00640 10 client.16338 handle_command_reply: tid=0 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 client.16338 shutdown 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 msgr2=0x558aca66b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca66b720 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009730 tx=0x7f4eb4005bd0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 shutdown_connections 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558aca6dc910 0x558aca6ded00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558aca665a00 0x558aca66b720 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558aca65b140 0x558aca66b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4e9be00640 0 client.16338 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 --2- 172.21.15.86:0/3019514456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558aca65a820 0x558aca66de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 >> 172.21.15.86:0/3019514456 conn(0x558aca656d30 msgr2=0x558aca655600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 shutdown_connections 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 1 -- 172.21.15.86:0/3019514456 wait complete. 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.755+0000 7f4ec402d740 20 client.16338 trim_cache size 0 max 16384 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:12:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:12:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:12:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:12:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4ea00a3e40 msgr2=0x7f4ea00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4ea00a3e40 0x7f4ea00a6300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558aca65bdc0 tx=0x7f4eac0d4600 comp rx=0 tx=0).stop 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc108560 msgr2=0x7f4ebc1a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc108560 0x7f4ebc1a48c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e9f0 tx=0x7f4eac00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 shutdown_connections 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4ea00a3e40 0x7f4ea00a6300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ebc10e620 0x7f4ebc19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ebc108f20 0x7f4ebc1a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 --2- 172.21.15.86:0/15547170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ebc108560 0x7f4ebc1a48c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 >> 172.21.15.86:0/15547170 conn(0x7f4ebc0764e0 msgr2=0x7f4ebc077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 shutdown_connections 2024-04-25T19:12:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:40.756+0000 7f4ec1200640 1 -- 172.21.15.86:0/15547170 wait complete. 2024-04-25T19:12:42.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:42 smithi183 ceph-mon[32344]: pgmap v303: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-25T19:12:42.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:42 smithi194 ceph-mon[32740]: pgmap v303: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-25T19:12:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:42 smithi086 ceph-mon[27783]: pgmap v303: 97 pgs: 97 active+clean; 4.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-25T19:12:44.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:44 smithi183 ceph-mon[32344]: pgmap v304: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-25T19:12:44.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:44 smithi194 ceph-mon[32740]: pgmap v304: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-25T19:12:44.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:44 smithi086 ceph-mon[27783]: pgmap v304: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-25T19:12:46.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:46 smithi183 ceph-mon[32344]: pgmap v305: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:12:46.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:46 smithi194 ceph-mon[32740]: pgmap v305: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:12:46.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:46 smithi086 ceph-mon[27783]: pgmap v305: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:12:46.823 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.823 INFO:tasks.workunit.client.0.smithi086.stdout:251/600: open fd = 3 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:252/600: open fd = 3 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:253/600: open fd = 3 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:254/600: open fd = 3 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:255/600: open fd = 3 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:256/600: open fd = 3 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:257/600: open fd = 3 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:258/600: open fd = 3 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:259/600: open fd = 3 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:260/600: open fd = 3 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:261/600: open fd = 3 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:262/600: open fd = 3 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.826 INFO:tasks.workunit.client.0.smithi086.stdout:263/600: open fd = 3 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:264/600: open fd = 3 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:265/600: open fd = 3 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:266/600: open fd = 3 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:267/600: open fd = 3 2024-04-25T19:12:46.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:268/600: open fd = 3 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:269/600: open fd = 3 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:270/600: open fd = 3 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:271/600: open fd = 3 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:272/600: open fd = 3 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:273/600: open fd = 3 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:274/600: open fd = 3 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:275/600: open fd = 3 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:276/600: open fd = 3 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:277/600: open fd = 3 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:278/600: open fd = 3 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:279/600: open fd = 3 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:280/600: open fd = 3 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:281/600: open fd = 3 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:282/600: open fd = 3 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:283/600: open fd = 3 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:284/600: open fd = 3 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:285/600: open fd = 3 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:286/600: open fd = 3 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:287/600: open fd = 3 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.832 INFO:tasks.workunit.client.0.smithi086.stdout:288/600: open fd = 3 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:289/600: open fd = 3 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:290/600: open fd = 3 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:291/600: open fd = 3 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:292/600: open fd = 3 2024-04-25T19:12:46.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:293/600: open fd = 3 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:294/600: open fd = 3 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:295/600: open fd = 3 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.834 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:296/600: open fd = 3 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:297/600: open fd = 3 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:298/600: open fd = 3 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:299/600: open fd = 3 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.835 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:300/600: open fd = 3 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:301/600: open fd = 3 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:302/600: open fd = 3 2024-04-25T19:12:46.836 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:303/600: open fd = 3 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:304/600: open fd = 3 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:305/600: open fd = 3 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.837 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:306/600: open fd = 3 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:307/600: open fd = 3 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:308/600: open fd = 3 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.838 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:309/600: open fd = 3 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:310/600: open fd = 3 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:311/600: open fd = 3 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:312/600: open fd = 3 2024-04-25T19:12:46.839 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:46.840 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:12:48.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:48 smithi086 ceph-mon[27783]: pgmap v306: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:12:48.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:48 smithi183 ceph-mon[32344]: pgmap v306: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:12:48.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:48 smithi194 ceph-mon[32740]: pgmap v306: 97 pgs: 97 active+clean; 4.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-25T19:12:50.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:50 smithi194 ceph-mon[32740]: pgmap v307: 97 pgs: 97 active+clean; 5.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-04-25T19:12:50.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:50 smithi086 ceph-mon[27783]: pgmap v307: 97 pgs: 97 active+clean; 5.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-04-25T19:12:50.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:50 smithi183 ceph-mon[32344]: pgmap v307: 97 pgs: 97 active+clean; 5.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-04-25T19:12:52.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:52 smithi194 ceph-mon[32740]: pgmap v308: 97 pgs: 97 active+clean; 5.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:12:52.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:52 smithi086 ceph-mon[27783]: pgmap v308: 97 pgs: 97 active+clean; 5.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:12:52.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:52 smithi183 ceph-mon[32344]: pgmap v308: 97 pgs: 97 active+clean; 5.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:12:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.900+0000 7f0b03400640 1 -- 172.21.15.86:0/804858450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 msgr2=0x7f0afc10c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.900+0000 7f0b03400640 1 --2- 172.21.15.86:0/804858450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc10c020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009900 tx=0x7f0ae402a480 comp rx=0 tx=0).stop 2024-04-25T19:12:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 -- 172.21.15.86:0/804858450 shutdown_connections 2024-04-25T19:12:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 --2- 172.21.15.86:0/804858450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc10c560 0x7f0afc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 --2- 172.21.15.86:0/804858450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc10c020 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 --2- 172.21.15.86:0/804858450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0afc103220 0x7f0afc103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 -- 172.21.15.86:0/804858450 >> 172.21.15.86:0/804858450 conn(0x7f0afc077fd0 msgr2=0x7f0afc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 -- 172.21.15.86:0/804858450 shutdown_connections 2024-04-25T19:12:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.901+0000 7f0b03400640 1 -- 172.21.15.86:0/804858450 wait complete. 2024-04-25T19:12:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 Processor -- start 2024-04-25T19:12:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 -- start start 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc103220 0x7f0afc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0afc10c560 0x7f0afc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc112200 con 0x7f0afc103b40 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc112080 con 0x7f0afc10c560 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b03400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc112380 con 0x7f0afc103220 2024-04-25T19:12:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0b00a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc103220 0x7f0afc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48570/0 (socket says 172.21.15.86:48570) 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.902+0000 7f0af3e00640 1 -- 172.21.15.86:0/2638423201 learned_addr learned my addr 172.21.15.86:0/2638423201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b01400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0afc10c560 0x7f0afc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b00a00640 1 -- 172.21.15.86:0/2638423201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0afc10c560 msgr2=0x7f0afc19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b00a00640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0afc10c560 0x7f0afc19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b00a00640 1 -- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 msgr2=0x7f0afc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b00a00640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b00a00640 1 -- 172.21.15.86:0/2638423201 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc19d6f0 con 0x7f0afc103220 2024-04-25T19:12:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0b00a00640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc103220 0x7f0afc198700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00e9f0 tx=0x7f0aec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.903+0000 7f0afa000640 1 -- 172.21.15.86:0/2638423201 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00ccc0 con 0x7f0afc103220 2024-04-25T19:12:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.904+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc19d9e0 con 0x7f0afc103220 2024-04-25T19:12:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.904+0000 7f0afa000640 1 -- 172.21.15.86:0/2638423201 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aec0108e0 con 0x7f0afc103220 2024-04-25T19:12:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.904+0000 7f0afa000640 1 -- 172.21.15.86:0/2638423201 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec018880 con 0x7f0afc103220 2024-04-25T19:12:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.904+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc1a5270 con 0x7f0afc103220 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 Processor -- start 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 -- start start 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 0x561aaf839b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf848e10 0x561aaf84b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561aaf79f900 con 0x561aaf839720 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561aaf745ef0 con 0x561aaf83a040 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b061f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561aaf79b8c0 con 0x561aaf848e10 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0afa000640 1 -- 172.21.15.86:0/2638423201 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0aec010430 con 0x7f0afc103220 2024-04-25T19:12:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.907+0000 7f0b01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 0x561aaf839b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 0x561aaf839b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48576/0 (socket says 172.21.15.86:48576) 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 -- 172.21.15.86:0/2928948869 learned_addr learned my addr 172.21.15.86:0/2928948869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf848e10 0x561aaf84b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b00a00640 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 -- 172.21.15.86:0/2928948869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf848e10 msgr2=0x561aaf84b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf848e10 0x561aaf84b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 -- 172.21.15.86:0/2928948869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 msgr2=0x561aaf8488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 -- 172.21.15.86:0/2928948869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aaf4b2210 con 0x561aaf839720 2024-04-25T19:12:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.908+0000 7f0b01400640 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 0x561aaf839b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009f50 tx=0x7f0af40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1efe98873f6a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0af1c00640 1 -- 172.21.15.86:0/2928948869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4004d20 con 0x561aaf839720 2024-04-25T19:12:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0af1c00640 1 -- 172.21.15.86:0/2928948869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af4004050 con 0x561aaf839720 2024-04-25T19:12:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0af1c00640 1 -- 172.21.15.86:0/2928948869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4005280 con 0x561aaf839720 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0afa000640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ae00a3dc0 0x7f0ae00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0afa000640 1 -- 172.21.15.86:0/2638423201 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f0aec0d0b70 con 0x7f0afc103220 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0b061f0740 1 -- 172.21.15.86:0/2928948869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 msgr2=0x561aaf839b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 0x561aaf839b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009f50 tx=0x7f0af40118d0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.909+0000 7f0b00a00640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ae00a3dc0 0x7f0ae00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 -- 172.21.15.86:0/2928948869 shutdown_connections 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf848e10 0x561aaf84b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2928948869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf839720 0x561aaf839b00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 -- 172.21.15.86:0/2928948869 >> 172.21.15.86:0/2928948869 conn(0x561aaf835d30 msgr2=0x561aaf836110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 -- 172.21.15.86:0/2928948869 shutdown_connections 2024-04-25T19:12:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 -- 172.21.15.86:0/2928948869 wait complete. 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b00a00640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ae00a3dc0 0x7f0ae00a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0ce300 tx=0x7f0aec009700 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 -- start start 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.910+0000 7f0b061f0740 1 Processor -- start 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf845d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 0x561aaf846250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf848e10 0x561aaf846790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561aaf84ba40 con 0x561aaf83a040 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561aaf759310 con 0x561aaf848e10 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561aaf84bbc0 con 0x561aaf839720 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b061f0740 10 client.0 _ll_register_callbacks cb 0x561aaf727cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 0x561aaf846250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 0x561aaf846250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48582/0 (socket says 172.21.15.86:48582) 2024-04-25T19:12:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0af3e00640 1 -- 172.21.15.86:0/2147239278 learned_addr learned my addr 172.21.15.86:0/2147239278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b01400640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf845d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.911+0000 7f0b01400640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf848e10 0x561aaf846790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0af3e00640 1 -- 172.21.15.86:0/2147239278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 msgr2=0x561aaf845d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf845d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0af3e00640 1 -- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf848e10 msgr2=0x561aaf846790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf848e10 0x561aaf846790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0af3e00640 1 -- 172.21.15.86:0/2147239278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aaf57e620 con 0x561aaf83a040 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 0x561aaf846250 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0ae402aa70 tx=0x7f0ae4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfecc5fce3dfd3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0aea000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0aea000640 1 -- 172.21.15.86:0/2147239278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4035040 con 0x561aaf83a040 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0aea000640 1 -- 172.21.15.86:0/2147239278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f0ae4005020 con 0x561aaf83a040 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.912+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af4004050 con 0x561aaf83a040 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0aea000640 1 -- 172.21.15.86:0/2147239278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ae4003e60 con 0x561aaf83a040 2024-04-25T19:12:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561aaf857c80 con 0x561aaf83a040 2024-04-25T19:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0aea000640 1 -- 172.21.15.86:0/2147239278 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af4004050 con 0x561aaf83a040 2024-04-25T19:12:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0aea000640 1 -- 172.21.15.86:0/2147239278 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0ae40314a0 con 0x561aaf83a040 2024-04-25T19:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 20 client.16350 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf859190 0x561aaf85b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 4 client.16350 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.913+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aaf7b38b0 con 0x561aaf859190 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.914+0000 7f0b01400640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf859190 0x561aaf85b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.914+0000 7f0b01400640 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf859190 0x561aaf85b580 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1adbf98a5471e9d server_cookie=93ef67be454cdbdb in_seq=0 out_seq=0 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.914+0000 7f0aea000640 10 client.16350 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0aea000640 1 -- 172.21.15.86:0/2147239278 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561aaf7b38b0 con 0x561aaf859190 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0aea000640 10 client.16350 handle_command_reply: tid=0 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 client.16350 shutdown 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 msgr2=0x561aaf846250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 0x561aaf846250 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0ae402aa70 tx=0x7f0ae4003820 comp rx=0 tx=0).stop 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 shutdown_connections 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf859190 0x561aaf85b580 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf848e10 0x561aaf846790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf83a040 0x561aaf846250 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2147239278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf845d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0aea000640 0 client.16350 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 >> 172.21.15.86:0/2147239278 conn(0x561aaf835d30 msgr2=0x561aaf83bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.915+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 shutdown_connections 2024-04-25T19:12:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.916+0000 7f0b061f0740 1 -- 172.21.15.86:0/2147239278 wait complete. 2024-04-25T19:12:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.916+0000 7f0b061f0740 20 client.16350 trim_cache size 0 max 16384 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 Processor -- start 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 -- start start 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf854900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf860e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1abb50 con 0x561aaf848e10 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1ad690 con 0x561aaf83a040 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.932+0000 7f0b061f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1ac220 con 0x561aaf839720 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0b01400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf854900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:56688/0 (socket says 172.21.15.86:56688) 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 -- 172.21.15.86:0/2515480703 learned_addr learned my addr 172.21.15.86:0/2515480703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0b01400640 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf860e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 -- 172.21.15.86:0/2515480703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 msgr2=0x561aaf854900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf854900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 -- 172.21.15.86:0/2515480703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 msgr2=0x561aaf860e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf860e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0af3e00640 1 -- 172.21.15.86:0/2515480703 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aaf85b8f0 con 0x561aaf83a040 2024-04-25T19:12:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.933+0000 7f0b01400640 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf860e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.934+0000 7f0af3e00640 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8465b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40318f0 tx=0x7f0ae402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d50109ef9a65d8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.934+0000 7f0af1200640 1 -- 172.21.15.86:0/2515480703 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4035030 con 0x561aaf83a040 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.934+0000 7f0af1200640 1 -- 172.21.15.86:0/2515480703 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae40057f0 con 0x561aaf83a040 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.934+0000 7f0af1200640 1 -- 172.21.15.86:0/2515480703 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4056c40 con 0x561aaf83a040 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 -- 172.21.15.86:0/2515480703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 msgr2=0x561aaf8465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8465b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40318f0 tx=0x7f0ae402a480 comp rx=0 tx=0).stop 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 -- 172.21.15.86:0/2515480703 shutdown_connections 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf860e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf83a040 0x561aaf8465b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 --2- 172.21.15.86:0/2515480703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf839720 0x561aaf854900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 -- 172.21.15.86:0/2515480703 >> 172.21.15.86:0/2515480703 conn(0x561aaf835d30 msgr2=0x561aaf841090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 -- 172.21.15.86:0/2515480703 shutdown_connections 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 -- 172.21.15.86:0/2515480703 wait complete. 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 -- start start 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.935+0000 7f0b061f0740 1 Processor -- start 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf839720 0x561aaf850ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 0x561aaf851020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf84e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1acfc0 con 0x561aaf848e10 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1ac8f0 con 0x561aaf839720 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1a9b50 con 0x561aaf83a040 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b061f0740 10 client.0 _ll_register_callbacks cb 0x7f0ae40044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 0x561aaf851020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b01400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf839720 0x561aaf850ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 0x561aaf851020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:37692/0 (socket says 172.21.15.86:37692) 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 -- 172.21.15.86:0/1619299612 learned_addr learned my addr 172.21.15.86:0/1619299612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b01400640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf84e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 -- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf839720 msgr2=0x561aaf850ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf839720 0x561aaf850ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 -- 172.21.15.86:0/1619299612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 msgr2=0x561aaf84e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf84e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0af3e00640 1 -- 172.21.15.86:0/1619299612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aaf84e910 con 0x561aaf83a040 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.936+0000 7f0b01400640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf839720 0x561aaf850ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0af3e00640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 0x561aaf851020 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae4031750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb1d0433ef97e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0aea000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0aea000640 1 -- 172.21.15.86:0/1619299612 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4056670 con 0x561aaf83a040 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0aea000640 1 -- 172.21.15.86:0/1619299612 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7f0ae4056980 con 0x561aaf83a040 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ae40057f0 con 0x561aaf83a040 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0aea000640 1 -- 172.21.15.86:0/1619299612 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ae4067050 con 0x561aaf83a040 2024-04-25T19:12:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0b061f0740 10 client.26452 fetch_fsmap learned FSMap version 29 2024-04-25T19:12:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.937+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561aaf8bf180 con 0x561aaf83a040 2024-04-25T19:12:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0aea000640 1 -- 172.21.15.86:0/1619299612 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae40057f0 con 0x561aaf83a040 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0aea000640 1 -- 172.21.15.86:0/1619299612 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0ae403cbc0 con 0x561aaf83a040 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0b061f0740 10 client.26452 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0b061f0740 10 client.26452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0b061f0740 20 client.26452 populate_metadata read hostname 'smithi086' 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0b061f0740 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf8bf7b0 0x561aaf8c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0b061f0740 4 client.26452 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.938+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aaf7ad320 con 0x561aaf8bf7b0 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.939+0000 7f0b01400640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf8bf7b0 0x561aaf8c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.940+0000 7f0b01400640 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf8bf7b0 0x561aaf8c1bd0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90849b8fa226996 server_cookie=db1b214837e329b2 in_seq=0 out_seq=0 2024-04-25T19:12:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.940+0000 7f0aea000640 10 client.26452 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0aea000640 1 -- 172.21.15.86:0/1619299612 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561aaf7ad320 con 0x561aaf8bf7b0 2024-04-25T19:12:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0aea000640 10 client.26452 handle_command_reply: tid=0 2024-04-25T19:12:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0b061f0740 1 client.26452 shutdown 2024-04-25T19:12:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 msgr2=0x561aaf851020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0b061f0740 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 0x561aaf851020 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae4031750 comp rx=0 tx=0).stop 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 shutdown_connections 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0b061f0740 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561aaf8bf7b0 0x561aaf8c1bd0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.954+0000 7f0b061f0740 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561aaf848e10 0x561aaf84e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0b061f0740 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561aaf83a040 0x561aaf851020 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0b061f0740 1 --2- 172.21.15.86:0/1619299612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561aaf839720 0x561aaf850ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 >> 172.21.15.86:0/1619299612 conn(0x561aaf835d30 msgr2=0x561aaf834600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0aea000640 0 client.26452 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 shutdown_connections 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0b061f0740 1 -- 172.21.15.86:0/1619299612 wait complete. 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.955+0000 7f0b061f0740 20 client.26452 trim_cache size 0 max 16384 2024-04-25T19:12:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:12:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:12:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:12:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:12:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ae00a3dc0 msgr2=0x7f0ae00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ae00a3dc0 0x7f0ae00a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0ce300 tx=0x7f0aec009700 comp rx=0 tx=0).stop 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc103220 msgr2=0x7f0afc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc103220 0x7f0afc198700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00e9f0 tx=0x7f0aec00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 shutdown_connections 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ae00a3dc0 0x7f0ae00a6280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0afc10c560 0x7f0afc19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0afc103b40 0x7f0afc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 --2- 172.21.15.86:0/2638423201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0afc103220 0x7f0afc198700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 >> 172.21.15.86:0/2638423201 conn(0x7f0afc077fd0 msgr2=0x7f0afc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 shutdown_connections 2024-04-25T19:12:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:12:53.956+0000 7f0b03400640 1 -- 172.21.15.86:0/2638423201 wait complete. 2024-04-25T19:12:54.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:54 smithi194 ceph-mon[32740]: pgmap v309: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:12:54.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:54 smithi086 ceph-mon[27783]: pgmap v309: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:12:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:54 smithi183 ceph-mon[32344]: pgmap v309: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:12:56.482 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:56 smithi194 ceph-mon[32740]: pgmap v310: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:12:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:56 smithi086 ceph-mon[27783]: pgmap v310: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:12:56.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:56 smithi183 ceph-mon[32344]: pgmap v310: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:12:57.175 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.5 2024-04-25T19:12:57.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:12:57.634 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:12:58.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:12:58 smithi086 ceph-mon[27783]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:12:58.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:12:58 smithi194 ceph-mon[32740]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:12:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:12:58 smithi183 ceph-mon[32344]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 -- 172.21.15.86:0/263006701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb806b0f0 msgr2=0x7fdcb810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 --2- 172.21.15.86:0/263006701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb806b0f0 0x7fdcb810f900 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b0a0 tx=0x7fdcb000b3b0 comp rx=0 tx=0).stop 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 -- 172.21.15.86:0/263006701 shutdown_connections 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 --2- 172.21.15.86:0/263006701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb806b0f0 0x7fdcb810f900 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 --2- 172.21.15.86:0/263006701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb8108a30 0x7fdcb8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 --2- 172.21.15.86:0/263006701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 -- 172.21.15.86:0/263006701 >> 172.21.15.86:0/263006701 conn(0x7fdcb80788f0 msgr2=0x7fdcb8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 -- 172.21.15.86:0/263006701 shutdown_connections 2024-04-25T19:12:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.960+0000 7fdcbd000700 1 -- 172.21.15.86:0/263006701 wait complete. 2024-04-25T19:12:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.961+0000 7fdcbd000700 1 Processor -- start 2024-04-25T19:12:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.961+0000 7fdcbd000700 1 -- start start 2024-04-25T19:12:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb806b0f0 0x7fdcb819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb8108a30 0x7fdcb819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcbd000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112bd0 con 0x7fdcb806b0f0 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcbd000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112a50 con 0x7fdcb8108a30 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcbd000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112d50 con 0x7fdcb806a750 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcb5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcb5e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58602/0 (socket says 172.21.15.86:58602) 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcb5e00700 1 -- 172.21.15.86:0/3156979758 learned_addr learned my addr 172.21.15.86:0/3156979758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcafe00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb806b0f0 0x7fdcb819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.962+0000 7fdcb6800700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb8108a30 0x7fdcb819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb5e00700 1 -- 172.21.15.86:0/3156979758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb8108a30 msgr2=0x7fdcb819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb5e00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb8108a30 0x7fdcb819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb5e00700 1 -- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb806b0f0 msgr2=0x7fdcb819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:12:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb5e00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb806b0f0 0x7fdcb819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb5e00700 1 -- 172.21.15.86:0/3156979758 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb81a0290 con 0x7fdcb806a750 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcafe00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb806b0f0 0x7fdcb819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb6800700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb8108a30 0x7fdcb819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcb5e00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb819b280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdca800eb30 tx=0x7fdca800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800cbd0 con 0x7fdcb806a750 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb81a0580 con 0x7fdcb806a750 2024-04-25T19:12:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca80044d0 con 0x7fdcb806a750 2024-04-25T19:12:59.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.963+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8003e30 con 0x7fdcb806a750 2024-04-25T19:12:59.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.964+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81a7e00 con 0x7fdcb806a750 2024-04-25T19:12:59.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.965+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb8069730 con 0x7fdcb806a750 2024-04-25T19:12:59.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.967+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdca8004000 con 0x7fdcb806a750 2024-04-25T19:12:59.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.969+0000 7fdcaea00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdc9c0a3e30 0x7fdc9c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:12:59.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.969+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8637+0+0 (secure 0 0 0) 0x7fdca80d0c60 con 0x7fdcb806a750 2024-04-25T19:12:59.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.969+0000 7fdcafe00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdc9c0a3e30 0x7fdc9c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:12:59.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.970+0000 7fdcafe00700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdc9c0a3e30 0x7fdc9c0a62f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdcb810fd20 tx=0x7fdca0009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:12:59.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:12:59.971+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdca80042f0 con 0x7fdcb806a750 2024-04-25T19:13:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.244+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcb81a8090 con 0x7fdcb806a750 2024-04-25T19:13:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.246+0000 7fdcaea00700 1 -- 172.21.15.86:0/3156979758 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fdca809d2c0 con 0x7fdcb806a750 2024-04-25T19:13:00.248 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:13:00.248 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:13:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.248+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdc9c0a3e30 msgr2=0x7fdc9c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.248+0000 7fdcbd000700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdc9c0a3e30 0x7fdc9c0a62f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdcb810fd20 tx=0x7fdca0009260 comp rx=0 tx=0).stop 2024-04-25T19:13:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 msgr2=0x7fdcb819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb819b280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdca800eb30 tx=0x7fdca800ee40 comp rx=0 tx=0).stop 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 shutdown_connections 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcb8108a30 0x7fdcb819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdc9c0a3e30 0x7fdc9c0a62f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcb806a750 0x7fdcb819b280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 --2- 172.21.15.86:0/3156979758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcb806b0f0 0x7fdcb819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 >> 172.21.15.86:0/3156979758 conn(0x7fdcb80788f0 msgr2=0x7fdcb80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 shutdown_connections 2024-04-25T19:13:00.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:00.249+0000 7fdcbd000700 1 -- 172.21.15.86:0/3156979758 wait complete. 2024-04-25T19:13:00.252 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:13:00.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:00 smithi194 ceph-mon[32740]: pgmap v312: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:13:00.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:00 smithi086 ceph-mon[27783]: pgmap v312: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:13:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:00 smithi183 ceph-mon[32344]: pgmap v312: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:13:01.076 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 75.5 --await-for 42.0' 2024-04-25T19:13:01.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 75.5 --await-for 42.0 2024-04-25T19:13:01.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:01 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3156979758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:13:01.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:01 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3156979758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:13:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:13:01.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:01 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3156979758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:13:02.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:02 smithi086 ceph-mon[27783]: pgmap v313: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:13:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:02 smithi086 ceph-mon[27783]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T19:13:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:02 smithi194 ceph-mon[32740]: pgmap v313: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:13:02.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:02 smithi194 ceph-mon[32740]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T19:13:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:02 smithi183 ceph-mon[32344]: pgmap v313: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:13:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:02 smithi183 ceph-mon[32344]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T19:13:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 -- 172.21.15.86:0/3907574318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 msgr2=0x7f4c741098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 --2- 172.21.15.86:0/3907574318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c741098c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4c600099c0 tx=0x7f4c60009cd0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 -- 172.21.15.86:0/3907574318 shutdown_connections 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 --2- 172.21.15.86:0/3907574318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c74109fa0 0x7f4c74111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 --2- 172.21.15.86:0/3907574318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c74104ec0 0x7f4c741052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 --2- 172.21.15.86:0/3907574318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c741098c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 -- 172.21.15.86:0/3907574318 >> 172.21.15.86:0/3907574318 conn(0x7f4c74100750 msgr2=0x7f4c74102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 -- 172.21.15.86:0/3907574318 shutdown_connections 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.694+0000 7f4c78c00700 1 -- 172.21.15.86:0/3907574318 wait complete. 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 Processor -- start 2024-04-25T19:13:03.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 -- start start 2024-04-25T19:13:03.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c7419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74114e60 con 0x7f4c74105870 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74114ce0 con 0x7f4c74109fa0 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.696+0000 7f4c78c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74114fe0 con 0x7f4c74104ec0 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c7419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c7419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38098/0 (socket says 172.21.15.86:38098) 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 -- 172.21.15.86:0/2714827295 learned_addr learned my addr 172.21.15.86:0/2714827295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c72800700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 -- 172.21.15.86:0/2714827295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c74104ec0 msgr2=0x7f4c7419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 -- 172.21.15.86:0/2714827295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c74109fa0 msgr2=0x7f4c7419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.697+0000 7f4c71400700 1 -- 172.21.15.86:0/2714827295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c741a0260 con 0x7f4c74105870 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.698+0000 7f4c71400700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c7419b790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000b580 tx=0x7f4c60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.698+0000 7f4c6aa00700 1 -- 172.21.15.86:0/2714827295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60009e60 con 0x7f4c74105870 2024-04-25T19:13:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.698+0000 7f4c6aa00700 1 -- 172.21.15.86:0/2714827295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c60003c70 con 0x7f4c74105870 2024-04-25T19:13:03.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.698+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c741a04f0 con 0x7f4c74105870 2024-04-25T19:13:03.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.698+0000 7f4c6aa00700 1 -- 172.21.15.86:0/2714827295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60004910 con 0x7f4c74105870 2024-04-25T19:13:03.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.698+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c741a7dd0 con 0x7f4c74105870 2024-04-25T19:13:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.701+0000 7f4c6aa00700 1 -- 172.21.15.86:0/2714827295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c600037c0 con 0x7f4c74105870 2024-04-25T19:13:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.703+0000 7f4c6aa00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c580a3ed0 0x7f4c580a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.703+0000 7f4c8364fb80 1 Processor -- start 2024-04-25T19:13:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.703+0000 7f4c6aa00700 1 -- 172.21.15.86:0/2714827295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f4c600d1140 con 0x7f4c74105870 2024-04-25T19:13:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.703+0000 7f4c71e00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c580a3ed0 0x7f4c580a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.703+0000 7f4c8364fb80 1 -- start start 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 0x55c790c75020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c75560 0x55c790c77950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c790bbd020 con 0x55c790c75560 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c790b638c0 con 0x55c790c6a6f0 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c790bb8fe0 con 0x55c790c69dd0 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c71e00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c580a3ed0 0x7f4c580a6390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009c60 tx=0x7f4c64009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c71400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 0x55c790c75020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c71e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c71400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 0x55c790c75020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48582/0 (socket says 172.21.15.86:48582) 2024-04-25T19:13:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c71400700 1 -- 172.21.15.86:0/3346604893 learned_addr learned my addr 172.21.15.86:0/3346604893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.704+0000 7f4c72800700 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c75560 0x55c790c77950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71e00700 1 -- 172.21.15.86:0/3346604893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 msgr2=0x55c790c75020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71e00700 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 0x55c790c75020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71e00700 1 -- 172.21.15.86:0/3346604893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c75560 msgr2=0x55c790c77950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71e00700 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c75560 0x55c790c77950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71e00700 1 -- 172.21.15.86:0/3346604893 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c790b59810 con 0x55c790c69dd0 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71400700 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 0x55c790c75020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c71e00700 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c6a1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c64005950 tx=0x7f4c64014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48251038d204ea6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c5f400700 1 -- 172.21.15.86:0/3346604893 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400ecf0 con 0x55c790c69dd0 2024-04-25T19:13:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c5f400700 1 -- 172.21.15.86:0/3346604893 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64004ce0 con 0x55c790c69dd0 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.705+0000 7f4c5f400700 1 -- 172.21.15.86:0/3346604893 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64026680 con 0x55c790c69dd0 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 -- 172.21.15.86:0/3346604893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 msgr2=0x55c790c6a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c6a1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c64005950 tx=0x7f4c64014000 comp rx=0 tx=0).stop 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 -- 172.21.15.86:0/3346604893 shutdown_connections 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c75560 0x55c790c77950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c6a1b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/3346604893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c6a6f0 0x55c790c75020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 -- 172.21.15.86:0/3346604893 >> 172.21.15.86:0/3346604893 conn(0x55c790b990e0 msgr2=0x55c790c662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 -- 172.21.15.86:0/3346604893 shutdown_connections 2024-04-25T19:13:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.706+0000 7f4c8364fb80 1 -- 172.21.15.86:0/3346604893 wait complete. 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.707+0000 7f4c8364fb80 1 -- start start 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.707+0000 7f4c8364fb80 1 Processor -- start 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c69dd0 0x55c790c74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c6a6f0 0x55c790c71ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c72400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c790c78050 con 0x55c790c69dd0 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c790b769f0 con 0x55c790c75560 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c790c781d0 con 0x55c790c6a6f0 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c8364fb80 10 client.0 _ll_register_callbacks cb 0x55c790b558b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c71e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c69dd0 0x55c790c74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c71e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c69dd0 0x55c790c74bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38116/0 (socket says 172.21.15.86:38116) 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c71e00700 1 -- 172.21.15.86:0/4130747196 learned_addr learned my addr 172.21.15.86:0/4130747196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:03.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c71400700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c6a6f0 0x55c790c71ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.708+0000 7f4c72800700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c72400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c71400700 1 -- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 msgr2=0x55c790c72400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c71400700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c72400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c71400700 1 -- 172.21.15.86:0/4130747196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c69dd0 msgr2=0x55c790c74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c71400700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c69dd0 0x55c790c74bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c71400700 1 -- 172.21.15.86:0/4130747196 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c790b0aa60 con 0x55c790c6a6f0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c71400700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c6a6f0 0x55c790c71ec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4c600a5750 tx=0x7f4c60003ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b63d58ac123a1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c53e00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c53e00700 1 -- 172.21.15.86:0/4130747196 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600b6060 con 0x55c790c6a6f0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c53e00700 1 -- 172.21.15.86:0/4130747196 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f4c600a10d0 con 0x55c790c6a6f0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.709+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c64004ce0 con 0x55c790c6a6f0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c53e00700 1 -- 172.21.15.86:0/4130747196 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c6009d7a0 con 0x55c790c6a6f0 2024-04-25T19:13:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 10 client.26470 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c790b53c80 con 0x55c790c6a6f0 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c53e00700 1 -- 172.21.15.86:0/4130747196 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64004ce0 con 0x55c790c6a6f0 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c53e00700 1 -- 172.21.15.86:0/4130747196 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4c600a57f0 con 0x55c790c6a6f0 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 10 client.26470 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 10 client.26470 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 20 client.26470 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790c84c90 0x55c790c87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 4 client.26470 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.710+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c790bca810 con 0x55c790c84c90 2024-04-25T19:13:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.711+0000 7f4c71e00700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790c84c90 0x55c790c87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.711+0000 7f4c71e00700 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790c84c90 0x55c790c87080 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f1774733d92900 server_cookie=d9df29142c8a088 in_seq=0 out_seq=0 2024-04-25T19:13:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.711+0000 7f4c53e00700 10 client.26470 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:03.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c53e00700 1 -- 172.21.15.86:0/4130747196 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c790bca810 con 0x55c790c84c90 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c53e00700 10 client.26470 handle_command_reply: tid=0 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 client.26470 shutdown 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c6a6f0 msgr2=0x55c790c71ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c6a6f0 0x55c790c71ec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4c600a5750 tx=0x7f4c60003ab0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 shutdown_connections 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790c84c90 0x55c790c87080 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c72400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c53e00700 0 client.26470 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c69dd0 0x55c790c74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4130747196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c6a6f0 0x55c790c71ec0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.712+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 >> 172.21.15.86:0/4130747196 conn(0x55c790b990e0 msgr2=0x55c790c65e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.713+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 shutdown_connections 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.713+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4130747196 wait complete. 2024-04-25T19:13:03.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.713+0000 7f4c8364fb80 20 client.26470 trim_cache size 0 max 16384 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 Processor -- start 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 -- start start 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c80fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c6d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c8c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741af5a0 con 0x55c790c6a6f0 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b10e0 con 0x55c790c75560 2024-04-25T19:13:03.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.734+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741afc70 con 0x55c790c69dd0 2024-04-25T19:13:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c80fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c6d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c80fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58646/0 (socket says 172.21.15.86:58646) 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 -- 172.21.15.86:0/4069411706 learned_addr learned my addr 172.21.15.86:0/4069411706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c72800700 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c8c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 -- 172.21.15.86:0/4069411706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 msgr2=0x55c790c8c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c8c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 -- 172.21.15.86:0/4069411706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 msgr2=0x55c790c6d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c6d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71e00700 1 -- 172.21.15.86:0/4069411706 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c790c6bc40 con 0x55c790c69dd0 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.735+0000 7f4c71400700 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c6d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:03.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.736+0000 7f4c71e00700 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c80fb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55c790c878f0 tx=0x7f4c6401f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac6e2d42393d03d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.736+0000 7f4c5ea00700 1 -- 172.21.15.86:0/4069411706 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64038070 con 0x55c790c69dd0 2024-04-25T19:13:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.736+0000 7f4c5ea00700 1 -- 172.21.15.86:0/4069411706 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64005070 con 0x55c790c69dd0 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.736+0000 7f4c5ea00700 1 -- 172.21.15.86:0/4069411706 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64003e20 con 0x55c790c69dd0 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4069411706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 msgr2=0x55c790c80fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c80fb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55c790c878f0 tx=0x7f4c6401f040 comp rx=0 tx=0).stop 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4069411706 shutdown_connections 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c8c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c80fb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/4069411706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c6d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.737+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4069411706 >> 172.21.15.86:0/4069411706 conn(0x55c790b990e0 msgr2=0x55c790c73f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.738+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4069411706 shutdown_connections 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.738+0000 7f4c8364fb80 1 -- 172.21.15.86:0/4069411706 wait complete. 2024-04-25T19:13:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.738+0000 7f4c8364fb80 1 -- start start 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.738+0000 7f4c8364fb80 1 Processor -- start 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c7da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c7aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b0a10 con 0x55c790c6a6f0 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741b0340 con 0x55c790c75560 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c741ad560 con 0x55c790c69dd0 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c8364fb80 10 client.0 _ll_register_callbacks cb 0x55c790b558b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:03.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c7da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c7da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38134/0 (socket says 172.21.15.86:38134) 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c72800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c7aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c71400700 1 -- 172.21.15.86:0/2601734964 learned_addr learned my addr 172.21.15.86:0/2601734964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.739+0000 7f4c71e00700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c72800700 1 -- 172.21.15.86:0/2601734964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 msgr2=0x55c790c7d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c72800700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c7d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c72800700 1 -- 172.21.15.86:0/2601734964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 msgr2=0x55c790c7da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c72800700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c7da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c72800700 1 -- 172.21.15.86:0/2601734964 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c790c6ad70 con 0x55c790c75560 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c71400700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c7da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c72800700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c7aa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55c790c6b7a0 tx=0x7f4c6c011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2628aa2556d2c822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c53e00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c53e00700 1 -- 172.21.15.86:0/2601734964 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c007bc0 con 0x55c790c75560 2024-04-25T19:13:03.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c64005070 con 0x55c790c75560 2024-04-25T19:13:03.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.740+0000 7f4c53e00700 1 -- 172.21.15.86:0/2601734964 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f4c6c007d60 con 0x55c790c75560 2024-04-25T19:13:03.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c53e00700 1 -- 172.21.15.86:0/2601734964 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c6c028420 con 0x55c790c75560 2024-04-25T19:13:03.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 10 client.26483 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:03.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c790c66820 con 0x55c790c75560 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c53e00700 1 -- 172.21.15.86:0/2601734964 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c64005070 con 0x55c790c75560 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c53e00700 1 -- 172.21.15.86:0/2601734964 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4c6c016700 con 0x55c790c75560 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 10 client.26483 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 10 client.26483 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 20 client.26483 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790ced2e0 0x55c790cef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 4 client.26483 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.5, "await_for": 42.0}] 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.741+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.5, "await_for": 42.0}) -- 0x55c790bc48b0 con 0x55c790ced2e0 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.742+0000 7f4c71e00700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790ced2e0 0x55c790cef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:03.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.742+0000 7f4c71e00700 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790ced2e0 0x55c790cef700 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570f85259ba00d3d server_cookie=adb5e33749dbd3eb in_seq=0 out_seq=0 2024-04-25T19:13:03.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.742+0000 7f4c53e00700 10 client.26483 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c53e00700 1 -- 172.21.15.86:0/2601734964 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c790bc48b0 con 0x55c790ced2e0 2024-04-25T19:13:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c53e00700 10 client.26483 handle_command_reply: tid=0 2024-04-25T19:13:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 client.26483 shutdown 2024-04-25T19:13:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 msgr2=0x55c790c7aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c7aa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55c790c6b7a0 tx=0x7f4c6c011e60 comp rx=0 tx=0).stop 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 shutdown_connections 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c790ced2e0 0x55c790cef700 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c790c75560 0x55c790c7aa40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c790c69dd0 0x55c790c7d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 --2- 172.21.15.86:0/2601734964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c790c6a6f0 0x55c790c7da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c53e00700 0 client.26483 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.757+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 >> 172.21.15.86:0/2601734964 conn(0x55c790b990e0 msgr2=0x55c790c6c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.758+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 shutdown_connections 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.758+0000 7f4c8364fb80 1 -- 172.21.15.86:0/2601734964 wait complete. 2024-04-25T19:13:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.758+0000 7f4c8364fb80 20 client.26483 trim_cache size 0 max 16384 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 30, 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "b9272999": { 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 30, 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:13:03.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 75.5, 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:03.763 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:13:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c580a3ed0 msgr2=0x7f4c580a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c580a3ed0 0x7f4c580a6390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009c60 tx=0x7f4c64009260 comp rx=0 tx=0).stop 2024-04-25T19:13:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 msgr2=0x7f4c7419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c7419b790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000b580 tx=0x7f4c60004270 comp rx=0 tx=0).stop 2024-04-25T19:13:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 shutdown_connections 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4c580a3ed0 0x7f4c580a6390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 --2- 172.21.15.86:0/2714827295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c74105870 0x7f4c7419b790 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 >> 172.21.15.86:0/2714827295 conn(0x7f4c74100750 msgr2=0x7f4c741011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 shutdown_connections 2024-04-25T19:13:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:03.759+0000 7f4c78c00700 1 -- 172.21.15.86:0/2714827295 wait complete. 2024-04-25T19:13:04.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:04 smithi086 ceph-mon[27783]: pgmap v315: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-25T19:13:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:04 smithi086 ceph-mon[27783]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T19:13:04.666 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b9272999', quiesce took 7.5 seconds. Will release after: 15.5 2024-04-25T19:13:04.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:04 smithi183 ceph-mon[32344]: pgmap v315: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-25T19:13:04.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:04 smithi183 ceph-mon[32344]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T19:13:04.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:04 smithi194 ceph-mon[32740]: pgmap v315: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-25T19:13:04.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:04 smithi194 ceph-mon[32740]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T19:13:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:06 smithi086 ceph-mon[27783]: pgmap v317: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 126 KiB/s wr, 48 op/s 2024-04-25T19:13:06.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:06 smithi183 ceph-mon[32344]: pgmap v317: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 126 KiB/s wr, 48 op/s 2024-04-25T19:13:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:06 smithi194 ceph-mon[32740]: pgmap v317: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 126 KiB/s wr, 48 op/s 2024-04-25T19:13:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 -- 172.21.15.86:0/3599836407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d4109470 msgr2=0x7f41d4111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 --2- 172.21.15.86:0/3599836407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d4109470 0x7f41d4111040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009be0 tx=0x7f41cc02a6c0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 -- 172.21.15.86:0/3599836407 shutdown_connections 2024-04-25T19:13:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 --2- 172.21.15.86:0/3599836407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d4109470 0x7f41d4111040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 --2- 172.21.15.86:0/3599836407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d4104d50 0x7f41d4108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 --2- 172.21.15.86:0/3599836407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d41043a0 0x7f41d4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 -- 172.21.15.86:0/3599836407 >> 172.21.15.86:0/3599836407 conn(0x7f41d40ffc40 msgr2=0x7f41d4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 -- 172.21.15.86:0/3599836407 shutdown_connections 2024-04-25T19:13:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.096+0000 7f41d9800640 1 -- 172.21.15.86:0/3599836407 wait complete. 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.097+0000 7f41d9800640 1 Processor -- start 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 -- start start 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d41043a0 0x7f41d41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 0x7f41d41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d419a720 0x7f41d419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41143e0 con 0x7f41d4109470 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4114260 con 0x7f41d419a720 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d9800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4114560 con 0x7f41d41043a0 2024-04-25T19:13:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 0x7f41d41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 0x7f41d41a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38158/0 (socket says 172.21.15.86:38158) 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41ca000640 1 -- 172.21.15.86:0/3922520475 learned_addr learned my addr 172.21.15.86:0/3922520475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.098+0000 7f41d2a00640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d41043a0 0x7f41d41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41d3400640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d419a720 0x7f41d419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41ca000640 1 -- 172.21.15.86:0/3922520475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d41043a0 msgr2=0x7f41d41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41ca000640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d41043a0 0x7f41d41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41ca000640 1 -- 172.21.15.86:0/3922520475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d419a720 msgr2=0x7f41d419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41ca000640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d419a720 0x7f41d419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41ca000640 1 -- 172.21.15.86:0/3922520475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d419b320 con 0x7f41d4109470 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41ca000640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 0x7f41d41a0ba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9f0 tx=0x7f41bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41cbe00640 1 -- 172.21.15.86:0/3922520475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00ccc0 con 0x7f41d4109470 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d41a7400 con 0x7f41d4109470 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41cbe00640 1 -- 172.21.15.86:0/3922520475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41bc0054d0 con 0x7f41d4109470 2024-04-25T19:13:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.099+0000 7f41cbe00640 1 -- 172.21.15.86:0/3922520475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc010430 con 0x7f41d4109470 2024-04-25T19:13:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.100+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d41a77b0 con 0x7f41d4109470 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 Processor -- start 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41cbe00640 1 -- 172.21.15.86:0/3922520475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41bc0046c0 con 0x7f41d4109470 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 -- start start 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646dacc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561646d12900 con 0x561646db7a00 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561646cb8ef0 con 0x561646dad140 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.103+0000 7f41dc678740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561646d0e8c0 con 0x561646dac820 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646dacc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48632/0 (socket says 172.21.15.86:48632) 2024-04-25T19:13:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 -- 172.21.15.86:0/3004167356 learned_addr learned my addr 172.21.15.86:0/3004167356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 -- 172.21.15.86:0/3004167356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 msgr2=0x561646dacc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646dacc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 -- 172.21.15.86:0/3004167356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 msgr2=0x561646db9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 -- 172.21.15.86:0/3004167356 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561646a25210 con 0x561646dad140 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41cbe00640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f41b80a3dc0 0x7f41b80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.104+0000 7f41d2a00640 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db74c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009cc0 tx=0x7f41c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5204d32e2d76f006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41cbe00640 1 -- 172.21.15.86:0/3922520475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f41bc0d0bc0 con 0x7f41d4109470 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41c3e00640 1 -- 172.21.15.86:0/3004167356 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4005220 con 0x561646dad140 2024-04-25T19:13:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41c3e00640 1 -- 172.21.15.86:0/3004167356 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c4004c50 con 0x561646dad140 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41c3e00640 1 -- 172.21.15.86:0/3004167356 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c40197b0 con 0x561646dad140 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41d3400640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f41b80a3dc0 0x7f41b80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41dc678740 1 -- 172.21.15.86:0/3004167356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 msgr2=0x561646db74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41d3400640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f41b80a3dc0 0x7f41b80a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009370 tx=0x7f41cc004a40 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.105+0000 7f41dc678740 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db74c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009cc0 tx=0x7f41c4011810 comp rx=0 tx=0).stop 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 -- 172.21.15.86:0/3004167356 shutdown_connections 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db74c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 --2- 172.21.15.86:0/3004167356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646dacc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 -- 172.21.15.86:0/3004167356 >> 172.21.15.86:0/3004167356 conn(0x561646da8d30 msgr2=0x561646da9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 -- 172.21.15.86:0/3004167356 shutdown_connections 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 -- 172.21.15.86:0/3004167356 wait complete. 2024-04-25T19:13:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 -- start start 2024-04-25T19:13:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.106+0000 7f41dc678740 1 Processor -- start 2024-04-25T19:13:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646db48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561646dba630 con 0x561646db7a00 2024-04-25T19:13:07.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561646ccc310 con 0x561646dad140 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561646dba7b0 con 0x561646dac820 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41dc678740 10 client.0 _ll_register_callbacks cb 0x561646c9acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38186/0 (socket says 172.21.15.86:38186) 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646db48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41d3400640 1 -- 172.21.15.86:0/4157012514 learned_addr learned my addr 172.21.15.86:0/4157012514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.107+0000 7f41ca000640 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41d2a00640 1 -- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 msgr2=0x561646db4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41d2a00640 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41d2a00640 1 -- 172.21.15.86:0/4157012514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 msgr2=0x561646db5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41d2a00640 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41d2a00640 1 -- 172.21.15.86:0/4157012514 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561646af1620 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41d2a00640 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646db48a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009230 tx=0x7f41c40050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b15015773ff3d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41c0c00640 1 -- 172.21.15.86:0/4157012514 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004a80 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41c0c00640 1 -- 172.21.15.86:0/4157012514 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f41c4019740 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.108+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c4004c50 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.109+0000 7f41c0c00640 1 -- 172.21.15.86:0/4157012514 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41c4033050 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.109+0000 7f41dc678740 10 client.26488 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.109+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561646dc67b0 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.109+0000 7f41c0c00640 1 -- 172.21.15.86:0/4157012514 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c4004c50 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.109+0000 7f41c0c00640 1 -- 172.21.15.86:0/4157012514 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f41c402e5d0 con 0x561646dac820 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41dc678740 10 client.26488 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41dc678740 10 client.26488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41dc678740 20 client.26488 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41dc678740 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646dc7b50 0x561646dc9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41dc678740 4 client.26488 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561646d268b0 con 0x561646dc7b50 2024-04-25T19:13:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41ca000640 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646dc7b50 0x561646dc9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41ca000640 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646dc7b50 0x561646dc9f40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998394c2b52bf76 server_cookie=36a3c67999e846f7 in_seq=0 out_seq=0 2024-04-25T19:13:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.110+0000 7f41c0c00640 10 client.26488 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.111+0000 7f41c0c00640 1 -- 172.21.15.86:0/4157012514 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561646d268b0 con 0x561646dc7b50 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.111+0000 7f41c0c00640 10 client.26488 handle_command_reply: tid=0 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.111+0000 7f41dc678740 1 client.26488 shutdown 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 msgr2=0x561646db48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646db48a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009230 tx=0x7f41c40050b0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 shutdown_connections 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646dc7b50 0x561646dc9f40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646db7a00 0x561646db5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646dad140 0x561646db4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41c0c00640 0 client.26488 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 --2- 172.21.15.86:0/4157012514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dac820 0x561646db48a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 >> 172.21.15.86:0/4157012514 conn(0x561646da8d30 msgr2=0x561646daea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 shutdown_connections 2024-04-25T19:13:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 1 -- 172.21.15.86:0/4157012514 wait complete. 2024-04-25T19:13:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.112+0000 7f41dc678740 20 client.26488 trim_cache size 0 max 16384 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 Processor -- start 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 -- start start 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646daf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646db4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646db4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41aff30 con 0x561646dac820 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41b1a70 con 0x561646db7a00 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41dc678740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41b0600 con 0x561646dad140 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.128+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646daf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646daf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38200/0 (socket says 172.21.15.86:38200) 2024-04-25T19:13:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 -- 172.21.15.86:0/4267766809 learned_addr learned my addr 172.21.15.86:0/4267766809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41ca000640 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646db4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d3400640 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646db4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 -- 172.21.15.86:0/4267766809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 msgr2=0x561646db4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646db4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 -- 172.21.15.86:0/4267766809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 msgr2=0x561646db4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646db4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 -- 172.21.15.86:0/4267766809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561646db4f40 con 0x561646dac820 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41d2a00640 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646daf1e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f41c40057c0 tx=0x7f41c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f37728d05b246a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41c3400640 1 -- 172.21.15.86:0/4267766809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c403c070 con 0x561646dac820 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.129+0000 7f41c3400640 1 -- 172.21.15.86:0/4267766809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c402edd0 con 0x561646dac820 2024-04-25T19:13:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41c3400640 1 -- 172.21.15.86:0/4267766809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4005380 con 0x561646dac820 2024-04-25T19:13:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 -- 172.21.15.86:0/4267766809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 msgr2=0x561646daf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646daf1e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f41c40057c0 tx=0x7f41c4000bc0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 -- 172.21.15.86:0/4267766809 shutdown_connections 2024-04-25T19:13:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646db4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646db4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 --2- 172.21.15.86:0/4267766809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646daf1e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 -- 172.21.15.86:0/4267766809 >> 172.21.15.86:0/4267766809 conn(0x561646da8d30 msgr2=0x561646da7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 -- 172.21.15.86:0/4267766809 shutdown_connections 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.130+0000 7f41dc678740 1 -- 172.21.15.86:0/4267766809 wait complete. 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 -- start start 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 Processor -- start 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646dbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646dbd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646e34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41b13a0 con 0x561646dac820 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41b0cd0 con 0x561646db7a00 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41adf30 con 0x561646dad140 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.131+0000 7f41dc678740 10 client.0 _ll_register_callbacks cb 0x7f41c4009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646dbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646dbcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38212/0 (socket says 172.21.15.86:38212) 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 -- 172.21.15.86:0/1600503562 learned_addr learned my addr 172.21.15.86:0/1600503562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d3400640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646e34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41ca000640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646dbd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 -- 172.21.15.86:0/1600503562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 msgr2=0x561646dbd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646dbd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 -- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 msgr2=0x561646e34770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646e34770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 -- 172.21.15.86:0/1600503562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561646e34d30 con 0x561646dac820 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41d2a00640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646dbcc00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005f30 tx=0x7f41c4009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e975447880b909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.132+0000 7f41c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:13:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41c0c00640 1 -- 172.21.15.86:0/1600503562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c403c070 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41c0c00640 1 -- 172.21.15.86:0/1600503562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f41c402e440 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c402edd0 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41c0c00640 1 -- 172.21.15.86:0/1600503562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41c402e640 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 10 client.16398 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561646e36020 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41c0c00640 1 -- 172.21.15.86:0/1600503562 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c402edd0 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41c0c00640 1 -- 172.21.15.86:0/1600503562 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f41c4019c40 con 0x561646dac820 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 20 client.16398 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646e36670 0x561646e38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 4 client.16398 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.133+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561646d20320 con 0x561646e36670 2024-04-25T19:13:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.134+0000 7f41ca000640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646e36670 0x561646e38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.135+0000 7f41ca000640 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646e36670 0x561646e38a90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692868f21363e010 server_cookie=9508489c518f7727 in_seq=0 out_seq=0 2024-04-25T19:13:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.135+0000 7f41c0c00640 10 client.16398 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41c0c00640 1 -- 172.21.15.86:0/1600503562 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561646d20320 con 0x561646e36670 2024-04-25T19:13:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41c0c00640 10 client.16398 handle_command_reply: tid=0 2024-04-25T19:13:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 client.16398 shutdown 2024-04-25T19:13:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 msgr2=0x561646dbcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646dbcc00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005f30 tx=0x7f41c4009e70 comp rx=0 tx=0).stop 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 shutdown_connections 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561646e36670 0x561646e38a90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561646db7a00 0x561646e34770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561646dad140 0x561646dbd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 --2- 172.21.15.86:0/1600503562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561646dac820 0x561646dbcc00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41c0c00640 0 client.16398 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.149+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 >> 172.21.15.86:0/1600503562 conn(0x561646da8d30 msgr2=0x561646db60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.150+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 shutdown_connections 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.150+0000 7f41dc678740 1 -- 172.21.15.86:0/1600503562 wait complete. 2024-04-25T19:13:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.150+0000 7f41dc678740 20 client.16398 trim_cache size 0 max 16384 2024-04-25T19:13:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:13:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:13:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:13:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f41b80a3dc0 msgr2=0x7f41b80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f41b80a3dc0 0x7f41b80a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009370 tx=0x7f41cc004a40 comp rx=0 tx=0).stop 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 msgr2=0x7f41d41a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 0x7f41d41a0ba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9f0 tx=0x7f41bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 shutdown_connections 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f41b80a3dc0 0x7f41b80a6280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d419a720 0x7f41d419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4109470 0x7f41d41a0ba0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 --2- 172.21.15.86:0/3922520475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d41043a0 0x7f41d41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 >> 172.21.15.86:0/3922520475 conn(0x7f41d40ffc40 msgr2=0x7f41d4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 shutdown_connections 2024-04-25T19:13:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:07.151+0000 7f41d9800640 1 -- 172.21.15.86:0/3922520475 wait complete. 2024-04-25T19:13:08.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:08 smithi086 ceph-mon[27783]: pgmap v318: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-04-25T19:13:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:08 smithi183 ceph-mon[32344]: pgmap v318: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-04-25T19:13:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:08 smithi194 ceph-mon[32740]: pgmap v318: 97 pgs: 97 active+clean; 5.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-04-25T19:13:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:10 smithi086 ceph-mon[27783]: pgmap v319: 97 pgs: 97 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 107 KiB/s wr, 46 op/s 2024-04-25T19:13:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:10 smithi183 ceph-mon[32344]: pgmap v319: 97 pgs: 97 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 107 KiB/s wr, 46 op/s 2024-04-25T19:13:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:10 smithi194 ceph-mon[32740]: pgmap v319: 97 pgs: 97 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 107 KiB/s wr, 46 op/s 2024-04-25T19:13:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:12 smithi086 ceph-mon[27783]: pgmap v320: 97 pgs: 97 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-04-25T19:13:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:12 smithi183 ceph-mon[32344]: pgmap v320: 97 pgs: 97 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-04-25T19:13:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:12 smithi194 ceph-mon[32740]: pgmap v320: 97 pgs: 97 active+clean; 1.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-04-25T19:13:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:14 smithi086 ceph-mon[27783]: pgmap v321: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T19:13:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:14 smithi183 ceph-mon[32344]: pgmap v321: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T19:13:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:14 smithi194 ceph-mon[32740]: pgmap v321: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T19:13:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:16 smithi086 ceph-mon[27783]: pgmap v322: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-25T19:13:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:16 smithi183 ceph-mon[32344]: pgmap v322: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-25T19:13:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:16 smithi194 ceph-mon[32740]: pgmap v322: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-25T19:13:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:18 smithi086 ceph-mon[27783]: pgmap v323: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:13:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:18 smithi183 ceph-mon[32344]: pgmap v323: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:13:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:18 smithi194 ceph-mon[32740]: pgmap v323: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T19:13:20.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:13:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 -- 172.21.15.86:0/2937427103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 msgr2=0x7fe71c06a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 --2- 172.21.15.86:0/2937427103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 0x7fe71c06a780 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe710009900 tx=0x7fe71002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 -- 172.21.15.86:0/2937427103 shutdown_connections 2024-04-25T19:13:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 --2- 172.21.15.86:0/2937427103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c10e8a0 0x7fe71c110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 --2- 172.21.15.86:0/2937427103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c06acc0 0x7fe71c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 --2- 172.21.15.86:0/2937427103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 0x7fe71c06a780 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 -- 172.21.15.86:0/2937427103 >> 172.21.15.86:0/2937427103 conn(0x7fe71c076b80 msgr2=0x7fe71c076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 -- 172.21.15.86:0/2937427103 shutdown_connections 2024-04-25T19:13:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.297+0000 7fe724200640 1 -- 172.21.15.86:0/2937427103 wait complete. 2024-04-25T19:13:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.298+0000 7fe724200640 1 Processor -- start 2024-04-25T19:13:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 -- start start 2024-04-25T19:13:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 0x7fe71c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 0x7fe71c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c10e8a0 0x7fe71c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1144e0 con 0x7fe71c06acc0 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c114360 con 0x7fe71c10e8a0 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe724200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c114660 con 0x7fe71c102f00 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe721800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 0x7fe71c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe721800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 0x7fe71c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58318/0 (socket says 172.21.15.86:58318) 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe721800640 1 -- 172.21.15.86:0/2664309213 learned_addr learned my addr 172.21.15.86:0/2664309213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.299+0000 7fe720e00640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 0x7fe71c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe722200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c10e8a0 0x7fe71c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe721800640 1 -- 172.21.15.86:0/2664309213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 msgr2=0x7fe71c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe721800640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 0x7fe71c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe721800640 1 -- 172.21.15.86:0/2664309213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c10e8a0 msgr2=0x7fe71c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe721800640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c10e8a0 0x7fe71c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe721800640 1 -- 172.21.15.86:0/2664309213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c19afe0 con 0x7fe71c06acc0 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe722200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c10e8a0 0x7fe71c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.300+0000 7fe721800640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 0x7fe71c1a0660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe710005ab0 tx=0x7fe710003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.301+0000 7fe71a000640 1 -- 172.21.15.86:0/2664309213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71003f070 con 0x7fe71c06acc0 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.301+0000 7fe71a000640 1 -- 172.21.15.86:0/2664309213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe710031b60 con 0x7fe71c06acc0 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.301+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c19b270 con 0x7fe71c06acc0 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.301+0000 7fe71a000640 1 -- 172.21.15.86:0/2664309213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710035030 con 0x7fe71c06acc0 2024-04-25T19:13:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.301+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a7400 con 0x7fe71c06acc0 2024-04-25T19:13:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 Processor -- start 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 -- start start 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe71a000640 1 -- 172.21.15.86:0/2664309213 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe710042070 con 0x7fe71c06acc0 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b178314c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b178321df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b17827a900 con 0x55b178314820 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b178220ef0 con 0x55b17831fa00 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.304+0000 7fe727038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b1782768c0 con 0x55b178315140 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe722200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b178314c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe722200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b178314c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58320/0 (socket says 172.21.15.86:58320) 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe722200640 1 -- 172.21.15.86:0/1882348183 learned_addr learned my addr 172.21.15.86:0/1882348183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b178321df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe721800640 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 -- 172.21.15.86:0/1882348183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 msgr2=0x55b17831f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 -- 172.21.15.86:0/1882348183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 msgr2=0x55b178314c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b178314c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 -- 172.21.15.86:0/1882348183 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b177f8d210 con 0x55b17831fa00 2024-04-25T19:13:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe722200640 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b178314c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.305+0000 7fe720e00640 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b178321df0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe708009bd0 tx=0x7fe708011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7c4c142ae0529a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.306+0000 7fe70ea00640 1 -- 172.21.15.86:0/1882348183 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708004230 con 0x55b17831fa00 2024-04-25T19:13:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.306+0000 7fe70ea00640 1 -- 172.21.15.86:0/1882348183 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe708009e40 con 0x55b17831fa00 2024-04-25T19:13:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.306+0000 7fe70ea00640 1 -- 172.21.15.86:0/1882348183 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708004c60 con 0x55b17831fa00 2024-04-25T19:13:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.306+0000 7fe71a000640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe7040a3dc0 0x7fe7040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.306+0000 7fe71a000640 1 -- 172.21.15.86:0/2664309213 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fe7100f0e90 con 0x7fe71c06acc0 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.306+0000 7fe722200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe7040a3dc0 0x7fe7040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 -- 172.21.15.86:0/1882348183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 msgr2=0x55b178321df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b178321df0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe708009bd0 tx=0x7fe708011810 comp rx=0 tx=0).stop 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 -- 172.21.15.86:0/1882348183 shutdown_connections 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b178321df0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 --2- 172.21.15.86:0/1882348183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b178314c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 -- 172.21.15.86:0/1882348183 >> 172.21.15.86:0/1882348183 conn(0x55b178310d30 msgr2=0x55b178311110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 -- 172.21.15.86:0/1882348183 shutdown_connections 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 -- 172.21.15.86:0/1882348183 wait complete. 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe722200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe7040a3dc0 0x7fe7040a6280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe714009fd0 tx=0x7fe714009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.307+0000 7fe727038740 1 -- start start 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 Processor -- start 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b17831d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b178322630 con 0x55b178314820 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b178234310 con 0x55b17831fa00 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b1783227b0 con 0x55b178315140 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe727038740 10 client.0 _ll_register_callbacks cb 0x55b178202cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.308+0000 7fe721800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe722200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b17831d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe720e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58322/0 (socket says 172.21.15.86:58322) 2024-04-25T19:13:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 -- 172.21.15.86:0/20858758 learned_addr learned my addr 172.21.15.86:0/20858758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 -- 172.21.15.86:0/20858758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 msgr2=0x55b17831cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 -- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 msgr2=0x55b17831d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b17831d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 -- 172.21.15.86:0/20858758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b178059620 con 0x55b178314820 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe722200640 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b17831d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe721800640 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c890 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe7100bce00 tx=0x7fe710009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8b6f42c43f8d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe703400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.309+0000 7fe703400640 1 -- 172.21.15.86:0/20858758 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710049070 con 0x55b178314820 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe703400640 1 -- 172.21.15.86:0/20858758 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fe7100b94e0 con 0x55b178314820 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b17828e8b0 con 0x55b178314820 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe703400640 1 -- 172.21.15.86:0/20858758 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7100bc9b0 con 0x55b178314820 2024-04-25T19:13:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b17832e7d0 con 0x55b178314820 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe703400640 1 -- 172.21.15.86:0/20858758 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b17828e8b0 con 0x55b178314820 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe703400640 1 -- 172.21.15.86:0/20858758 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe71003b460 con 0x55b178314820 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 20 client.16410 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17832fc30 0x55b178332080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 4 client.16410 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.310+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b178288320 con 0x55b17832fc30 2024-04-25T19:13:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.311+0000 7fe720e00640 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17832fc30 0x55b178332080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.311+0000 7fe720e00640 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17832fc30 0x55b178332080 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423e69c096295db1 server_cookie=4968e30864804bdb in_seq=0 out_seq=0 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.311+0000 7fe703400640 10 client.16410 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe703400640 1 -- 172.21.15.86:0/20858758 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b178288320 con 0x55b17832fc30 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe703400640 10 client.16410 handle_command_reply: tid=0 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 client.16410 shutdown 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 msgr2=0x55b17831c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c890 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe7100bce00 tx=0x7fe710009620 comp rx=0 tx=0).stop 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 shutdown_connections 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17832fc30 0x55b178332080 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b17831fa00 0x55b17831d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 --2- 172.21.15.86:0/20858758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c890 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 >> 172.21.15.86:0/20858758 conn(0x55b178310d30 msgr2=0x55b178316ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.312+0000 7fe703400640 0 client.16410 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.313+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 shutdown_connections 2024-04-25T19:13:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.313+0000 7fe727038740 1 -- 172.21.15.86:0/20858758 wait complete. 2024-04-25T19:13:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.313+0000 7fe727038740 20 client.16410 trim_cache size 0 max 16384 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 Processor -- start 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 -- start start 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 0x55b1783171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1afe50 con 0x55b178314820 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b1990 con 0x55b178310d30 2024-04-25T19:13:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.329+0000 7fe727038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b0520 con 0x55b178315140 2024-04-25T19:13:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe720e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 0x55b1783171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe722200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 0x55b1783171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48238/0 (socket says 172.21.15.86:48238) 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe720e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58330/0 (socket says 172.21.15.86:58330) 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe722200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35134/0 (socket says 172.21.15.86:35134) 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 -- 172.21.15.86:0/2154194547 learned_addr learned my addr 172.21.15.86:0/2154194547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 -- 172.21.15.86:0/2154194547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 msgr2=0x55b17831cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 -- 172.21.15.86:0/2154194547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 msgr2=0x55b17831c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 -- 172.21.15.86:0/2154194547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17830b2a0 con 0x55b178310d30 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe721800640 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 0x55b1783171e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7100c5500 tx=0x7fe7100f1220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f6d2b53d2ddb40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.330+0000 7fe70e000640 1 -- 172.21.15.86:0/2154194547 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710049040 con 0x55b178310d30 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe70e000640 1 -- 172.21.15.86:0/2154194547 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7100d5020 con 0x55b178310d30 2024-04-25T19:13:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe70e000640 1 -- 172.21.15.86:0/2154194547 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100bc7d0 con 0x55b178310d30 2024-04-25T19:13:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 -- 172.21.15.86:0/2154194547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 msgr2=0x55b1783171e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 0x55b1783171e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7100c5500 tx=0x7fe7100f1220 comp rx=0 tx=0).stop 2024-04-25T19:13:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 -- 172.21.15.86:0/2154194547 shutdown_connections 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178315140 0x55b17831cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178314820 0x55b17831c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 --2- 172.21.15.86:0/2154194547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178310d30 0x55b1783171e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.331+0000 7fe727038740 1 -- 172.21.15.86:0/2154194547 >> 172.21.15.86:0/2154194547 conn(0x55b17830c260 msgr2=0x55b17831c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.332+0000 7fe727038740 1 -- 172.21.15.86:0/2154194547 shutdown_connections 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.332+0000 7fe727038740 1 -- 172.21.15.86:0/2154194547 wait complete. 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.332+0000 7fe727038740 1 -- start start 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.332+0000 7fe727038740 1 Processor -- start 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.332+0000 7fe727038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 0x55b178327a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.332+0000 7fe727038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178314820 0x55b178327f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe727038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178315140 0x55b17831ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe727038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b12c0 con 0x55b178310d30 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe727038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1b0bf0 con 0x55b178315140 2024-04-25T19:13:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe727038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1ade50 con 0x55b178314820 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe727038740 10 client.0 _ll_register_callbacks cb 0x7fe710009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 0x55b178327a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 0x55b178327a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58338/0 (socket says 172.21.15.86:58338) 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 -- 172.21.15.86:0/118643358 learned_addr learned my addr 172.21.15.86:0/118643358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe720e00640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178314820 0x55b178327f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe722200640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178315140 0x55b17831ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 -- 172.21.15.86:0/118643358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178314820 msgr2=0x55b178327f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178314820 0x55b178327f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 -- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178315140 msgr2=0x55b17831ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178315140 0x55b17831ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 -- 172.21.15.86:0/118643358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b178320620 con 0x55b178310d30 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe722200640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178315140 0x55b17831ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:13:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe721800640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 0x55b178327a00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7100c55c0 tx=0x7fe7100c5960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb5da0ff1288ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.333+0000 7fe703400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe703400640 1 -- 172.21.15.86:0/118643358 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710049070 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe703400640 1 -- 172.21.15.86:0/118643358 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fe71003b460 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7100d5020 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe703400640 1 -- 172.21.15.86:0/118643358 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7100bc7d0 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe727038740 10 client.16416 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b178320620 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe703400640 1 -- 172.21.15.86:0/118643358 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7100d5020 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe703400640 1 -- 172.21.15.86:0/118643358 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe7100bc2c0 con 0x55b178310d30 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe727038740 10 client.16416 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe727038740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.334+0000 7fe727038740 20 client.16416 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.335+0000 7fe727038740 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17831fa00 0x55b178397eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.335+0000 7fe727038740 4 client.16416 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:13:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.335+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b178282190 con 0x55b17831fa00 2024-04-25T19:13:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.335+0000 7fe720e00640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17831fa00 0x55b178397eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.335+0000 7fe720e00640 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17831fa00 0x55b178397eb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335fad674f64ce6a server_cookie=a66d033fe42f26c1 in_seq=0 out_seq=0 2024-04-25T19:13:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.335+0000 7fe703400640 10 client.16416 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.347+0000 7fe703400640 1 -- 172.21.15.86:0/118643358 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b178282190 con 0x55b17831fa00 2024-04-25T19:13:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.347+0000 7fe703400640 10 client.16416 handle_command_reply: tid=0 2024-04-25T19:13:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 client.16416 shutdown 2024-04-25T19:13:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 msgr2=0x55b178327a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 0x55b178327a00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe7100c55c0 tx=0x7fe7100c5960 comp rx=0 tx=0).stop 2024-04-25T19:13:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 shutdown_connections 2024-04-25T19:13:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b17831fa00 0x55b178397eb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b178315140 0x55b17831ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b178314820 0x55b178327f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 --2- 172.21.15.86:0/118643358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b178310d30 0x55b178327a00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe703400640 0 client.16416 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 >> 172.21.15.86:0/118643358 conn(0x55b17830c260 msgr2=0x55b17831d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 shutdown_connections 2024-04-25T19:13:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 1 -- 172.21.15.86:0/118643358 wait complete. 2024-04-25T19:13:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.348+0000 7fe727038740 20 client.16416 trim_cache size 0 max 16384 2024-04-25T19:13:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:13:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:13:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:13:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe7040a3dc0 msgr2=0x7fe7040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe7040a3dc0 0x7fe7040a6280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe714009fd0 tx=0x7fe714009500 comp rx=0 tx=0).stop 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 msgr2=0x7fe71c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 0x7fe71c1a0660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe710005ab0 tx=0x7fe710003c10 comp rx=0 tx=0).stop 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 shutdown_connections 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe7040a3dc0 0x7fe7040a6280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe71c10e8a0 0x7fe71c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe71c102f00 0x7fe71c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 --2- 172.21.15.86:0/2664309213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe71c06acc0 0x7fe71c1a0660 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 >> 172.21.15.86:0/2664309213 conn(0x7fe71c076b80 msgr2=0x7fe71c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 shutdown_connections 2024-04-25T19:13:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:20.349+0000 7fe724200640 1 -- 172.21.15.86:0/2664309213 wait complete. 2024-04-25T19:13:20.629 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:13:20.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:20 smithi086 ceph-mon[27783]: pgmap v324: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T19:13:20.784 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:20 smithi194 ceph-mon[32740]: pgmap v324: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T19:13:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:20 smithi183 ceph-mon[32344]: pgmap v324: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T19:13:21.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:21 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:13:21.865 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:21 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:13:21.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:21 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:13:22.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:22 smithi086 ceph-mon[27783]: pgmap v325: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 -- 172.21.15.86:0/1798740215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c102300 msgr2=0x7fd03c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 --2- 172.21.15.86:0/1798740215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c102300 0x7fd03c10e7f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b0a0 tx=0x7fd03400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 -- 172.21.15.86:0/1798740215 shutdown_connections 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 --2- 172.21.15.86:0/1798740215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c102300 0x7fd03c10e7f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 --2- 172.21.15.86:0/1798740215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c107990 0x7fd03c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 --2- 172.21.15.86:0/1798740215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c101940 0x7fd03c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 -- 172.21.15.86:0/1798740215 >> 172.21.15.86:0/1798740215 conn(0x7fd03c0788f0 msgr2=0x7fd03c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 -- 172.21.15.86:0/1798740215 shutdown_connections 2024-04-25T19:13:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.799+0000 7fd041c00700 1 -- 172.21.15.86:0/1798740215 wait complete. 2024-04-25T19:13:22.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.800+0000 7fd041c00700 1 Processor -- start 2024-04-25T19:13:22.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.800+0000 7fd041c00700 1 -- start start 2024-04-25T19:13:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd041c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 0x7fd03c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd041c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c102300 0x7fd03c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd041c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c107990 0x7fd03c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd041c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c111b40 con 0x7fd03c102300 2024-04-25T19:13:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd041c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1119c0 con 0x7fd03c107990 2024-04-25T19:13:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd041c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c111cc0 con 0x7fd03c101940 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd03aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 0x7fd03c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd03aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 0x7fd03c1034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35154/0 (socket says 172.21.15.86:35154) 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.801+0000 7fd03aa00700 1 -- 172.21.15.86:0/2807366079 learned_addr learned my addr 172.21.15.86:0/2807366079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03b400700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c107990 0x7fd03c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03a000700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c102300 0x7fd03c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03aa00700 1 -- 172.21.15.86:0/2807366079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c107990 msgr2=0x7fd03c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03aa00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c107990 0x7fd03c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03aa00700 1 -- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c102300 msgr2=0x7fd03c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03aa00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c102300 0x7fd03c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03aa00700 1 -- 172.21.15.86:0/2807366079 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c1a7750 con 0x7fd03c101940 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03a000700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c102300 0x7fd03c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.802+0000 7fd03aa00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 0x7fd03c1034f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9a0 tx=0x7fd02c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.803+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00cbd0 con 0x7fd03c101940 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.803+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c00cd70 con 0x7fd03c101940 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.803+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c018680 con 0x7fd03c101940 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.803+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c1a7a40 con 0x7fd03c101940 2024-04-25T19:13:22.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.803+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c1a7f30 con 0x7fd03c101940 2024-04-25T19:13:22.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.804+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c069730 con 0x7fd03c101940 2024-04-25T19:13:22.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.806+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd02c010430 con 0x7fd03c101940 2024-04-25T19:13:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.807+0000 7fd033400700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0200a3e30 0x7fd0200a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:22.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.807+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fd02c0d0a10 con 0x7fd03c101940 2024-04-25T19:13:22.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.808+0000 7fd03a000700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0200a3e30 0x7fd0200a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:22.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.808+0000 7fd03a000700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0200a3e30 0x7fd0200a62f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0790f0 tx=0x7fd028009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:22.812+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd02c09cfe0 con 0x7fd03c101940 2024-04-25T19:13:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:22 smithi183 ceph-mon[32344]: pgmap v325: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:13:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:22 smithi194 ceph-mon[32740]: pgmap v325: 97 pgs: 97 active+clean; 1.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:13:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.104+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd03c02e140 con 0x7fd03c101940 2024-04-25T19:13:23.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.105+0000 7fd033400700 1 -- 172.21.15.86:0/2807366079 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fd02c085e20 con 0x7fd03c101940 2024-04-25T19:13:23.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:13:23.107 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0200a3e30 msgr2=0x7fd0200a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0200a3e30 0x7fd0200a62f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0790f0 tx=0x7fd028009260 comp rx=0 tx=0).stop 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 msgr2=0x7fd03c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 0x7fd03c1034f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9a0 tx=0x7fd02c00ecb0 comp rx=0 tx=0).stop 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 shutdown_connections 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd03c107990 0x7fd03c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0200a3e30 0x7fd0200a62f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd03c101940 0x7fd03c1034f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 --2- 172.21.15.86:0/2807366079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd03c102300 0x7fd03c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 >> 172.21.15.86:0/2807366079 conn(0x7fd03c0788f0 msgr2=0x7fd03c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 shutdown_connections 2024-04-25T19:13:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:23.108+0000 7fd041c00700 1 -- 172.21.15.86:0/2807366079 wait complete. 2024-04-25T19:13:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:13:23.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2807366079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:23 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2807366079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:23 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2807366079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.983 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.984 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:23.984 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:23 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:24.395 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id b9272999 --release --await' 2024-04-25T19:13:24.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id b9272999 --release --await 2024-04-25T19:13:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:13:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:24 smithi183 ceph-mon[32344]: pgmap v326: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-04-25T19:13:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:24 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T19:13:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:24 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi194 to 176.2M 2024-04-25T19:13:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:24 smithi183 ceph-mon[32344]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T19:13:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:24 smithi183 ceph-mon[32344]: Unable to set osd_memory_target on smithi194 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-25T19:13:24.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:24 smithi086 ceph-mon[27783]: pgmap v326: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-04-25T19:13:24.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:24 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T19:13:24.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:24 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi194 to 176.2M 2024-04-25T19:13:24.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:24 smithi086 ceph-mon[27783]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T19:13:24.962 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:24 smithi086 ceph-mon[27783]: Unable to set osd_memory_target on smithi194 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-25T19:13:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:24 smithi194 ceph-mon[32740]: pgmap v326: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-04-25T19:13:24.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:24 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-25T19:13:24.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:24 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi194 to 176.2M 2024-04-25T19:13:24.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:24 smithi194 ceph-mon[32740]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T19:13:24.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:24 smithi194 ceph-mon[32740]: Unable to set osd_memory_target on smithi194 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-04-25T19:13:26.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:26 smithi086 ceph-mon[27783]: pgmap v327: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-04-25T19:13:26.803 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:26 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:26.804 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:26 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:26 smithi183 ceph-mon[32344]: pgmap v327: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-04-25T19:13:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:26 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:26 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:26 smithi194 ceph-mon[32740]: pgmap v327: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-04-25T19:13:26.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:26 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:26.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:26 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:27.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 -- 172.21.15.86:0/2340111302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 msgr2=0x7fd6e81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 --2- 172.21.15.86:0/2340111302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e81098c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40099c0 tx=0x7fd6d4009cd0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 -- 172.21.15.86:0/2340111302 shutdown_connections 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 --2- 172.21.15.86:0/2340111302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e8109fa0 0x7fd6e8111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 --2- 172.21.15.86:0/2340111302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 0x7fd6e81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 --2- 172.21.15.86:0/2340111302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e81098c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 -- 172.21.15.86:0/2340111302 >> 172.21.15.86:0/2340111302 conn(0x7fd6e8100750 msgr2=0x7fd6e8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 -- 172.21.15.86:0/2340111302 shutdown_connections 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.290+0000 7fd6ef000700 1 -- 172.21.15.86:0/2340111302 wait complete. 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.291+0000 7fd6ef000700 1 Processor -- start 2024-04-25T19:13:27.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.291+0000 7fd6ef000700 1 -- start start 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.291+0000 7fd6ef000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 0x7fd6e819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.291+0000 7fd6ef000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.291+0000 7fd6ef000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e8109fa0 0x7fd6e819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.291+0000 7fd6ef000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114e30 con 0x7fd6e8105870 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6ef000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114cb0 con 0x7fd6e8104ec0 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6ef000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114fb0 con 0x7fd6e8109fa0 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 0x7fd6e819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58378/0 (socket says 172.21.15.86:58378) 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7400700 1 -- 172.21.15.86:0/17802990 learned_addr learned my addr 172.21.15.86:0/17802990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6ed000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e8109fa0 0x7fd6e819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7e00700 1 -- 172.21.15.86:0/17802990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e8109fa0 msgr2=0x7fd6e819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e8109fa0 0x7fd6e819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7e00700 1 -- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 msgr2=0x7fd6e819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.292+0000 7fd6e7e00700 1 -- 172.21.15.86:0/17802990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e81a0310 con 0x7fd6e8104ec0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.293+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 0x7fd6e819b300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800e9f0 tx=0x7fd6d800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.293+0000 7fd6e4c00700 1 -- 172.21.15.86:0/17802990 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800c9c0 con 0x7fd6e8104ec0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.293+0000 7fd6e4c00700 1 -- 172.21.15.86:0/17802990 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d800cb60 con 0x7fd6e8104ec0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.293+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e81a0600 con 0x7fd6e8104ec0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.293+0000 7fd6e4c00700 1 -- 172.21.15.86:0/17802990 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8018680 con 0x7fd6e8104ec0 2024-04-25T19:13:27.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.293+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81a7e80 con 0x7fd6e8104ec0 2024-04-25T19:13:27.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.296+0000 7fd6e4c00700 1 -- 172.21.15.86:0/17802990 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6d8010430 con 0x7fd6e8104ec0 2024-04-25T19:13:27.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6e4c00700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6cc0a3e30 0x7fd6cc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 Processor -- start 2024-04-25T19:13:27.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6e4c00700 1 -- 172.21.15.86:0/17802990 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fd6d80d09b0 con 0x7fd6e8104ec0 2024-04-25T19:13:27.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 -- start start 2024-04-25T19:13:27.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6e7400700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6cc0a3e30 0x7fd6cc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 0x561e3df67d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df72bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df730f0 0x561e3df754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e3debaf90 con 0x561e3df68240 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e3de61830 con 0x561e3df730f0 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.298+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561e3deb6f50 con 0x561e3df67920 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 0x561e3df67d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6ed000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df730f0 0x561e3df754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 0x561e3df67d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35170/0 (socket says 172.21.15.86:35170) 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 -- 172.21.15.86:0/3154747712 learned_addr learned my addr 172.21.15.86:0/3154747712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7400700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6cc0a3e30 0x7fd6cc0a62f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4003760 tx=0x7fd6d4005fb0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7400700 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df72bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 -- 172.21.15.86:0/3154747712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df730f0 msgr2=0x561e3df754e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df730f0 0x561e3df754e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 -- 172.21.15.86:0/3154747712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 msgr2=0x561e3df72bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df72bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.299+0000 7fd6e7e00700 1 -- 172.21.15.86:0/3154747712 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3de57780 con 0x561e3df67920 2024-04-25T19:13:27.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.300+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 0x561e3df67d00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80a4fc0 tx=0x7fd6d809c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848ab19f48fe0875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:27.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.300+0000 7fd6dd600700 1 -- 172.21.15.86:0/3154747712 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d809ab80 con 0x561e3df67920 2024-04-25T19:13:27.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.300+0000 7fd6dd600700 1 -- 172.21.15.86:0/3154747712 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d80870b0 con 0x561e3df67920 2024-04-25T19:13:27.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.300+0000 7fd6dd600700 1 -- 172.21.15.86:0/3154747712 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80a52c0 con 0x561e3df67920 2024-04-25T19:13:27.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/3154747712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 msgr2=0x561e3df67d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 0x561e3df67d00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80a4fc0 tx=0x7fd6d809c4f0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/3154747712 shutdown_connections 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df730f0 0x561e3df754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df67920 0x561e3df67d00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/3154747712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df72bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/3154747712 >> 172.21.15.86:0/3154747712 conn(0x561e3de966a0 msgr2=0x561e3df63dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/3154747712 shutdown_connections 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/3154747712 wait complete. 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.301+0000 7fd6f9a4eb80 1 -- start start 2024-04-25T19:13:27.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 Processor -- start 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df6fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e3df75c10 con 0x561e3df68240 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e3de74960 con 0x561e3df67920 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561e3df75d90 con 0x561e3df730f0 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.302+0000 7fd6f9a4eb80 10 client.0 _ll_register_callbacks cb 0x561e3de53820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df728d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:48276/0 (socket says 172.21.15.86:48276) 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 -- 172.21.15.86:0/2224200488 learned_addr learned my addr 172.21.15.86:0/2224200488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6ed000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7400700 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df6fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 -- 172.21.15.86:0/2224200488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 msgr2=0x561e3df70110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df70110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 -- 172.21.15.86:0/2224200488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 msgr2=0x561e3df6fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df6fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 -- 172.21.15.86:0/2224200488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3de089d0 con 0x561e3df67920 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df728d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8099250 tx=0x7fd6d80cdfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47397d5d01f2c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6d2000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.303+0000 7fd6d2000700 1 -- 172.21.15.86:0/2224200488 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80a5260 con 0x561e3df67920 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d80870b0 con 0x561e3df67920 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6d2000700 1 -- 172.21.15.86:0/2224200488 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fd6d809dbb0 con 0x561e3df67920 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6d2000700 1 -- 172.21.15.86:0/2224200488 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6d8082050 con 0x561e3df67920 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 10 client.26537 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e3de51bf0 con 0x561e3df67920 2024-04-25T19:13:27.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6d2000700 1 -- 172.21.15.86:0/2224200488 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d80870b0 con 0x561e3df67920 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6d2000700 1 -- 172.21.15.86:0/2224200488 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd6d80107b0 con 0x561e3df67920 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 10 client.26537 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 10 client.26537 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 20 client.26537 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3df82820 0x561e3df84c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 4 client.26537 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.304+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e3dec8780 con 0x561e3df82820 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.305+0000 7fd6e7400700 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3df82820 0x561e3df84c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.305+0000 7fd6e7400700 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3df82820 0x561e3df84c10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b1ca0907c9bcdb server_cookie=f39a313ad1c73811 in_seq=0 out_seq=0 2024-04-25T19:13:27.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.305+0000 7fd6d2000700 10 client.26537 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:27.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6d2000700 1 -- 172.21.15.86:0/2224200488 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e3dec8780 con 0x561e3df82820 2024-04-25T19:13:27.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6d2000700 10 client.26537 handle_command_reply: tid=0 2024-04-25T19:13:27.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6f9a4eb80 1 client.26537 shutdown 2024-04-25T19:13:27.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 msgr2=0x561e3df728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df728d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8099250 tx=0x7fd6d80cdfd0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 shutdown_connections 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.306+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3df82820 0x561e3df84c10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df70110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df728d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/2224200488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df68240 0x561e3df6fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 >> 172.21.15.86:0/2224200488 conn(0x561e3de966a0 msgr2=0x561e3df639c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6d2000700 0 client.26537 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 shutdown_connections 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/2224200488 wait complete. 2024-04-25T19:13:27.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.307+0000 7fd6f9a4eb80 20 client.26537 trim_cache size 0 max 16384 2024-04-25T19:13:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.328+0000 7fd6f9a4eb80 1 Processor -- start 2024-04-25T19:13:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.328+0000 7fd6f9a4eb80 1 -- start start 2024-04-25T19:13:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df68240 0x561e3df87850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df730f0 0x561e3df87d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81af780 con 0x561e3df730f0 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b12c0 con 0x561e3df67920 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81afe50 con 0x561e3df68240 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6ed000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df730f0 0x561e3df87d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6ed000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df730f0 0x561e3df87d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58416/0 (socket says 172.21.15.86:58416) 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6ed000700 1 -- 172.21.15.86:0/1729877362 learned_addr learned my addr 172.21.15.86:0/1729877362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7400700 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df68240 0x561e3df87850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7e00700 1 -- 172.21.15.86:0/1729877362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df68240 msgr2=0x561e3df87850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df68240 0x561e3df87850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7e00700 1 -- 172.21.15.86:0/1729877362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df730f0 msgr2=0x561e3df87d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df730f0 0x561e3df87d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.329+0000 7fd6e7e00700 1 -- 172.21.15.86:0/1729877362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3df62b10 con 0x561e3df67920 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.330+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df71b10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800c5b0 tx=0x7fd6d8005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b76c5e7b79c517b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.330+0000 7fd6dcc00700 1 -- 172.21.15.86:0/1729877362 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d809c3c0 con 0x561e3df67920 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.330+0000 7fd6dcc00700 1 -- 172.21.15.86:0/1729877362 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d80870c0 con 0x561e3df67920 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.330+0000 7fd6dcc00700 1 -- 172.21.15.86:0/1729877362 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80983e0 con 0x561e3df67920 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/1729877362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 msgr2=0x561e3df71b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df71b10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800c5b0 tx=0x7fd6d8005bf0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/1729877362 shutdown_connections 2024-04-25T19:13:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df730f0 0x561e3df87d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df67920 0x561e3df71b10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/1729877362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df68240 0x561e3df87850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/1729877362 >> 172.21.15.86:0/1729877362 conn(0x561e3de966a0 msgr2=0x561e3df71400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/1729877362 shutdown_connections 2024-04-25T19:13:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.331+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/1729877362 wait complete. 2024-04-25T19:13:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.332+0000 7fd6f9a4eb80 1 -- start start 2024-04-25T19:13:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.332+0000 7fd6f9a4eb80 1 Processor -- start 2024-04-25T19:13:27.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.332+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 0x561e3df87570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df68240 0x561e3df87ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6f9a4eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b0bf0 con 0x561e3df67920 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b0520 con 0x561e3df68240 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6f9a4eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81ad740 con 0x561e3df730f0 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6f9a4eb80 10 client.0 _ll_register_callbacks cb 0x561e3de53820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 0x561e3df87570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 0x561e3df87570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58418/0 (socket says 172.21.15.86:58418) 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7e00700 1 -- 172.21.15.86:0/4066109156 learned_addr learned my addr 172.21.15.86:0/4066109156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7400700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df68240 0x561e3df87ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6ed000700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7400700 1 -- 172.21.15.86:0/4066109156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 msgr2=0x561e3df85a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7400700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df85a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7400700 1 -- 172.21.15.86:0/4066109156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 msgr2=0x561e3df87570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7400700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 0x561e3df87570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7400700 1 -- 172.21.15.86:0/4066109156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3df86120 con 0x561e3df68240 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.333+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 0x561e3df87570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6e7400700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df68240 0x561e3df87ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009750 tx=0x7fd6d4009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb9b3e574c035cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6d2000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6d2000700 1 -- 172.21.15.86:0/4066109156 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4036070 con 0x561e3df68240 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d80870c0 con 0x561e3df68240 2024-04-25T19:13:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6d2000700 1 -- 172.21.15.86:0/4066109156 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fd6d4032620 con 0x561e3df68240 2024-04-25T19:13:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6d2000700 1 -- 172.21.15.86:0/4066109156 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6d4042050 con 0x561e3df68240 2024-04-25T19:13:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6f9a4eb80 10 client.26543 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.334+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e3dff1dc0 con 0x561e3df68240 2024-04-25T19:13:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6d2000700 1 -- 172.21.15.86:0/4066109156 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d80870c0 con 0x561e3df68240 2024-04-25T19:13:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6d2000700 1 -- 172.21.15.86:0/4066109156 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd6d4024470 con 0x561e3df68240 2024-04-25T19:13:27.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6f9a4eb80 10 client.26543 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6f9a4eb80 10 client.26543 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6f9a4eb80 20 client.26543 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3dff2480 0x561e3dff48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6f9a4eb80 4 client.26543 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b9272999", "release": true, "await": true}] 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b9272999", "release": true, "await": true}) -- 0x561e3dec2820 con 0x561e3dff2480 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.335+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3dff2480 0x561e3dff48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.336+0000 7fd6e7e00700 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3dff2480 0x561e3dff48a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d6c3feb46196d9 server_cookie=73f7ca015200e822 in_seq=0 out_seq=0 2024-04-25T19:13:27.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.336+0000 7fd6d2000700 10 client.26543 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:27.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6d2000700 1 -- 172.21.15.86:0/4066109156 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561e3dec2820 con 0x561e3dff2480 2024-04-25T19:13:27.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6d2000700 10 client.26543 handle_command_reply: tid=0 2024-04-25T19:13:27.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 client.26543 shutdown 2024-04-25T19:13:27.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df68240 msgr2=0x561e3df87ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df68240 0x561e3df87ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009750 tx=0x7fd6d4009cd0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 shutdown_connections 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x561e3dff2480 0x561e3dff48a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e3df730f0 0x561e3df85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e3df67920 0x561e3df87570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 --2- 172.21.15.86:0/4066109156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e3df68240 0x561e3df87ab0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6d2000700 0 client.26543 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.339+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 >> 172.21.15.86:0/4066109156 conn(0x561e3de966a0 msgr2=0x561e3df6ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 shutdown_connections 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6f9a4eb80 1 -- 172.21.15.86:0/4066109156 wait complete. 2024-04-25T19:13:27.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6f9a4eb80 20 client.26543 trim_cache size 0 max 16384 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 32, 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "b9272999": { 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 32, 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:13:27.343 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 75.5, 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:27.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:27.345 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:13:27.345 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:13:27.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6cc0a3e30 msgr2=0x7fd6cc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6ef000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6cc0a3e30 0x7fd6cc0a62f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4003760 tx=0x7fd6d4005fb0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 msgr2=0x7fd6e819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:27.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.340+0000 7fd6ef000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 0x7fd6e819b300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800e9f0 tx=0x7fd6d800ed00 comp rx=0 tx=0).stop 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 shutdown_connections 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd6cc0a3e30 0x7fd6cc0a62f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e8109fa0 0x7fd6e819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e8104ec0 0x7fd6e819b300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 --2- 172.21.15.86:0/17802990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e8105870 0x7fd6e819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 >> 172.21.15.86:0/17802990 conn(0x7fd6e8100750 msgr2=0x7fd6e8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 shutdown_connections 2024-04-25T19:13:27.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:27.341+0000 7fd6ef000700 1 -- 172.21.15.86:0/17802990 wait complete. 2024-04-25T19:13:27.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:13:27.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:27 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:13:28.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:13:28.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:27 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:13:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:13:28.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:27 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:13:28.189 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b9272999', seconds elapsed: 31.0 2024-04-25T19:13:28.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:28 smithi194 ceph-mon[32740]: pgmap v328: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-04-25T19:13:28.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:28 smithi194 ceph-mon[32740]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-25T19:13:28.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:28 smithi194 ceph-mon[32740]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-25T19:13:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:28 smithi086 ceph-mon[27783]: pgmap v328: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-04-25T19:13:29.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:28 smithi086 ceph-mon[27783]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-25T19:13:29.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:28 smithi086 ceph-mon[27783]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-25T19:13:29.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:28 smithi183 ceph-mon[32344]: pgmap v328: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-04-25T19:13:29.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:28 smithi183 ceph-mon[32344]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-25T19:13:29.170 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:28 smithi183 ceph-mon[32344]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-25T19:13:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:30 smithi194 ceph-mon[32740]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T19:13:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:30 smithi086 ceph-mon[27783]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T19:13:31.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:30 smithi183 ceph-mon[32344]: pgmap v329: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T19:13:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:32 smithi194 ceph-mon[32740]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.3 KiB/s wr, 10 op/s 2024-04-25T19:13:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:32 smithi086 ceph-mon[27783]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.3 KiB/s wr, 10 op/s 2024-04-25T19:13:33.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:32 smithi183 ceph-mon[32344]: pgmap v330: 97 pgs: 97 active+clean; 1.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.3 KiB/s wr, 10 op/s 2024-04-25T19:13:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.492+0000 7fae69600640 1 -- 172.21.15.86:0/1288863442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640318a0 con 0x7fae6c10e930 2024-04-25T19:13:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 -- 172.21.15.86:0/1288863442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 msgr2=0x7fae6c110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 --2- 172.21.15.86:0/1288863442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c110d20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae6402a3f0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 -- 172.21.15.86:0/1288863442 shutdown_connections 2024-04-25T19:13:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 --2- 172.21.15.86:0/1288863442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c110d20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 --2- 172.21.15.86:0/1288863442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c1016a0 0x7fae6c10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 --2- 172.21.15.86:0/1288863442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae6c100d80 0x7fae6c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 -- 172.21.15.86:0/1288863442 >> 172.21.15.86:0/1288863442 conn(0x7fae6c0771d0 msgr2=0x7fae6c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 -- 172.21.15.86:0/1288863442 shutdown_connections 2024-04-25T19:13:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.493+0000 7fae71600640 1 -- 172.21.15.86:0/1288863442 wait complete. 2024-04-25T19:13:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 Processor -- start 2024-04-25T19:13:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 -- start start 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c100d80 0x7fae6c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae6c1016a0 0x7fae6c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c114410 con 0x7fae6c10e930 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c114290 con 0x7fae6c100d80 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.494+0000 7fae71600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c114590 con 0x7fae6c1016a0 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38486/0 (socket says 172.21.15.86:38486) 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 -- 172.21.15.86:0/1148775625 learned_addr learned my addr 172.21.15.86:0/1148775625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6aa00640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c100d80 0x7fae6c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6a000640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae6c1016a0 0x7fae6c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 -- 172.21.15.86:0/1148775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae6c1016a0 msgr2=0x7fae6c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae6c1016a0 0x7fae6c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 -- 172.21.15.86:0/1148775625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c100d80 msgr2=0x7fae6c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c100d80 0x7fae6c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 -- 172.21.15.86:0/1148775625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c19afc0 con 0x7fae6c10e930 2024-04-25T19:13:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6aa00640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c100d80 0x7fae6c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.495+0000 7fae6b400640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c19a7f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fae64005940 tx=0x7fae64031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.496+0000 7fae63400640 1 -- 172.21.15.86:0/1148775625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6403f030 con 0x7fae6c10e930 2024-04-25T19:13:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.496+0000 7fae63400640 1 -- 172.21.15.86:0/1148775625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae640043d0 con 0x7fae6c10e930 2024-04-25T19:13:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.496+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c19b250 con 0x7fae6c10e930 2024-04-25T19:13:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.496+0000 7fae63400640 1 -- 172.21.15.86:0/1148775625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64035070 con 0x7fae6c10e930 2024-04-25T19:13:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.496+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1a7480 con 0x7fae6c10e930 2024-04-25T19:13:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.499+0000 7fae63400640 1 -- 172.21.15.86:0/1148775625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae64042030 con 0x7fae6c10e930 2024-04-25T19:13:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.499+0000 7fae745df740 1 Processor -- start 2024-04-25T19:13:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 -- start start 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e92974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9299df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4e91f2900 con 0x55c4e9297a00 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c4e9198ef0 con 0x55c4e928c820 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae745df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c4e91ee8c0 con 0x55c4e928d140 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9299df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9299df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38488/0 (socket says 172.21.15.86:38488) 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae6a000640 1 -- 172.21.15.86:0/3074667627 learned_addr learned my addr 172.21.15.86:0/3074667627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae6a000640 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.500+0000 7fae6aa00640 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e92974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6a000640 1 -- 172.21.15.86:0/3074667627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 msgr2=0x55c4e92974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6a000640 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e92974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6a000640 1 -- 172.21.15.86:0/3074667627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 msgr2=0x55c4e928cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6a000640 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6a000640 1 -- 172.21.15.86:0/3074667627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e8f05210 con 0x55c4e9297a00 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae63400640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae500a3dc0 0x7fae500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6a000640 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9299df0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae58009560 tx=0x7fae58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba3154ef10f079e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae63400640 1 -- 172.21.15.86:0/1148775625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fae640f0b30 con 0x7fae6c10e930 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae6aa00640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae500a3dc0 0x7fae500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae57400640 1 -- 172.21.15.86:0/3074667627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58004050 con 0x55c4e9297a00 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae57400640 1 -- 172.21.15.86:0/3074667627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5800fc70 con 0x55c4e9297a00 2024-04-25T19:13:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.501+0000 7fae57400640 1 -- 172.21.15.86:0/3074667627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58003920 con 0x55c4e9297a00 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 -- 172.21.15.86:0/3074667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 msgr2=0x55c4e9299df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae6aa00640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae500a3dc0 0x7fae500a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55c4e928ddc0 tx=0x7fae5c009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9299df0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae58009560 tx=0x7fae58004730 comp rx=0 tx=0).stop 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 -- 172.21.15.86:0/3074667627 shutdown_connections 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9299df0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e92974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 --2- 172.21.15.86:0/3074667627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 -- 172.21.15.86:0/3074667627 >> 172.21.15.86:0/3074667627 conn(0x55c4e9288d30 msgr2=0x55c4e9289110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 -- 172.21.15.86:0/3074667627 shutdown_connections 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.502+0000 7fae745df740 1 -- 172.21.15.86:0/3074667627 wait complete. 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.503+0000 7fae745df740 1 -- start start 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.503+0000 7fae745df740 1 Processor -- start 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.503+0000 7fae745df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928c820 0x55c4e92949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.503+0000 7fae745df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e9294f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.503+0000 7fae745df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9295440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae745df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4e929a630 con 0x55c4e9297a00 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae745df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c4e91ac310 con 0x55c4e928d140 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae745df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c4e929a7b0 con 0x55c4e928c820 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae745df740 10 client.0 _ll_register_callbacks cb 0x55c4e917acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9295440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e9294f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9295440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38504/0 (socket says 172.21.15.86:38504) 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6b400640 1 -- 172.21.15.86:0/4293073745 learned_addr learned my addr 172.21.15.86:0/4293073745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6aa00640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928c820 0x55c4e92949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6a000640 1 -- 172.21.15.86:0/4293073745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928c820 msgr2=0x55c4e92949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6a000640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928c820 0x55c4e92949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6a000640 1 -- 172.21.15.86:0/4293073745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 msgr2=0x55c4e9295440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6a000640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9295440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6a000640 1 -- 172.21.15.86:0/4293073745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e8fd1620 con 0x55c4e928d140 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6aa00640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928c820 0x55c4e92949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.504+0000 7fae6b400640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9295440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae6a000640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e9294f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55c4e928a150 tx=0x7fae58009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403f93b71af8a066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae4be00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae4be00640 1 -- 172.21.15.86:0/4293073745 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58029640 con 0x55c4e928d140 2024-04-25T19:13:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5800fc70 con 0x55c4e928d140 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae4be00640 1 -- 172.21.15.86:0/4293073745 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fae5801f440 con 0x55c4e928d140 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae4be00640 1 -- 172.21.15.86:0/4293073745 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae580297e0 con 0x55c4e928d140 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae745df740 10 client.26555 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.505+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4e92a68c0 con 0x55c4e928d140 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae4be00640 1 -- 172.21.15.86:0/4293073745 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5800fc70 con 0x55c4e928d140 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae4be00640 1 -- 172.21.15.86:0/4293073745 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fae58027020 con 0x55c4e928d140 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae745df740 10 client.26555 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae745df740 10 client.26555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae745df740 20 client.26555 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae745df740 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e92a7c10 0x55c4e92aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae745df740 4 client.26555 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4e92068b0 con 0x55c4e92a7c10 2024-04-25T19:13:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.506+0000 7fae6aa00640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e92a7c10 0x55c4e92aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.507+0000 7fae6aa00640 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e92a7c10 0x55c4e92aa000 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2506dfbf38aa3d server_cookie=7dca20744b0ac873 in_seq=0 out_seq=0 2024-04-25T19:13:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.507+0000 7fae4be00640 10 client.26555 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae4be00640 1 -- 172.21.15.86:0/4293073745 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4e92068b0 con 0x55c4e92a7c10 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae4be00640 10 client.26555 handle_command_reply: tid=0 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 client.26555 shutdown 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 msgr2=0x55c4e9294f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e9294f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55c4e928a150 tx=0x7fae58009eb0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 shutdown_connections 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e92a7c10 0x55c4e92aa000 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9295440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e9294f00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 --2- 172.21.15.86:0/4293073745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928c820 0x55c4e92949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae4be00640 0 client.26555 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 >> 172.21.15.86:0/4293073745 conn(0x55c4e9288d30 msgr2=0x55c4e928ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 shutdown_connections 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 1 -- 172.21.15.86:0/4293073745 wait complete. 2024-04-25T19:13:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.508+0000 7fae745df740 20 client.26555 trim_cache size 0 max 16384 2024-04-25T19:13:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.524+0000 7fae745df740 1 Processor -- start 2024-04-25T19:13:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.524+0000 7fae745df740 1 -- start start 2024-04-25T19:13:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.524+0000 7fae745df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.524+0000 7fae745df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e9295260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae745df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9287d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae745df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1afed0 con 0x55c4e9297a00 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae745df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b1a10 con 0x55c4e928c820 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae745df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b05a0 con 0x55c4e928d140 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e9295260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9287d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e9295260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:37282/0 (socket says 172.21.15.86:37282) 2024-04-25T19:13:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6a000640 1 -- 172.21.15.86:0/66458072 learned_addr learned my addr 172.21.15.86:0/66458072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6aa00640 1 -- 172.21.15.86:0/66458072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 msgr2=0x55c4e9295260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6aa00640 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e9295260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6aa00640 1 -- 172.21.15.86:0/66458072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 msgr2=0x55c4e9287d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6aa00640 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9287d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.525+0000 7fae6aa00640 1 -- 172.21.15.86:0/66458072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e928cf70 con 0x55c4e928c820 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.526+0000 7fae6aa00640 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928f1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55c4e92aa700 tx=0x7fae5c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7471822fbe72c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.526+0000 7fae56a00640 1 -- 172.21.15.86:0/66458072 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0211d0 con 0x55c4e928c820 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.526+0000 7fae56a00640 1 -- 172.21.15.86:0/66458072 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c02b430 con 0x55c4e928c820 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.526+0000 7fae56a00640 1 -- 172.21.15.86:0/66458072 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c021e20 con 0x55c4e928c820 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 -- 172.21.15.86:0/66458072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 msgr2=0x55c4e928f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928f1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55c4e92aa700 tx=0x7fae5c017000 comp rx=0 tx=0).stop 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 -- 172.21.15.86:0/66458072 shutdown_connections 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e9297a00 0x55c4e9287d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e928d140 0x55c4e9295260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 --2- 172.21.15.86:0/66458072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928c820 0x55c4e928f1e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 -- 172.21.15.86:0/66458072 >> 172.21.15.86:0/66458072 conn(0x55c4e9288d30 msgr2=0x55c4e9287600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 -- 172.21.15.86:0/66458072 shutdown_connections 2024-04-25T19:13:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 -- 172.21.15.86:0/66458072 wait complete. 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 -- start start 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.527+0000 7fae745df740 1 Processor -- start 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 0x55c4e929cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e929d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e9297a00 0x55c4e930c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b1340 con 0x55c4e928c820 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b0c70 con 0x55c4e928d140 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1aded0 con 0x55c4e9297a00 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae745df740 10 client.0 _ll_register_callbacks cb 0x7fae58011680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 0x55c4e929cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 0x55c4e929cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38528/0 (socket says 172.21.15.86:38528) 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae6aa00640 1 -- 172.21.15.86:0/2915519504 learned_addr learned my addr 172.21.15.86:0/2915519504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae6a000640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e929d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.528+0000 7fae6b400640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e9297a00 0x55c4e930c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6aa00640 1 -- 172.21.15.86:0/2915519504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e9297a00 msgr2=0x55c4e930c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6aa00640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e9297a00 0x55c4e930c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6aa00640 1 -- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 msgr2=0x55c4e929d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6aa00640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e929d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6aa00640 1 -- 172.21.15.86:0/2915519504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e930c570 con 0x55c4e928c820 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6a000640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e929d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae6aa00640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 0x55c4e929cad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fae5c021410 tx=0x7fae5c02b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8607c608ba4ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae4be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae4be00640 1 -- 172.21.15.86:0/2915519504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004a40 con 0x55c4e928c820 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae4be00640 1 -- 172.21.15.86:0/2915519504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fae5c004be0 con 0x55c4e928c820 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.529+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5c02b430 con 0x55c4e928c820 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae4be00640 1 -- 172.21.15.86:0/2915519504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae5c043050 con 0x55c4e928c820 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 10 client.16458 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4e930d7b0 con 0x55c4e928c820 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae4be00640 1 -- 172.21.15.86:0/2915519504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c02b430 con 0x55c4e928c820 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae4be00640 1 -- 172.21.15.86:0/2915519504 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fae5c0055d0 con 0x55c4e928c820 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 20 client.16458 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e930dec0 0x55c4e93102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 4 client.16458 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.530+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4e9200320 con 0x55c4e930dec0 2024-04-25T19:13:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.531+0000 7fae6a000640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e930dec0 0x55c4e93102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.531+0000 7fae6a000640 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e930dec0 0x55c4e93102b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c028fd5487f5e7 server_cookie=a242920af7fc8bcf in_seq=0 out_seq=0 2024-04-25T19:13:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.531+0000 7fae4be00640 10 client.16458 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae4be00640 1 -- 172.21.15.86:0/2915519504 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4e9200320 con 0x55c4e930dec0 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae4be00640 10 client.16458 handle_command_reply: tid=0 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 client.16458 shutdown 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 msgr2=0x55c4e929cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 0x55c4e929cad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fae5c021410 tx=0x7fae5c02b5e0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 shutdown_connections 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c4e930dec0 0x55c4e93102b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c4e9297a00 0x55c4e930c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4e928d140 0x55c4e929d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 --2- 172.21.15.86:0/2915519504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4e928c820 0x55c4e929cad0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 >> 172.21.15.86:0/2915519504 conn(0x55c4e9288d30 msgr2=0x55c4e92961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae4be00640 0 client.16458 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 shutdown_connections 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.546+0000 7fae745df740 1 -- 172.21.15.86:0/2915519504 wait complete. 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.547+0000 7fae745df740 20 client.16458 trim_cache size 0 max 16384 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:13:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:13:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.547+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae500a3dc0 msgr2=0x7fae500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.547+0000 7fae71600640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae500a3dc0 0x7fae500a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55c4e928ddc0 tx=0x7fae5c009500 comp rx=0 tx=0).stop 2024-04-25T19:13:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.547+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 msgr2=0x7fae6c19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.547+0000 7fae71600640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c19a7f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fae64005940 tx=0x7fae64031a10 comp rx=0 tx=0).stop 2024-04-25T19:13:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 shutdown_connections 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fae500a3dc0 0x7fae500a6280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae6c10e930 0x7fae6c19a7f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae6c1016a0 0x7fae6c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 --2- 172.21.15.86:0/1148775625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae6c100d80 0x7fae6c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 >> 172.21.15.86:0/1148775625 conn(0x7fae6c0771d0 msgr2=0x7fae6c10fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 shutdown_connections 2024-04-25T19:13:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:33.548+0000 7fae71600640 1 -- 172.21.15.86:0/1148775625 wait complete. 2024-04-25T19:13:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:34 smithi194 ceph-mon[32740]: pgmap v331: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-25T19:13:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:34 smithi086 ceph-mon[27783]: pgmap v331: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-25T19:13:35.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:34 smithi183 ceph-mon[32344]: pgmap v331: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-25T19:13:36.969 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:36 smithi183 ceph-mon[32344]: pgmap v332: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T19:13:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:36 smithi194 ceph-mon[32740]: pgmap v332: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T19:13:37.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:36 smithi086 ceph-mon[27783]: pgmap v332: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T19:13:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:38 smithi194 ceph-mon[32740]: pgmap v333: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T19:13:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:38 smithi086 ceph-mon[27783]: pgmap v333: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T19:13:39.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:38 smithi183 ceph-mon[32344]: pgmap v333: 97 pgs: 97 active+clean; 2.0 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T19:13:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:40 smithi194 ceph-mon[32740]: pgmap v334: 97 pgs: 97 active+clean; 2.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-25T19:13:41.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:40 smithi086 ceph-mon[27783]: pgmap v334: 97 pgs: 97 active+clean; 2.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-25T19:13:41.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:40 smithi183 ceph-mon[32344]: pgmap v334: 97 pgs: 97 active+clean; 2.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-25T19:13:42.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:42 smithi194 ceph-mon[32740]: pgmap v335: 97 pgs: 97 active+clean; 2.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:13:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:42 smithi086 ceph-mon[27783]: pgmap v335: 97 pgs: 97 active+clean; 2.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:13:43.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:42 smithi183 ceph-mon[32344]: pgmap v335: 97 pgs: 97 active+clean; 2.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:13:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:44 smithi194 ceph-mon[32740]: pgmap v336: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:13:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:44 smithi086 ceph-mon[27783]: pgmap v336: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:13:45.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:44 smithi183 ceph-mon[32344]: pgmap v336: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:13:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 -- 172.21.15.86:0/4055794280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c104d60 msgr2=0x7fb81c108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 --2- 172.21.15.86:0/4055794280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c104d60 0x7fb81c108da0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb804009900 tx=0x7fb80402a3f0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 -- 172.21.15.86:0/4055794280 shutdown_connections 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 --2- 172.21.15.86:0/4055794280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb81c109480 0x7fb81c111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 --2- 172.21.15.86:0/4055794280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c104d60 0x7fb81c108da0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 --2- 172.21.15.86:0/4055794280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 0x7fb81c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 -- 172.21.15.86:0/4055794280 >> 172.21.15.86:0/4055794280 conn(0x7fb81c0ffc50 msgr2=0x7fb81c102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 -- 172.21.15.86:0/4055794280 shutdown_connections 2024-04-25T19:13:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.693+0000 7fb821000640 1 -- 172.21.15.86:0/4055794280 wait complete. 2024-04-25T19:13:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 Processor -- start 2024-04-25T19:13:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 -- start start 2024-04-25T19:13:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 0x7fb81c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c109480 0x7fb81c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb81c19a700 0x7fb81c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1143a0 con 0x7fb81c1043b0 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c114220 con 0x7fb81c19a700 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.694+0000 7fb821000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c114520 con 0x7fb81c109480 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb811400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c109480 0x7fb81c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb811400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c109480 0x7fb81c1a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46478/0 (socket says 172.21.15.86:46478) 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb811400640 1 -- 172.21.15.86:0/1302422330 learned_addr learned my addr 172.21.15.86:0/1302422330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 0x7fb81c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb81a800640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb81c19a700 0x7fb81c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 -- 172.21.15.86:0/1302422330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c109480 msgr2=0x7fb81c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c109480 0x7fb81c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 -- 172.21.15.86:0/1302422330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb81c19a700 msgr2=0x7fb81c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb81c19a700 0x7fb81c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 -- 172.21.15.86:0/1302422330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c19b300 con 0x7fb81c1043b0 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.695+0000 7fb819e00640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 0x7fb81c1a0640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e9f0 tx=0x7fb80c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.696+0000 7fb813400640 1 -- 172.21.15.86:0/1302422330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00ccc0 con 0x7fb81c1043b0 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.696+0000 7fb813400640 1 -- 172.21.15.86:0/1302422330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80c0108e0 con 0x7fb81c1043b0 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.696+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c1a73e0 con 0x7fb81c1043b0 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.696+0000 7fb813400640 1 -- 172.21.15.86:0/1302422330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c018880 con 0x7fb81c1043b0 2024-04-25T19:13:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.696+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c1a77e0 con 0x7fb81c1043b0 2024-04-25T19:13:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.699+0000 7fb823fc8740 1 Processor -- start 2024-04-25T19:13:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.699+0000 7fb813400640 1 -- 172.21.15.86:0/1302422330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb80c010430 con 0x7fb81c1043b0 2024-04-25T19:13:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.699+0000 7fb823fc8740 1 -- start start 2024-04-25T19:13:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51b2820 0x55c9a51b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51bda00 0x55c9a51bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb823fc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5118900 con 0x55c9a51b2820 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb823fc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9a50beef0 con 0x55c9a51b3140 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb823fc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c9a51148c0 con 0x55c9a51bda00 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb81a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51b2820 0x55c9a51b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb819e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb811400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51bda00 0x55c9a51bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb81a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51b2820 0x55c9a51b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46868/0 (socket says 172.21.15.86:46868) 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb81a800640 1 -- 172.21.15.86:0/3797954247 learned_addr learned my addr 172.21.15.86:0/3797954247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb811400640 1 -- 172.21.15.86:0/3797954247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 msgr2=0x55c9a51bd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb811400640 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb811400640 1 -- 172.21.15.86:0/3797954247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51b2820 msgr2=0x55c9a51b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb811400640 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51b2820 0x55c9a51b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.700+0000 7fb811400640 1 -- 172.21.15.86:0/3797954247 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a4e2b210 con 0x55c9a51bda00 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb813400640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb8000a3dc0 0x7fb8000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb811400640 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51bda00 0x55c9a51bfdf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb804004770 tx=0x7fb804031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe66d1c02733d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb813400640 1 -- 172.21.15.86:0/1302422330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fb80c0d0bd0 con 0x7fb81c1043b0 2024-04-25T19:13:46.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb81a800640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb8000a3dc0 0x7fb8000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb80b400640 1 -- 172.21.15.86:0/3797954247 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804040070 con 0x55c9a51bda00 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb80b400640 1 -- 172.21.15.86:0/3797954247 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804004030 con 0x55c9a51bda00 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.701+0000 7fb80b400640 1 -- 172.21.15.86:0/3797954247 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804035040 con 0x55c9a51bda00 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb81a800640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb8000a3dc0 0x7fb8000a6280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb814011630 tx=0x7fb814009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 -- 172.21.15.86:0/3797954247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51bda00 msgr2=0x55c9a51bfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51bda00 0x55c9a51bfdf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb804004770 tx=0x7fb804031b10 comp rx=0 tx=0).stop 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 -- 172.21.15.86:0/3797954247 shutdown_connections 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51bda00 0x55c9a51bfdf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3797954247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51b2820 0x55c9a51b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 -- 172.21.15.86:0/3797954247 >> 172.21.15.86:0/3797954247 conn(0x55c9a51aed30 msgr2=0x55c9a51af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 -- 172.21.15.86:0/3797954247 shutdown_connections 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.702+0000 7fb823fc8740 1 -- 172.21.15.86:0/3797954247 wait complete. 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.703+0000 7fb823fc8740 1 -- start start 2024-04-25T19:13:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.703+0000 7fb823fc8740 1 Processor -- start 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.703+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.703+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb823fc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9a51c05d0 con 0x55c9a51bda00 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb823fc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9a50d2310 con 0x55c9a51b3140 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb823fc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c9a51c0750 con 0x55c9a51b2820 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb823fc8740 10 client.0 _ll_register_callbacks cb 0x55c9a50a0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51ba930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46498/0 (socket says 172.21.15.86:46498) 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 -- 172.21.15.86:0/852002666 learned_addr learned my addr 172.21.15.86:0/852002666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb81a800640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb811400640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 -- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 msgr2=0x55c9a51bae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 -- 172.21.15.86:0/852002666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 msgr2=0x55c9a51bb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51bb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 -- 172.21.15.86:0/852002666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a4ef7620 con 0x55c9a51b2820 2024-04-25T19:13:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb819e00640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51ba930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb80c09cae0 tx=0x7fb80c00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876752b983352c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.704+0000 7fb7ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:13:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb7ffe00640 1 -- 172.21.15.86:0/852002666 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c010720 con 0x55c9a51b2820 2024-04-25T19:13:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb804004030 con 0x55c9a51b2820 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb7ffe00640 1 -- 172.21.15.86:0/852002666 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fb80c010a90 con 0x55c9a51b2820 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb7ffe00640 1 -- 172.21.15.86:0/852002666 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb80c09cb50 con 0x55c9a51b2820 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb823fc8740 10 client.26566 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9a51cc860 con 0x55c9a51b2820 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb7ffe00640 1 -- 172.21.15.86:0/852002666 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804004030 con 0x55c9a51b2820 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb7ffe00640 1 -- 172.21.15.86:0/852002666 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb80c0988a0 con 0x55c9a51b2820 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb823fc8740 10 client.26566 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb823fc8740 10 client.26566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.705+0000 7fb823fc8740 20 client.26566 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.706+0000 7fb823fc8740 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a51cdd20 0x55c9a51d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.706+0000 7fb823fc8740 4 client.26566 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.706+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9a512c8b0 con 0x55c9a51cdd20 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.706+0000 7fb811400640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a51cdd20 0x55c9a51d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.706+0000 7fb7ffe00640 10 client.26566 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.706+0000 7fb811400640 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a51cdd20 0x55c9a51d0110 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4531303b2be76fa0 server_cookie=c388484c42672306 in_seq=0 out_seq=0 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.707+0000 7fb7ffe00640 1 -- 172.21.15.86:0/852002666 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9a512c8b0 con 0x55c9a51cdd20 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.707+0000 7fb7ffe00640 10 client.26566 handle_command_reply: tid=0 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.707+0000 7fb823fc8740 1 client.26566 shutdown 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 msgr2=0x55c9a51ba930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51ba930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb80c09cae0 tx=0x7fb80c00ce60 comp rx=0 tx=0).stop 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 shutdown_connections 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a51cdd20 0x55c9a51d0110 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 --2- 172.21.15.86:0/852002666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51ba930 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 >> 172.21.15.86:0/852002666 conn(0x55c9a51aed30 msgr2=0x55c9a51b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb7ffe00640 0 client.26566 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 shutdown_connections 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 1 -- 172.21.15.86:0/852002666 wait complete. 2024-04-25T19:13:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.708+0000 7fb823fc8740 20 client.26566 trim_cache size 0 max 16384 2024-04-25T19:13:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 Processor -- start 2024-04-25T19:13:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 -- start start 2024-04-25T19:13:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51b5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1b0000 con 0x55c9a51bda00 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1b1b40 con 0x55c9a51b3140 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.724+0000 7fb823fc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1b06d0 con 0x55c9a51b2820 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb81a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46508/0 (socket says 172.21.15.86:46508) 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb811400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 -- 172.21.15.86:0/281509549 learned_addr learned my addr 172.21.15.86:0/281509549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 -- 172.21.15.86:0/281509549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 msgr2=0x55c9a51b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 -- 172.21.15.86:0/281509549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 msgr2=0x55c9a51d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.725+0000 7fb819e00640 1 -- 172.21.15.86:0/281509549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a51d0740 con 0x55c9a51b2820 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.726+0000 7fb819e00640 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c9490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb80c000c00 tx=0x7fb80c0d4710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff417001453efce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.726+0000 7fb80aa00640 1 -- 172.21.15.86:0/281509549 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c010720 con 0x55c9a51b2820 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.726+0000 7fb80aa00640 1 -- 172.21.15.86:0/281509549 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80c0a5710 con 0x55c9a51b2820 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.726+0000 7fb80aa00640 1 -- 172.21.15.86:0/281509549 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0ce4d0 con 0x55c9a51b2820 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 -- 172.21.15.86:0/281509549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 msgr2=0x55c9a51c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c9490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb80c000c00 tx=0x7fb80c0d4710 comp rx=0 tx=0).stop 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 -- 172.21.15.86:0/281509549 shutdown_connections 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 --2- 172.21.15.86:0/281509549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c9490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 -- 172.21.15.86:0/281509549 >> 172.21.15.86:0/281509549 conn(0x55c9a51aed30 msgr2=0x55c9a51b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 -- 172.21.15.86:0/281509549 shutdown_connections 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 -- 172.21.15.86:0/281509549 wait complete. 2024-04-25T19:13:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 -- start start 2024-04-25T19:13:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.727+0000 7fb823fc8740 1 Processor -- start 2024-04-25T19:13:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1b1470 con 0x55c9a51bda00 2024-04-25T19:13:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1b0da0 con 0x55c9a51b3140 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1ae000 con 0x55c9a51b2820 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb823fc8740 10 client.0 _ll_register_callbacks cb 0x7fb80c09c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb819e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb819e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:46520/0 (socket says 172.21.15.86:46520) 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb819e00640 1 -- 172.21.15.86:0/3543894146 learned_addr learned my addr 172.21.15.86:0/3543894146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb81a800640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.728+0000 7fb811400640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb819e00640 1 -- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 msgr2=0x55c9a51c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb819e00640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb819e00640 1 -- 172.21.15.86:0/3543894146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 msgr2=0x55c9a51c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb819e00640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb819e00640 1 -- 172.21.15.86:0/3543894146 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a51c39f0 con 0x55c9a51b2820 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb811400640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb819e00640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c5b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb80c09ae00 tx=0x7fb80c098cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d691c3529d29c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb7ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb7ffe00640 1 -- 172.21.15.86:0/3543894146 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0ce6d0 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb80c0a5710 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb7ffe00640 1 -- 172.21.15.86:0/3543894146 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fb80c09bfa0 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb7ffe00640 1 -- 172.21.15.86:0/3543894146 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb80c010720 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb823fc8740 10 client.26572 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.729+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9a5233ff0 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb7ffe00640 1 -- 172.21.15.86:0/3543894146 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80c0a5710 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb7ffe00640 1 -- 172.21.15.86:0/3543894146 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb80c087130 con 0x55c9a51b2820 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb823fc8740 10 client.26572 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb823fc8740 10 client.26572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb823fc8740 20 client.26572 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a5234390 0x55c9a52367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb823fc8740 4 client.26572 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9a5126320 con 0x55c9a5234390 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.730+0000 7fb811400640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a5234390 0x55c9a52367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.731+0000 7fb811400640 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a5234390 0x55c9a52367b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69019eb4aeaff1a1 server_cookie=74c183c5465fe799 in_seq=0 out_seq=0 2024-04-25T19:13:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.731+0000 7fb7ffe00640 10 client.26572 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.746+0000 7fb7ffe00640 1 -- 172.21.15.86:0/3543894146 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9a5126320 con 0x55c9a5234390 2024-04-25T19:13:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.746+0000 7fb7ffe00640 10 client.26572 handle_command_reply: tid=0 2024-04-25T19:13:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.746+0000 7fb823fc8740 1 client.26572 shutdown 2024-04-25T19:13:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.746+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 msgr2=0x55c9a51c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c5b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb80c09ae00 tx=0x7fb80c098cb0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 shutdown_connections 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c9a5234390 0x55c9a52367b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9a51bda00 0x55c9a51c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb7ffe00640 0 client.26572 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9a51b3140 0x55c9a51c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 --2- 172.21.15.86:0/3543894146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c9a51b2820 0x55c9a51c5b10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 >> 172.21.15.86:0/3543894146 conn(0x55c9a51aed30 msgr2=0x55c9a51ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 shutdown_connections 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 1 -- 172.21.15.86:0/3543894146 wait complete. 2024-04-25T19:13:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.747+0000 7fb823fc8740 20 client.26572 trim_cache size 0 max 16384 2024-04-25T19:13:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:13:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:13:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:13:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb8000a3dc0 msgr2=0x7fb8000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb8000a3dc0 0x7fb8000a6280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb814011630 tx=0x7fb814009500 comp rx=0 tx=0).stop 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 msgr2=0x7fb81c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 0x7fb81c1a0640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e9f0 tx=0x7fb80c00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 shutdown_connections 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb8000a3dc0 0x7fb8000a6280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb81c19a700 0x7fb81c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb81c109480 0x7fb81c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 --2- 172.21.15.86:0/1302422330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c1043b0 0x7fb81c1a0640 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 >> 172.21.15.86:0/1302422330 conn(0x7fb81c0ffc50 msgr2=0x7fb81c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 shutdown_connections 2024-04-25T19:13:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:46.748+0000 7fb821000640 1 -- 172.21.15.86:0/1302422330 wait complete. 2024-04-25T19:13:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:46 smithi194 ceph-mon[32740]: pgmap v337: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:13:47.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:46 smithi086 ceph-mon[27783]: pgmap v337: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:13:47.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:46 smithi183 ceph-mon[32344]: pgmap v337: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:13:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:48 smithi194 ceph-mon[32740]: pgmap v338: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:13:49.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:48 smithi086 ceph-mon[27783]: pgmap v338: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:13:49.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:48 smithi183 ceph-mon[32344]: pgmap v338: 97 pgs: 97 active+clean; 2.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:13:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:50 smithi086 ceph-mon[27783]: pgmap v339: 97 pgs: 97 active+clean; 2.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-04-25T19:13:51.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:50 smithi183 ceph-mon[32344]: pgmap v339: 97 pgs: 97 active+clean; 2.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-04-25T19:13:51.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:50 smithi194 ceph-mon[32740]: pgmap v339: 97 pgs: 97 active+clean; 2.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-04-25T19:13:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:52 smithi086 ceph-mon[27783]: pgmap v340: 97 pgs: 97 active+clean; 2.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:53.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:52 smithi183 ceph-mon[32344]: pgmap v340: 97 pgs: 97 active+clean; 2.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:53.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:52 smithi194 ceph-mon[32740]: pgmap v340: 97 pgs: 97 active+clean; 2.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:55.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:54 smithi086 ceph-mon[27783]: pgmap v341: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:13:55.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:54 smithi183 ceph-mon[32344]: pgmap v341: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:13:55.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:54 smithi194 ceph-mon[32740]: pgmap v341: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:13:56.976 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.2 2024-04-25T19:13:56.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:13:57.050 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:56 smithi086 ceph-mon[27783]: pgmap v342: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:57.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:56 smithi183 ceph-mon[32344]: pgmap v342: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:57.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:56 smithi194 ceph-mon[32740]: pgmap v342: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:57.427 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:13:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:13:58 smithi086 ceph-mon[27783]: pgmap v343: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:13:58 smithi183 ceph-mon[32344]: pgmap v343: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:59.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:13:58 smithi194 ceph-mon[32740]: pgmap v343: 97 pgs: 97 active+clean; 2.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.680+0000 7fb23ac00700 1 -- 172.21.15.86:0/4157690723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2341092d0 msgr2=0x7fb2341096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.680+0000 7fb23ac00700 1 --2- 172.21.15.86:0/4157690723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2341092d0 0x7fb2341096b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb2240099c0 tx=0x7fb224009cd0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 -- 172.21.15.86:0/4157690723 shutdown_connections 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 --2- 172.21.15.86:0/4157690723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23410e3b0 0x7fb234115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 --2- 172.21.15.86:0/4157690723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2341092d0 0x7fb2341096b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 --2- 172.21.15.86:0/4157690723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb234109c80 0x7fb23410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 -- 172.21.15.86:0/4157690723 >> 172.21.15.86:0/4157690723 conn(0x7fb234076210 msgr2=0x7fb234107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 -- 172.21.15.86:0/4157690723 shutdown_connections 2024-04-25T19:13:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.681+0000 7fb23ac00700 1 -- 172.21.15.86:0/4157690723 wait complete. 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 Processor -- start 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 -- start start 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2341092d0 0x7fb2341a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 0x7fb2341a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23410e3b0 0x7fb23419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2341192b0 con 0x7fb234109c80 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb234119130 con 0x7fb2341092d0 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.682+0000 7fb23ac00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb234119430 con 0x7fb23410e3b0 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb22be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 0x7fb2341a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb233e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2341092d0 0x7fb2341a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23410e3b0 0x7fb23419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb22be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 0x7fb2341a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35884/0 (socket says 172.21.15.86:35884) 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb22be00700 1 -- 172.21.15.86:0/536936404 learned_addr learned my addr 172.21.15.86:0/536936404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 -- 172.21.15.86:0/536936404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2341092d0 msgr2=0x7fb2341a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2341092d0 0x7fb2341a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 -- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 msgr2=0x7fb2341a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 0x7fb2341a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 -- 172.21.15.86:0/536936404 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23419fd90 con 0x7fb23410e3b0 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb22be00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 0x7fb2341a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb233e00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2341092d0 0x7fb2341a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.683+0000 7fb238c00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23410e3b0 0x7fb23419f630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb22c007f10 tx=0x7fb22c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.684+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c004db0 con 0x7fb23410e3b0 2024-04-25T19:13:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.684+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2341a0080 con 0x7fb23410e3b0 2024-04-25T19:13:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.684+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c003a00 con 0x7fb23410e3b0 2024-04-25T19:13:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.684+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c004580 con 0x7fb23410e3b0 2024-04-25T19:13:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.684+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2341ac2f0 con 0x7fb23410e3b0 2024-04-25T19:13:59.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.685+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb234069730 con 0x7fb23410e3b0 2024-04-25T19:13:59.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.687+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb22c01c020 con 0x7fb23410e3b0 2024-04-25T19:13:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.688+0000 7fb231600700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2180a3db0 0x7fb2180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.689+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7fb22c0d0230 con 0x7fb23410e3b0 2024-04-25T19:13:59.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.689+0000 7fb233e00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2180a3db0 0x7fb2180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.689+0000 7fb233e00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2180a3db0 0x7fb2180a6270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb22400b580 tx=0x7fb2240058e0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.691+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb22c010070 con 0x7fb23410e3b0 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 -- 172.21.15.86:0/1411720808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8101310 msgr2=0x7f07b81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1411720808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8101310 0x7f07b81016f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009900 tx=0x7f07ac02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 -- 172.21.15.86:0/1411720808 shutdown_connections 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1411720808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b8106190 0x7f07b810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1411720808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101cc0 0x7f07b8105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1411720808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8101310 0x7f07b81016f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 -- 172.21.15.86:0/1411720808 >> 172.21.15.86:0/1411720808 conn(0x7f07b8077cc0 msgr2=0x7f07b80780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 -- 172.21.15.86:0/1411720808 shutdown_connections 2024-04-25T19:13:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.884+0000 7f07bfe00640 1 -- 172.21.15.86:0/1411720808 wait complete. 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 Processor -- start 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 -- start start 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101310 0x7f07b8102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 0x7f07b8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b81035d0 0x7f07b81a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111140 con 0x7f07b8106190 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8110fc0 con 0x7f07b81035d0 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.885+0000 7f07bfe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81112c0 con 0x7f07b8101310 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 0x7f07b8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101310 0x7f07b8102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b81035d0 0x7f07b81a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 0x7f07b8103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49150/0 (socket says 172.21.15.86:49150) 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101310 0x7f07b8102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58720/0 (socket says 172.21.15.86:58720) 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bca00640 1 -- 172.21.15.86:0/3102342298 learned_addr learned my addr 172.21.15.86:0/3102342298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bde00640 1 -- 172.21.15.86:0/3102342298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101310 msgr2=0x7f07b8102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bde00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101310 0x7f07b8102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bde00640 1 -- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 msgr2=0x7f07b8103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bde00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 0x7f07b8103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bde00640 1 -- 172.21.15.86:0/3102342298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b81a7020 con 0x7f07b81035d0 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.886+0000 7f07bca00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 0x7f07b8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.887+0000 7f07bde00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b81035d0 0x7f07b81a6920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f07b0005fd0 tx=0x7f07b000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.887+0000 7f07b6000640 1 -- 172.21.15.86:0/3102342298 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b000cd80 con 0x7f07b81035d0 2024-04-25T19:13:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.887+0000 7f07b6000640 1 -- 172.21.15.86:0/3102342298 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b0005020 con 0x7f07b81035d0 2024-04-25T19:13:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.887+0000 7f07b6000640 1 -- 172.21.15.86:0/3102342298 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0010690 con 0x7f07b81035d0 2024-04-25T19:13:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.887+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b81a7310 con 0x7f07b81035d0 2024-04-25T19:13:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.887+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a7810 con 0x7f07b81035d0 2024-04-25T19:13:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.890+0000 7f07b6000640 1 -- 172.21.15.86:0/3102342298 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f07b00054d0 con 0x7f07b81035d0 2024-04-25T19:13:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07b6000640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07a00a3e70 0x7f07a00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 Processor -- start 2024-04-25T19:13:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07b6000640 1 -- 172.21.15.86:0/3102342298 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f07b00d0dc0 con 0x7f07b81035d0 2024-04-25T19:13:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 -- start start 2024-04-25T19:13:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07bd400640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07a00a3e70 0x7f07a00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3db820 0x559b9d3dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3e6a00 0x559b9d3e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b9d341900 con 0x559b9d3db820 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b9d2e7ef0 con 0x559b9d3dc140 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.892+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559b9d33d8c0 con 0x559b9d3e6a00 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3e64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55078/0 (socket says 172.21.15.86:55078) 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3e6a00 0x559b9d3e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bca00640 1 -- 172.21.15.86:0/3447484102 learned_addr learned my addr 172.21.15.86:0/3447484102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bd400640 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3db820 0x559b9d3dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 -- 172.21.15.86:0/3447484102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 msgr2=0x559b9d3e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 -- 172.21.15.86:0/3447484102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3db820 msgr2=0x559b9d3dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3db820 0x559b9d3dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 -- 172.21.15.86:0/3447484102 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b9d054210 con 0x559b9d3e6a00 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bd400640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07a00a3e70 0x7f07a00a6330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005ab0 tx=0x7f07ac000bc0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.893+0000 7f07bde00640 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3e6a00 0x559b9d3e8df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07b00ce440 tx=0x7f07b009cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d9a06de20469c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.894+0000 7f07aa600640 1 -- 172.21.15.86:0/3447484102 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b009af90 con 0x559b9d3e6a00 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.894+0000 7f07aa600640 1 -- 172.21.15.86:0/3447484102 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b00bbe50 con 0x559b9d3e6a00 2024-04-25T19:13:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.894+0000 7f07aa600640 1 -- 172.21.15.86:0/3447484102 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a5470 con 0x559b9d3e6a00 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3447484102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3e6a00 msgr2=0x559b9d3e8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3e6a00 0x559b9d3e8df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07b00ce440 tx=0x7f07b009cc90 comp rx=0 tx=0).stop 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3447484102 shutdown_connections 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3e6a00 0x559b9d3e8df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3447484102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3db820 0x559b9d3dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3447484102 >> 172.21.15.86:0/3447484102 conn(0x559b9d3d7d30 msgr2=0x559b9d3d8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3447484102 shutdown_connections 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.895+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3447484102 wait complete. 2024-04-25T19:13:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.896+0000 7f07c2dcc740 1 -- start start 2024-04-25T19:13:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.896+0000 7f07c2dcc740 1 Processor -- start 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3e3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3dc140 0x559b9d3e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3e6a00 0x559b9d3e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b9d3e9630 con 0x559b9d3dc140 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b9d2fb310 con 0x559b9d3e6a00 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559b9d3e97b0 con 0x559b9d3db820 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07c2dcc740 10 client.0 _ll_register_callbacks cb 0x559b9d2c9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3dc140 0x559b9d3e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3e3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58744/0 (socket says 172.21.15.86:58744) 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 -- 172.21.15.86:0/3687746198 learned_addr learned my addr 172.21.15.86:0/3687746198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bde00640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3e6a00 0x559b9d3e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 -- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3e6a00 msgr2=0x559b9d3e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3e6a00 0x559b9d3e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 -- 172.21.15.86:0/3687746198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3dc140 msgr2=0x559b9d3e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3dc140 0x559b9d3e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bd400640 1 -- 172.21.15.86:0/3687746198 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b9d120620 con 0x559b9d3db820 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.897+0000 7f07bca00640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3dc140 0x559b9d3e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f07bd400640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3e3980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07ac033000 tx=0x7f07ac02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4a85aa1df0ddf54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f079ea00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f079ea00640 1 -- 172.21.15.86:0/3687746198 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac035040 con 0x559b9d3db820 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b00bbe50 con 0x559b9d3db820 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f079ea00640 1 -- 172.21.15.86:0/3687746198 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f07ac004d00 con 0x559b9d3db820 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f079ea00640 1 -- 172.21.15.86:0/3687746198 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07ac0035f0 con 0x559b9d3db820 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f07c2dcc740 10 client.26590 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.898+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b9d3f58c0 con 0x559b9d3db820 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f079ea00640 1 -- 172.21.15.86:0/3687746198 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b00bbe50 con 0x559b9d3db820 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f079ea00640 1 -- 172.21.15.86:0/3687746198 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07ac0327e0 con 0x559b9d3db820 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f07c2dcc740 10 client.26590 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f07c2dcc740 10 client.26590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f07c2dcc740 20 client.26590 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d3f6d30 0x559b9d3f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f07c2dcc740 4 client.26590 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:13:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.899+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b9d3558b0 con 0x559b9d3f6d30 2024-04-25T19:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.900+0000 7f07bca00640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d3f6d30 0x559b9d3f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.900+0000 7f07bca00640 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d3f6d30 0x559b9d3f91a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4795b3ba4dc53c server_cookie=f6f50b02c77b8752 in_seq=0 out_seq=0 2024-04-25T19:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.900+0000 7f079ea00640 10 client.26590 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f079ea00640 1 -- 172.21.15.86:0/3687746198 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b9d3558b0 con 0x559b9d3f6d30 2024-04-25T19:13:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f079ea00640 10 client.26590 handle_command_reply: tid=0 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 client.26590 shutdown 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 msgr2=0x559b9d3e3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3e3980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07ac033000 tx=0x7f07ac02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 shutdown_connections 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d3f6d30 0x559b9d3f91a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3e6a00 0x559b9d3e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3dc140 0x559b9d3e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f079ea00640 0 client.26590 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/3687746198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3e3980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 >> 172.21.15.86:0/3687746198 conn(0x559b9d3d7d30 msgr2=0x559b9d3ddab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 shutdown_connections 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.901+0000 7f07c2dcc740 1 -- 172.21.15.86:0/3687746198 wait complete. 2024-04-25T19:13:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.902+0000 7f07c2dcc740 20 client.26590 trim_cache size 0 max 16384 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.917+0000 7f07c2dcc740 1 Processor -- start 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.917+0000 7f07c2dcc740 1 -- start start 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3db820 0x559b9d3f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 0x559b9d3feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d3feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81aff80 con 0x559b9d3e6a00 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b1ac0 con 0x559b9d3db820 2024-04-25T19:13:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b0650 con 0x559b9d3dc140 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 0x559b9d3feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d3feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 0x559b9d3feab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58760/0 (socket says 172.21.15.86:58760) 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 -- 172.21.15.86:0/50709921 learned_addr learned my addr 172.21.15.86:0/50709921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bd400640 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3db820 0x559b9d3f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 -- 172.21.15.86:0/50709921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3db820 msgr2=0x559b9d3f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3db820 0x559b9d3f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 -- 172.21.15.86:0/50709921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 msgr2=0x559b9d3feff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d3feff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.918+0000 7f07bca00640 1 -- 172.21.15.86:0/50709921 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b9d3f9820 con 0x559b9d3dc140 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.919+0000 7f07bca00640 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 0x559b9d3feab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07a40058c0 tx=0x7f07a4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6461944573161bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.919+0000 7f07a9c00640 1 -- 172.21.15.86:0/50709921 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400fca0 con 0x559b9d3dc140 2024-04-25T19:13:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.919+0000 7f07a9c00640 1 -- 172.21.15.86:0/50709921 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a400fe40 con 0x559b9d3dc140 2024-04-25T19:13:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.919+0000 7f07a9c00640 1 -- 172.21.15.86:0/50709921 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4022170 con 0x559b9d3dc140 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 -- 172.21.15.86:0/50709921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 msgr2=0x559b9d3feab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 0x559b9d3feab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07a40058c0 tx=0x7f07a4005f90 comp rx=0 tx=0).stop 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 -- 172.21.15.86:0/50709921 shutdown_connections 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d3feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3dc140 0x559b9d3feab0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/50709921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3db820 0x559b9d3f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 -- 172.21.15.86:0/50709921 >> 172.21.15.86:0/50709921 conn(0x559b9d3d7d30 msgr2=0x559b9d3d8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 -- 172.21.15.86:0/50709921 shutdown_connections 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 -- 172.21.15.86:0/50709921 wait complete. 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 -- start start 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.920+0000 7f07c2dcc740 1 Processor -- start 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d4542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b13f0 con 0x559b9d3e6a00 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b0d20 con 0x559b9d3dc140 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81adf80 con 0x559b9d3db820 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07c2dcc740 10 client.0 _ll_register_callbacks cb 0x7f07ac004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07bde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d4542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07bca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07bca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55106/0 (socket says 172.21.15.86:55106) 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07bca00640 1 -- 172.21.15.86:0/1558330560 learned_addr learned my addr 172.21.15.86:0/1558330560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07bde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d4542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49186/0 (socket says 172.21.15.86:49186) 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.921+0000 7f07bd400640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bca00640 1 -- 172.21.15.86:0/1558330560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 msgr2=0x559b9d3ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bca00640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bca00640 1 -- 172.21.15.86:0/1558330560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 msgr2=0x559b9d4542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bca00640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d4542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bca00640 1 -- 172.21.15.86:0/1558330560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b9d454920 con 0x559b9d3dc140 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bde00640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d4542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07bca00640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3ec4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07a4010010 tx=0x7f07a4009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42be0ce88bc73c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f079ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f079ea00640 1 -- 172.21.15.86:0/1558330560 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4022170 con 0x559b9d3dc140 2024-04-25T19:13:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a400fe40 con 0x559b9d3dc140 2024-04-25T19:13:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f079ea00640 1 -- 172.21.15.86:0/1558330560 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f07a4022310 con 0x559b9d3dc140 2024-04-25T19:13:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.922+0000 7f079ea00640 1 -- 172.21.15.86:0/1558330560 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07a4022510 con 0x559b9d3dc140 2024-04-25T19:13:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 10 client.26603 fetch_fsmap learned FSMap version 29 2024-04-25T19:13:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b9d45ce80 con 0x559b9d3dc140 2024-04-25T19:13:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f079ea00640 1 -- 172.21.15.86:0/1558330560 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a400fe40 con 0x559b9d3dc140 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f079ea00640 1 -- 172.21.15.86:0/1558330560 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07a4010ae0 con 0x559b9d3dc140 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 10 client.26603 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 20 client.26603 populate_metadata read hostname 'smithi086' 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d45d520 0x559b9d45f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 4 client.26603 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b9d34f320 con 0x559b9d45d520 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.923+0000 7f07bd400640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d45d520 0x559b9d45f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.924+0000 7f07bd400640 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d45d520 0x559b9d45f940 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f54b98d497369f8 server_cookie=86cdb849bcc73790 in_seq=0 out_seq=0 2024-04-25T19:13:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.924+0000 7f079ea00640 10 client.26603 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.938+0000 7f079ea00640 1 -- 172.21.15.86:0/1558330560 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b9d34f320 con 0x559b9d45d520 2024-04-25T19:13:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.938+0000 7f079ea00640 10 client.26603 handle_command_reply: tid=0 2024-04-25T19:13:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.938+0000 7f07c2dcc740 1 client.26603 shutdown 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.938+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 msgr2=0x559b9d3ec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.938+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3ec4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07a4010010 tx=0x7f07a4009dd0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 shutdown_connections 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559b9d45d520 0x559b9d45f940 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b9d3e6a00 0x559b9d4542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f079ea00640 0 client.26603 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9d3dc140 0x559b9d3ec4d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 --2- 172.21.15.86:0/1558330560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559b9d3db820 0x559b9d3ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 >> 172.21.15.86:0/1558330560 conn(0x559b9d3d7d30 msgr2=0x559b9d3d6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 shutdown_connections 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 1 -- 172.21.15.86:0/1558330560 wait complete. 2024-04-25T19:13:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.939+0000 7f07c2dcc740 20 client.26603 trim_cache size 0 max 16384 2024-04-25T19:13:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:13:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:13:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:13:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:13:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07a00a3e70 msgr2=0x7f07a00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07a00a3e70 0x7f07a00a6330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005ab0 tx=0x7f07ac000bc0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b81035d0 msgr2=0x7f07b81a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b81035d0 0x7f07b81a6920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f07b0005fd0 tx=0x7f07b000ee60 comp rx=0 tx=0).stop 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 shutdown_connections 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07a00a3e70 0x7f07a00a6330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b81035d0 0x7f07b81a6920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8106190 0x7f07b8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 --2- 172.21.15.86:0/3102342298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b8101310 0x7f07b8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 >> 172.21.15.86:0/3102342298 conn(0x7f07b8077cc0 msgr2=0x7f07b8104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 shutdown_connections 2024-04-25T19:13:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:13:59.940+0000 7f07bfe00640 1 -- 172.21.15.86:0/3102342298 wait complete. 2024-04-25T19:13:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.982+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb23402df80 con 0x7fb23410e3b0 2024-04-25T19:13:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.983+0000 7fb231600700 1 -- 172.21.15.86:0/536936404 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fb22c09c800 con 0x7fb23410e3b0 2024-04-25T19:13:59.985 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:13:59.985 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:13:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.986+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2180a3db0 msgr2=0x7fb2180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.986+0000 7fb23ac00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2180a3db0 0x7fb2180a6270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb22400b580 tx=0x7fb2240058e0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.986+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23410e3b0 msgr2=0x7fb23419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.986+0000 7fb23ac00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23410e3b0 0x7fb23419f630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb22c007f10 tx=0x7fb22c00d3b0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 shutdown_connections 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb2180a3db0 0x7fb2180a6270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23410e3b0 0x7fb23419f630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2341092d0 0x7fb2341a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 --2- 172.21.15.86:0/536936404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb234109c80 0x7fb2341a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 >> 172.21.15.86:0/536936404 conn(0x7fb234076210 msgr2=0x7fb234076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 shutdown_connections 2024-04-25T19:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:13:59.987+0000 7fb23ac00700 1 -- 172.21.15.86:0/536936404 wait complete. 2024-04-25T19:13:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:14:00.921 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 89.2 --await-for 42.0' 2024-04-25T19:14:00.921 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 89.2 --await-for 42.0 2024-04-25T19:14:01.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:00 smithi086 ceph-mon[27783]: pgmap v344: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:14:01.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:00 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/536936404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:14:01.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:00 smithi183 ceph-mon[32344]: pgmap v344: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:14:01.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:00 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/536936404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:14:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:00 smithi194 ceph-mon[32740]: pgmap v344: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:14:01.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:00 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/536936404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:14:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:14:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:02 smithi086 ceph-mon[27783]: pgmap v345: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:14:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:02 smithi183 ceph-mon[32344]: pgmap v345: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:14:03.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:02 smithi194 ceph-mon[32740]: pgmap v345: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 -- 172.21.15.86:0/1057905990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 msgr2=0x7f5060111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 --2- 172.21.15.86:0/1057905990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f5060111b70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f505800b0d0 tx=0x7f505800b3e0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 -- 172.21.15.86:0/1057905990 shutdown_connections 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 --2- 172.21.15.86:0/1057905990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f5060111b70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 --2- 172.21.15.86:0/1057905990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5060104ec0 0x7f50601052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 --2- 172.21.15.86:0/1057905990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060105870 0x7f50601098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.587+0000 7f5067c00700 1 -- 172.21.15.86:0/1057905990 >> 172.21.15.86:0/1057905990 conn(0x7f5060100750 msgr2=0x7f5060102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.588+0000 7f5067c00700 1 -- 172.21.15.86:0/1057905990 shutdown_connections 2024-04-25T19:14:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.588+0000 7f5067c00700 1 -- 172.21.15.86:0/1057905990 wait complete. 2024-04-25T19:14:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 Processor -- start 2024-04-25T19:14:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 -- start start 2024-04-25T19:14:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 0x7f506019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5060105870 0x7f506019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f506019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5060114e90 con 0x7f5060109fa0 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5060114d10 con 0x7f5060104ec0 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.589+0000 7f5067c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5060115010 con 0x7f5060105870 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f506019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 0x7f506019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 0x7f506019b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55120/0 (socket says 172.21.15.86:55120) 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 -- 172.21.15.86:0/1820059155 learned_addr learned my addr 172.21.15.86:0/1820059155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f505fe00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5060105870 0x7f506019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 -- 172.21.15.86:0/1820059155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5060105870 msgr2=0x7f506019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5060105870 0x7f506019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 -- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 msgr2=0x7f506019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f506019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065200700 1 -- 172.21.15.86:0/1820059155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50601a0210 con 0x7f5060104ec0 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.590+0000 7f5065c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f506019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.591+0000 7f5065200700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 0x7f506019b200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f505400eb30 tx=0x7f505400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.591+0000 7f505d600700 1 -- 172.21.15.86:0/1820059155 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505400cbd0 con 0x7f5060104ec0 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.591+0000 7f505d600700 1 -- 172.21.15.86:0/1820059155 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50540044d0 con 0x7f5060104ec0 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.591+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50601a0500 con 0x7f5060104ec0 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.591+0000 7f505d600700 1 -- 172.21.15.86:0/1820059155 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054003e30 con 0x7f5060104ec0 2024-04-25T19:14:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.591+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601a7d80 con 0x7f5060104ec0 2024-04-25T19:14:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.594+0000 7f505d600700 1 -- 172.21.15.86:0/1820059155 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5054004000 con 0x7f5060104ec0 2024-04-25T19:14:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.596+0000 7f505d600700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f50480a3e50 0x7f50480a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.596+0000 7f5072531b80 1 Processor -- start 2024-04-25T19:14:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.596+0000 7f5072531b80 1 -- start start 2024-04-25T19:14:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.596+0000 7f505d600700 1 -- 172.21.15.86:0/1820059155 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f50540d0c30 con 0x7f5060104ec0 2024-04-25T19:14:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.596+0000 7f505fe00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f50480a3e50 0x7f50480a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5072531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee061d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5072531b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5072531b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee06f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5072531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557eedfb5020 con 0x557eee0619b0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5072531b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557eedf5b8c0 con 0x557eee0622d0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5072531b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557eedfb0fe0 con 0x557eee06d180 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5065200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee061d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f505fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5065200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee061d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49204/0 (socket says 172.21.15.86:49204) 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5065200700 1 -- 172.21.15.86:0/1548709193 learned_addr learned my addr 172.21.15.86:0/1548709193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f5065c00700 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee06f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.597+0000 7f505fe00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f50480a3e50 0x7f50480a6310 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f506010b400 tx=0x7f504c009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5065200700 1 -- 172.21.15.86:0/1548709193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 msgr2=0x557eee06f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5065200700 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee06f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5065200700 1 -- 172.21.15.86:0/1548709193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 msgr2=0x557eee06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5065200700 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5065200700 1 -- 172.21.15.86:0/1548709193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eedf51810 con 0x557eee0619b0 2024-04-25T19:14:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5065200700 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee061d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50540a5240 tx=0x7f5054098da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef344374e3a6f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5052000700 1 -- 172.21.15.86:0/1548709193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505409c460 con 0x557eee0619b0 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5052000700 1 -- 172.21.15.86:0/1548709193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50540042f0 con 0x557eee0619b0 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.598+0000 7f5052000700 1 -- 172.21.15.86:0/1548709193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50540bd070 con 0x557eee0619b0 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 -- 172.21.15.86:0/1548709193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 msgr2=0x557eee061d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee061d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50540a5240 tx=0x7f5054098da0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 -- 172.21.15.86:0/1548709193 shutdown_connections 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee06f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee061d90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 --2- 172.21.15.86:0/1548709193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 -- 172.21.15.86:0/1548709193 >> 172.21.15.86:0/1548709193 conn(0x557eedf90730 msgr2=0x557eee05de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 -- 172.21.15.86:0/1548709193 shutdown_connections 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.599+0000 7f5072531b80 1 -- 172.21.15.86:0/1548709193 wait complete. 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.600+0000 7f5072531b80 1 -- start start 2024-04-25T19:14:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.600+0000 7f5072531b80 1 Processor -- start 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.600+0000 7f5072531b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 0x557eee06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.600+0000 7f5072531b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee069c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5072531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee06d180 0x557eee06a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5072531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557eee06fca0 con 0x557eee06d180 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5072531b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557eedf6e9f0 con 0x557eee0622d0 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5072531b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557eee06fe20 con 0x557eee0619b0 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5072531b80 10 client.0 _ll_register_callbacks cb 0x557eedf4d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f505fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 0x557eee06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee06d180 0x557eee06a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f505fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee069c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55140/0 (socket says 172.21.15.86:55140) 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f505fe00700 1 -- 172.21.15.86:0/2115589846 learned_addr learned my addr 172.21.15.86:0/2115589846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 0x557eee06c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58818/0 (socket says 172.21.15.86:58818) 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 -- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 msgr2=0x557eee069c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee069c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 -- 172.21.15.86:0/2115589846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee06d180 msgr2=0x557eee06a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee06d180 0x557eee06a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f5065200700 1 -- 172.21.15.86:0/2115589846 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eedf02a60 con 0x557eee0619b0 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.601+0000 7f505fe00700 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5065200700 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 0x557eee06c900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50540044a0 tx=0x7f5054003790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c301d9c250fa9098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5046a00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:14:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5046a00700 1 -- 172.21.15.86:0/2115589846 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505409c460 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5046a00700 1 -- 172.21.15.86:0/2115589846 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f505409cd60 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50540042f0 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5046a00700 1 -- 172.21.15.86:0/2115589846 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f505409cb00 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5072531b80 10 client.26608 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557eedf4bc80 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.602+0000 7f5046a00700 1 -- 172.21.15.86:0/2115589846 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50540042f0 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5046a00700 1 -- 172.21.15.86:0/2115589846 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f50540a52e0 con 0x557eee0619b0 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5072531b80 10 client.26608 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5072531b80 10 client.26608 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5072531b80 20 client.26608 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5072531b80 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee07c850 0x557eee07ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5072531b80 4 client.26608 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:14:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eedfc2810 con 0x557eee07c850 2024-04-25T19:14:03.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.603+0000 7f505fe00700 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee07c850 0x557eee07ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.604+0000 7f505fe00700 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee07c850 0x557eee07ec40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c850e2234ec43 server_cookie=5c548069ee26cf3 in_seq=0 out_seq=0 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.604+0000 7f5046a00700 10 client.26608 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5046a00700 1 -- 172.21.15.86:0/2115589846 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eedfc2810 con 0x557eee07c850 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5046a00700 10 client.26608 handle_command_reply: tid=0 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 client.26608 shutdown 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 msgr2=0x557eee06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 0x557eee06c900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50540044a0 tx=0x7f5054003790 comp rx=0 tx=0).stop 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 shutdown_connections 2024-04-25T19:14:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee07c850 0x557eee07ec40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee06d180 0x557eee06a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee0619b0 0x557eee06c900 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 --2- 172.21.15.86:0/2115589846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 >> 172.21.15.86:0/2115589846 conn(0x557eedf90730 msgr2=0x557eee05da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5046a00700 0 client.26608 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 shutdown_connections 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.605+0000 7f5072531b80 1 -- 172.21.15.86:0/2115589846 wait complete. 2024-04-25T19:14:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.606+0000 7f5072531b80 20 client.26608 trim_cache size 0 max 16384 2024-04-25T19:14:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.626+0000 7f5072531b80 1 Processor -- start 2024-04-25T19:14:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.626+0000 7f5072531b80 1 -- start start 2024-04-25T19:14:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5072531b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0619b0 0x557eee068c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5072531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 0x557eee066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5072531b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5072531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50601af6f0 con 0x557eee0622d0 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5072531b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b1230 con 0x557eee0619b0 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5072531b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f50601afdc0 con 0x557eee06d180 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f505fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 0x557eee066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f505fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 0x557eee066ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49210/0 (socket says 172.21.15.86:49210) 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f505fe00700 1 -- 172.21.15.86:0/1424463501 learned_addr learned my addr 172.21.15.86:0/1424463501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5065200700 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0619b0 0x557eee068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.627+0000 7f5065c00700 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5065200700 1 -- 172.21.15.86:0/1424463501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 msgr2=0x557eee0673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5065200700 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee0673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5065200700 1 -- 172.21.15.86:0/1424463501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 msgr2=0x557eee066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5065200700 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 0x557eee066ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5065200700 1 -- 172.21.15.86:0/1424463501 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eee068170 con 0x557eee0619b0 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f505fe00700 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 0x557eee066ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5065200700 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0619b0 0x557eee068c00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f505409ca90 tx=0x7f5054003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364272802fcd959a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5051600700 1 -- 172.21.15.86:0/1424463501 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054098da0 con 0x557eee0619b0 2024-04-25T19:14:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5051600700 1 -- 172.21.15.86:0/1424463501 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50540042f0 con 0x557eee0619b0 2024-04-25T19:14:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.628+0000 7f5051600700 1 -- 172.21.15.86:0/1424463501 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505409cd60 con 0x557eee0619b0 2024-04-25T19:14:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 -- 172.21.15.86:0/1424463501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0619b0 msgr2=0x557eee068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0619b0 0x557eee068c00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f505409ca90 tx=0x7f5054003890 comp rx=0 tx=0).stop 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 -- 172.21.15.86:0/1424463501 shutdown_connections 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee0673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0619b0 0x557eee068c00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 --2- 172.21.15.86:0/1424463501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0622d0 0x557eee066ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 -- 172.21.15.86:0/1424463501 >> 172.21.15.86:0/1424463501 conn(0x557eedf90730 msgr2=0x557eee0684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 -- 172.21.15.86:0/1424463501 shutdown_connections 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.629+0000 7f5072531b80 1 -- 172.21.15.86:0/1424463501 wait complete. 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 -- start start 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 Processor -- start 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee07f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee07f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee07fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b0b60 con 0x557eee0619b0 2024-04-25T19:14:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b0490 con 0x557eee0622d0 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f50601ad6b0 con 0x557eee06d180 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.630+0000 7f5072531b80 10 client.0 _ll_register_callbacks cb 0x557eedf4d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee07f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee07fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee07f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49220/0 (socket says 172.21.15.86:49220) 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065200700 1 -- 172.21.15.86:0/55047156 learned_addr learned my addr 172.21.15.86:0/55047156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f505fe00700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee07f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 -- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 msgr2=0x557eee07f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee07f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 -- 172.21.15.86:0/55047156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 msgr2=0x557eee07f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee07f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 -- 172.21.15.86:0/55047156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eee0e1b00 con 0x557eee06d180 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065200700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee07f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5065c00700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee07fdf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f505800b770 tx=0x7f5058003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26387149ee7e14f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.631+0000 7f5046a00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5046a00700 1 -- 172.21.15.86:0/55047156 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505802c060 con 0x557eee06d180 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5046a00700 1 -- 172.21.15.86:0/55047156 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8637+0+0 (secure 0 0 0) 0x7f50580079b0 con 0x557eee06d180 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50540042f0 con 0x557eee06d180 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5046a00700 1 -- 172.21.15.86:0/55047156 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5058007bb0 con 0x557eee06d180 2024-04-25T19:14:03.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 10 client.26614 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557eee0e2de0 con 0x557eee06d180 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5046a00700 1 -- 172.21.15.86:0/55047156 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50540042f0 con 0x557eee06d180 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5046a00700 1 -- 172.21.15.86:0/55047156 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5058011070 con 0x557eee06d180 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 10 client.26614 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 10 client.26614 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 20 client.26614 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.632+0000 7f5072531b80 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee0e3560 0x557eee0e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.633+0000 7f5072531b80 4 client.26614 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.2, "await_for": 42.0}] 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.633+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.2, "await_for": 42.0}) -- 0x557eedfbc8b0 con 0x557eee0e3560 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.633+0000 7f5065200700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee0e3560 0x557eee0e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.633+0000 7f5065200700 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee0e3560 0x557eee0e5980 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa16601fbde5aed server_cookie=11fbffcba6bd676c in_seq=0 out_seq=0 2024-04-25T19:14:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.633+0000 7f5046a00700 10 client.26614 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5046a00700 1 -- 172.21.15.86:0/55047156 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557eedfbc8b0 con 0x557eee0e3560 2024-04-25T19:14:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5046a00700 10 client.26614 handle_command_reply: tid=0 2024-04-25T19:14:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 client.26614 shutdown 2024-04-25T19:14:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 msgr2=0x557eee07fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee07fdf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f505800b770 tx=0x7f5058003c30 comp rx=0 tx=0).stop 2024-04-25T19:14:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 shutdown_connections 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557eee0e3560 0x557eee0e5980 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eee06d180 0x557eee07fdf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eee0619b0 0x557eee07f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 --2- 172.21.15.86:0/55047156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557eee0622d0 0x557eee07f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 >> 172.21.15.86:0/55047156 conn(0x557eedf90730 msgr2=0x557eee068d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5046a00700 0 client.26614 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 shutdown_connections 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.649+0000 7f5072531b80 1 -- 172.21.15.86:0/55047156 wait complete. 2024-04-25T19:14:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.650+0000 7f5072531b80 20 client.26614 trim_cache size 0 max 16384 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 34, 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "4ddc9463": { 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 34, 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:14:03.653 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 89.2, 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:03.654 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.650+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f50480a3e50 msgr2=0x7f50480a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.650+0000 7f5067c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f50480a3e50 0x7f50480a6310 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f506010b400 tx=0x7f504c009260 comp rx=0 tx=0).stop 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.650+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 msgr2=0x7f506019b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.650+0000 7f5067c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 0x7f506019b200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f505400eb30 tx=0x7f505400ee40 comp rx=0 tx=0).stop 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 shutdown_connections 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5060109fa0 0x7f506019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5060104ec0 0x7f506019b200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f50480a3e50 0x7f50480a6310 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 --2- 172.21.15.86:0/1820059155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5060105870 0x7f506019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 >> 172.21.15.86:0/1820059155 conn(0x7f5060100750 msgr2=0x7f5060101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 shutdown_connections 2024-04-25T19:14:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:03.651+0000 7f5067c00700 1 -- 172.21.15.86:0/1820059155 wait complete. 2024-04-25T19:14:04.623 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4ddc9463', quiesce took 7.6 seconds. Will release after: 29.2 2024-04-25T19:14:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:04 smithi086 ceph-mon[27783]: pgmap v346: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T19:14:05.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:04 smithi183 ceph-mon[32344]: pgmap v346: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T19:14:05.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:04 smithi194 ceph-mon[32740]: pgmap v346: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T19:14:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:06 smithi086 ceph-mon[27783]: pgmap v347: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-25T19:14:07.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:06 smithi183 ceph-mon[32344]: pgmap v347: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-25T19:14:07.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:06 smithi194 ceph-mon[32740]: pgmap v347: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-25T19:14:08.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:07 smithi086 ceph-mon[27783]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T19:14:08.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:07 smithi183 ceph-mon[32344]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T19:14:08.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:07 smithi194 ceph-mon[32740]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T19:14:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:08 smithi086 ceph-mon[27783]: pgmap v349: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T19:14:09.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:08 smithi183 ceph-mon[32344]: pgmap v349: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T19:14:09.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:08 smithi194 ceph-mon[32740]: pgmap v349: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T19:14:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:10 smithi086 ceph-mon[27783]: pgmap v350: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 88 KiB/s wr, 37 op/s 2024-04-25T19:14:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:10 smithi183 ceph-mon[32344]: pgmap v350: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 88 KiB/s wr, 37 op/s 2024-04-25T19:14:11.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:10 smithi194 ceph-mon[32740]: pgmap v350: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 88 KiB/s wr, 37 op/s 2024-04-25T19:14:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:12 smithi086 ceph-mon[27783]: pgmap v351: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 88 KiB/s wr, 37 op/s 2024-04-25T19:14:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 -- 172.21.15.86:0/2390285834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc1092d0 msgr2=0x7f03bc110ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 --2- 172.21.15.86:0/2390285834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc1092d0 0x7f03bc110ea0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009ca0 tx=0x7f03b402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 -- 172.21.15.86:0/2390285834 shutdown_connections 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 --2- 172.21.15.86:0/2390285834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc1092d0 0x7f03bc110ea0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 --2- 172.21.15.86:0/2390285834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc104bb0 0x7f03bc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 --2- 172.21.15.86:0/2390285834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03bc104200 0x7f03bc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 -- 172.21.15.86:0/2390285834 >> 172.21.15.86:0/2390285834 conn(0x7f03bc0ffa60 msgr2=0x7f03bc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 -- 172.21.15.86:0/2390285834 shutdown_connections 2024-04-25T19:14:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.084+0000 7f03c0a00640 1 -- 172.21.15.86:0/2390285834 wait complete. 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.085+0000 7f03c0a00640 1 Processor -- start 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.085+0000 7f03c0a00640 1 -- start start 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.085+0000 7f03c0a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03bc104200 0x7f03bc19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.085+0000 7f03c0a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 0x7f03bc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.085+0000 7f03c0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc19eff0 0x7f03bc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03c0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc114270 con 0x7f03bc19eff0 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03c0a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1140f0 con 0x7f03bc1092d0 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03c0a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1143f0 con 0x7f03bc104200 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc19eff0 0x7f03bc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 0x7f03bc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 0x7f03bc19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:53856/0 (socket says 172.21.15.86:53856) 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 -- 172.21.15.86:0/963267660 learned_addr learned my addr 172.21.15.86:0/963267660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03ba800640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03bc104200 0x7f03bc19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 -- 172.21.15.86:0/963267660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03bc104200 msgr2=0x7f03bc19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03bc104200 0x7f03bc19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 -- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc19eff0 msgr2=0x7f03bc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc19eff0 0x7f03bc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03b1e00640 1 -- 172.21.15.86:0/963267660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc19fae0 con 0x7f03bc1092d0 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.086+0000 7f03bb200640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc19eff0 0x7f03bc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.087+0000 7f03b1e00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 0x7f03bc19ac70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9a0 tx=0x7f03a400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.087+0000 7f03b3e00640 1 -- 172.21.15.86:0/963267660 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a400cd80 con 0x7f03bc1092d0 2024-04-25T19:14:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.087+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc19fdd0 con 0x7f03bc1092d0 2024-04-25T19:14:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.087+0000 7f03b3e00640 1 -- 172.21.15.86:0/963267660 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a40035f0 con 0x7f03bc1092d0 2024-04-25T19:14:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.087+0000 7f03b3e00640 1 -- 172.21.15.86:0/963267660 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4010690 con 0x7f03bc1092d0 2024-04-25T19:14:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.087+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc1a7610 con 0x7f03bc1092d0 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.090+0000 7f03c423f740 1 Processor -- start 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.090+0000 7f03c423f740 1 -- start start 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03b3e00640 1 -- 172.21.15.86:0/963267660 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f03a4003fd0 con 0x7f03bc1092d0 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03c423f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 0x55fab4b07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03c423f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b082d0 0x55fab4b12650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03c423f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03c423f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fab4a6d900 con 0x55fab4b079b0 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03c423f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fab4a13ef0 con 0x55fab4b082d0 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03c423f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fab4a698c0 con 0x55fab4b12b90 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 0x55fab4b07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 0x55fab4b07d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55048/0 (socket says 172.21.15.86:55048) 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03bb200640 1 -- 172.21.15.86:0/1995713739 learned_addr learned my addr 172.21.15.86:0/1995713739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03ba800640 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b082d0 0x55fab4b12650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.091+0000 7f03b1e00640 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03bb200640 1 -- 172.21.15.86:0/1995713739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 msgr2=0x55fab4b14f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03bb200640 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b14f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03bb200640 1 -- 172.21.15.86:0/1995713739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b082d0 msgr2=0x55fab4b12650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03bb200640 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b082d0 0x55fab4b12650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03bb200640 1 -- 172.21.15.86:0/1995713739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fab4780210 con 0x55fab4b079b0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03ba800640 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b082d0 0x55fab4b12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03bb200640 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 0x55fab4b07d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009ca0 tx=0x7f03b4005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a1819f3a07e6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03ab400640 1 -- 172.21.15.86:0/1995713739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4031450 con 0x55fab4b079b0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03ab400640 1 -- 172.21.15.86:0/1995713739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b4036040 con 0x55fab4b079b0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03ab400640 1 -- 172.21.15.86:0/1995713739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004460 con 0x55fab4b079b0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.092+0000 7f03b3e00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f03a00a3d10 0x7f03a00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03b3e00640 1 -- 172.21.15.86:0/963267660 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8681+0+0 (secure 0 0 0) 0x7f03a40d0ac0 con 0x7f03bc1092d0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03ba800640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f03a00a3d10 0x7f03a00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 -- 172.21.15.86:0/1995713739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 msgr2=0x55fab4b07d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 0x55fab4b07d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009ca0 tx=0x7f03b4005f80 comp rx=0 tx=0).stop 2024-04-25T19:14:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 -- 172.21.15.86:0/1995713739 shutdown_connections 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b14f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b082d0 0x55fab4b12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 --2- 172.21.15.86:0/1995713739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b079b0 0x55fab4b07d90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 -- 172.21.15.86:0/1995713739 >> 172.21.15.86:0/1995713739 conn(0x55fab4b03ec0 msgr2=0x55fab4b042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.093+0000 7f03c423f740 1 -- 172.21.15.86:0/1995713739 shutdown_connections 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.094+0000 7f03c423f740 1 -- 172.21.15.86:0/1995713739 wait complete. 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.094+0000 7f03ba800640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f03a00a3d10 0x7f03a00a61d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55fab4b08f50 tx=0x7f03ac005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.094+0000 7f03c423f740 1 -- start start 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.094+0000 7f03c423f740 1 Processor -- start 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b079b0 0x55fab4b0fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b10080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 0x55fab4b105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fab4b155e0 con 0x55fab4b082d0 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fab4a27310 con 0x55fab4b12b90 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fab4b15760 con 0x55fab4b079b0 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03c423f740 10 client.0 _ll_register_callbacks cb 0x55fab49f5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b10080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03bb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 0x55fab4b105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b10080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55056/0 (socket says 172.21.15.86:55056) 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03ba800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b079b0 0x55fab4b0fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03bb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 0x55fab4b105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:53874/0 (socket says 172.21.15.86:53874) 2024-04-25T19:14:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 -- 172.21.15.86:0/3171644792 learned_addr learned my addr 172.21.15.86:0/3171644792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 -- 172.21.15.86:0/3171644792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b079b0 msgr2=0x55fab4b0fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b079b0 0x55fab4b0fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 -- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 msgr2=0x55fab4b105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 0x55fab4b105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03b1e00640 1 -- 172.21.15.86:0/3171644792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fab484c620 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.095+0000 7f03bb200640 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 0x55fab4b105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f03b1e00640 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b10080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f03a40ce170 tx=0x7f03a40bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3676785c60a525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f039fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f039fe00640 1 -- 172.21.15.86:0/3171644792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40d7070 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f039fe00640 1 -- 172.21.15.86:0/3171644792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8681+0+0 (secure 0 0 0) 0x7f03a40037a0 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b4036040 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f039fe00640 1 -- 172.21.15.86:0/3171644792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03a409c4a0 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f03c423f740 10 client.16530 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fab4b218c0 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.096+0000 7f039fe00640 1 -- 172.21.15.86:0/3171644792 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b4036040 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f039fe00640 1 -- 172.21.15.86:0/3171644792 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f03a40ce390 con 0x55fab4b082d0 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03c423f740 10 client.16530 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03c423f740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03c423f740 20 client.16530 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03c423f740 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b22d30 0x55fab4b25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03c423f740 4 client.16530 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fab4a818b0 con 0x55fab4b22d30 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.097+0000 7f03ba800640 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b22d30 0x55fab4b25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.098+0000 7f03ba800640 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b22d30 0x55fab4b25120 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4597cfcccd390498 server_cookie=b6e3b2506213858f in_seq=0 out_seq=0 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.098+0000 7f039fe00640 10 client.16530 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f039fe00640 1 -- 172.21.15.86:0/3171644792 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fab4a818b0 con 0x55fab4b22d30 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f039fe00640 10 client.16530 handle_command_reply: tid=0 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 client.16530 shutdown 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 msgr2=0x55fab4b10080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b10080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f03a40ce170 tx=0x7f03a40bd0d0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 shutdown_connections 2024-04-25T19:14:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b22d30 0x55fab4b25120 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b12b90 0x55fab4b105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b10080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 --2- 172.21.15.86:0/3171644792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b079b0 0x55fab4b0fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f039fe00640 0 client.16530 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 >> 172.21.15.86:0/3171644792 conn(0x55fab4b03ec0 msgr2=0x55fab4b09c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.099+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 shutdown_connections 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.100+0000 7f03c423f740 1 -- 172.21.15.86:0/3171644792 wait complete. 2024-04-25T19:14:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.100+0000 7f03c423f740 20 client.16530 trim_cache size 0 max 16384 2024-04-25T19:14:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 Processor -- start 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 -- start start 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b082d0 0x55fab4b2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 0x55fab4b2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1afe20 con 0x55fab4b12b90 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b1960 con 0x55fab4b079b0 2024-04-25T19:14:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03c423f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b04f0 con 0x55fab4b082d0 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.116+0000 7f03bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 0x55fab4b2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 0x55fab4b2af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55062/0 (socket says 172.21.15.86:55062) 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 -- 172.21.15.86:0/4085623228 learned_addr learned my addr 172.21.15.86:0/4085623228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03b1e00640 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b082d0 0x55fab4b2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03ba800640 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 -- 172.21.15.86:0/4085623228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b082d0 msgr2=0x55fab4b2a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b082d0 0x55fab4b2a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 -- 172.21.15.86:0/4085623228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 msgr2=0x55fab4b1e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 -- 172.21.15.86:0/4085623228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fab4b25490 con 0x55fab4b12b90 2024-04-25T19:14:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03b1e00640 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b082d0 0x55fab4b2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.117+0000 7f03bb200640 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 0x55fab4b2af20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f03b402aeb0 tx=0x7f03b4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe8344f63e73f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03aaa00640 1 -- 172.21.15.86:0/4085623228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4032eb0 con 0x55fab4b12b90 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03aaa00640 1 -- 172.21.15.86:0/4085623228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b4036040 con 0x55fab4b12b90 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03aaa00640 1 -- 172.21.15.86:0/4085623228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b402fa10 con 0x55fab4b12b90 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 -- 172.21.15.86:0/4085623228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 msgr2=0x55fab4b2af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 0x55fab4b2af20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f03b402aeb0 tx=0x7f03b4009a20 comp rx=0 tx=0).stop 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 -- 172.21.15.86:0/4085623228 shutdown_connections 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b12b90 0x55fab4b2af20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b082d0 0x55fab4b2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 --2- 172.21.15.86:0/4085623228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 -- 172.21.15.86:0/4085623228 >> 172.21.15.86:0/4085623228 conn(0x55fab4b03ec0 msgr2=0x55fab4b0ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 -- 172.21.15.86:0/4085623228 shutdown_connections 2024-04-25T19:14:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.118+0000 7f03c423f740 1 -- 172.21.15.86:0/4085623228 wait complete. 2024-04-25T19:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.119+0000 7f03c423f740 1 -- start start 2024-04-25T19:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.119+0000 7f03c423f740 1 Processor -- start 2024-04-25T19:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.119+0000 7f03c423f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.119+0000 7f03c423f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03c423f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03c423f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b1290 con 0x55fab4b082d0 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03c423f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b0bc0 con 0x55fab4b079b0 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03c423f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1ade20 con 0x55fab4b12b90 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03c423f740 10 client.0 _ll_register_callbacks cb 0x7f03a4004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03ba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b17f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55070/0 (socket says 172.21.15.86:55070) 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 -- 172.21.15.86:0/486129884 learned_addr learned my addr 172.21.15.86:0/486129884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03bb200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 -- 172.21.15.86:0/486129884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 msgr2=0x55fab4b184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b184a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 -- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 msgr2=0x55fab4b1ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 -- 172.21.15.86:0/486129884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fab4b18a90 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03ba800640 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f03b1e00640 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b17f60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f03a4098dc0 tx=0x7f03a4087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c58c057fc2cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.120+0000 7f039fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f039fe00640 1 -- 172.21.15.86:0/486129884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40d7070 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f039fe00640 1 -- 172.21.15.86:0/486129884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8681+0+0 (secure 0 0 0) 0x7f03a409bfb0 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b4036040 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f039fe00640 1 -- 172.21.15.86:0/486129884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03a409c2c0 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 10 client.16536 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fab4b89070 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f039fe00640 1 -- 172.21.15.86:0/486129884 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b4036040 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f039fe00640 1 -- 172.21.15.86:0/486129884 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f03a40986b0 con 0x55fab4b082d0 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 10 client.16536 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 20 client.16536 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b89660 0x55fab4b8ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 4 client.16536 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.121+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fab4a7b320 con 0x55fab4b89660 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.122+0000 7f03ba800640 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b89660 0x55fab4b8ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.122+0000 7f03ba800640 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b89660 0x55fab4b8ba80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652c040e44ce0611 server_cookie=9c4a73a146b37d0b in_seq=0 out_seq=0 2024-04-25T19:14:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.122+0000 7f039fe00640 10 client.16536 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f039fe00640 1 -- 172.21.15.86:0/486129884 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fab4a7b320 con 0x55fab4b89660 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f039fe00640 10 client.16536 handle_command_reply: tid=0 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 client.16536 shutdown 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 msgr2=0x55fab4b17f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b17f60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f03a4098dc0 tx=0x7f03a4087040 comp rx=0 tx=0).stop 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 shutdown_connections 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55fab4b89660 0x55fab4b8ba80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fab4b12b90 0x55fab4b184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fab4b082d0 0x55fab4b17f60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f039fe00640 0 client.16536 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 --2- 172.21.15.86:0/486129884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fab4b079b0 0x55fab4b1ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 >> 172.21.15.86:0/486129884 conn(0x55fab4b03ec0 msgr2=0x55fab4b093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 shutdown_connections 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.137+0000 7f03c423f740 1 -- 172.21.15.86:0/486129884 wait complete. 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c423f740 20 client.16536 trim_cache size 0 max 16384 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:14:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f03a00a3d10 msgr2=0x7f03a00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f03a00a3d10 0x7f03a00a61d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55fab4b08f50 tx=0x7f03ac005f90 comp rx=0 tx=0).stop 2024-04-25T19:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 msgr2=0x7f03bc19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 0x7f03bc19ac70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f03a400e9a0 tx=0x7f03a400ee60 comp rx=0 tx=0).stop 2024-04-25T19:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 shutdown_connections 2024-04-25T19:14:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f03a00a3d10 0x7f03a00a61d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03bc19eff0 0x7f03bc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03bc1092d0 0x7f03bc19ac70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 --2- 172.21.15.86:0/963267660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03bc104200 0x7f03bc19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.138+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 >> 172.21.15.86:0/963267660 conn(0x7f03bc0ffa60 msgr2=0x7f03bc101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.139+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 shutdown_connections 2024-04-25T19:14:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:13.139+0000 7f03c0a00640 1 -- 172.21.15.86:0/963267660 wait complete. 2024-04-25T19:14:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:12 smithi183 ceph-mon[32344]: pgmap v351: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 88 KiB/s wr, 37 op/s 2024-04-25T19:14:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:12 smithi194 ceph-mon[32740]: pgmap v351: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 88 KiB/s wr, 37 op/s 2024-04-25T19:14:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:14 smithi086 ceph-mon[27783]: pgmap v352: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-25T19:14:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:14 smithi086 ceph-mon[27783]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T19:14:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:14 smithi183 ceph-mon[32344]: pgmap v352: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-25T19:14:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:14 smithi183 ceph-mon[32344]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T19:14:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:14 smithi194 ceph-mon[32740]: pgmap v352: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-25T19:14:14.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:14 smithi194 ceph-mon[32740]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T19:14:16.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:16 smithi086 ceph-mon[27783]: pgmap v354: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-04-25T19:14:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:16 smithi183 ceph-mon[32344]: pgmap v354: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-04-25T19:14:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:16 smithi194 ceph-mon[32740]: pgmap v354: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-04-25T19:14:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:18 smithi086 ceph-mon[27783]: pgmap v355: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-25T19:14:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:18 smithi183 ceph-mon[32344]: pgmap v355: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-25T19:14:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:18 smithi194 ceph-mon[32740]: pgmap v355: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-04-25T19:14:20.784 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:20 smithi194 ceph-mon[32740]: pgmap v356: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-04-25T19:14:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:20 smithi086 ceph-mon[27783]: pgmap v356: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-04-25T19:14:20.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:20 smithi183 ceph-mon[32344]: pgmap v356: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-04-25T19:14:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:22 smithi086 ceph-mon[27783]: pgmap v357: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-04-25T19:14:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:22 smithi183 ceph-mon[32344]: pgmap v357: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-04-25T19:14:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:22 smithi194 ceph-mon[32740]: pgmap v357: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-04-25T19:14:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:24 smithi086 ceph-mon[27783]: pgmap v358: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-04-25T19:14:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:24 smithi183 ceph-mon[32344]: pgmap v358: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-04-25T19:14:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:24 smithi194 ceph-mon[32740]: pgmap v358: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-04-25T19:14:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 -- 172.21.15.86:0/491300766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f126810c5d0 msgr2=0x7f126810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 --2- 172.21.15.86:0/491300766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f126810c5d0 0x7f126810ea00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1260009ca0 tx=0x7f126002a950 comp rx=0 tx=0).stop 2024-04-25T19:14:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 -- 172.21.15.86:0/491300766 shutdown_connections 2024-04-25T19:14:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 --2- 172.21.15.86:0/491300766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f126810c5d0 0x7f126810ea00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 --2- 172.21.15.86:0/491300766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1268103b20 0x7f126810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 --2- 172.21.15.86:0/491300766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1268103200 0x7f12681035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 -- 172.21.15.86:0/491300766 >> 172.21.15.86:0/491300766 conn(0x7f12680771d0 msgr2=0x7f12680775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 -- 172.21.15.86:0/491300766 shutdown_connections 2024-04-25T19:14:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.280+0000 7f126dc00640 1 -- 172.21.15.86:0/491300766 wait complete. 2024-04-25T19:14:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.281+0000 7f126dc00640 1 Processor -- start 2024-04-25T19:14:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.281+0000 7f126dc00640 1 -- start start 2024-04-25T19:14:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f126dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1268103200 0x7f12681a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f126dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 0x7f12681a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f126dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 0x7f126819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f126dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1268112220 con 0x7f1268103b20 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f126dc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12681120a0 con 0x7f1268103200 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f126dc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12681123a0 con 0x7f126810c5d0 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f125fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 0x7f12681a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f1267400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 0x7f126819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f125fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 0x7f12681a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52044/0 (socket says 172.21.15.86:52044) 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f1267400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 0x7f126819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47710/0 (socket says 172.21.15.86:47710) 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f125fe00640 1 -- 172.21.15.86:0/3790797071 learned_addr learned my addr 172.21.15.86:0/3790797071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.282+0000 7f1266a00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1268103200 0x7f12681a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f1267400640 1 -- 172.21.15.86:0/3790797071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1268103200 msgr2=0x7f12681a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f1267400640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1268103200 0x7f12681a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f1267400640 1 -- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 msgr2=0x7f12681a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f1267400640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 0x7f12681a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f1267400640 1 -- 172.21.15.86:0/3790797071 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126819afd0 con 0x7f126810c5d0 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f125fe00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 0x7f12681a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f1267400640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 0x7f126819a710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f126002af40 tx=0x7f1260003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f125f400640 1 -- 172.21.15.86:0/3790797071 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260034040 con 0x7f126810c5d0 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126819b260 con 0x7f126810c5d0 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f125f400640 1 -- 172.21.15.86:0/3790797071 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f126002fd20 con 0x7f126810c5d0 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f125f400640 1 -- 172.21.15.86:0/3790797071 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600044d0 con 0x7f126810c5d0 2024-04-25T19:14:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.283+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a73f0 con 0x7f126810c5d0 2024-04-25T19:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f125f400640 1 -- 172.21.15.86:0/3790797071 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1260037020 con 0x7f126810c5d0 2024-04-25T19:14:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 Processor -- start 2024-04-25T19:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 -- start start 2024-04-25T19:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d052c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d05d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d05fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56272cfb8900 con 0x56272d05da00 2024-04-25T19:14:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56272cf5eef0 con 0x56272d053140 2024-04-25T19:14:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.287+0000 7f1270a1f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56272cfb48c0 con 0x56272d052820 2024-04-25T19:14:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f125fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d05fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f125fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d05fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52048/0 (socket says 172.21.15.86:52048) 2024-04-25T19:14:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f125fe00640 1 -- 172.21.15.86:0/45089079 learned_addr learned my addr 172.21.15.86:0/45089079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f1266a00640 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d05d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f125fe00640 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d052c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f1266a00640 1 -- 172.21.15.86:0/45089079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 msgr2=0x56272d052c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f1266a00640 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d052c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f1266a00640 1 -- 172.21.15.86:0/45089079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 msgr2=0x56272d05fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f1266a00640 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d05fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f1266a00640 1 -- 172.21.15.86:0/45089079 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56272cccb210 con 0x56272d053140 2024-04-25T19:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f125fe00640 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d05fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.288+0000 7f125f400640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f124c0a3e40 0x7f124c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.289+0000 7f125f400640 1 -- 172.21.15.86:0/3790797071 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7f12600f1400 con 0x7f126810c5d0 2024-04-25T19:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.289+0000 7f1266a00640 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d05d4c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1258009c20 tx=0x7f1258011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d1f8f11715f1bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.289+0000 7f125fe00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f124c0a3e40 0x7f124c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.289+0000 7f1257e00640 1 -- 172.21.15.86:0/45089079 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125801e070 con 0x56272d053140 2024-04-25T19:14:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.289+0000 7f1257e00640 1 -- 172.21.15.86:0/45089079 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258009dd0 con 0x56272d053140 2024-04-25T19:14:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.289+0000 7f1257e00640 1 -- 172.21.15.86:0/45089079 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258018730 con 0x56272d053140 2024-04-25T19:14:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 -- 172.21.15.86:0/45089079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 msgr2=0x56272d05d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f125fe00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f124c0a3e40 0x7f124c0a6300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56272d050150 tx=0x7f1250004730 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d05d4c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1258009c20 tx=0x7f1258011810 comp rx=0 tx=0).stop 2024-04-25T19:14:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 -- 172.21.15.86:0/45089079 shutdown_connections 2024-04-25T19:14:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d05fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d05d4c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 --2- 172.21.15.86:0/45089079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d052c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 -- 172.21.15.86:0/45089079 >> 172.21.15.86:0/45089079 conn(0x56272d04ed30 msgr2=0x56272d04f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 -- 172.21.15.86:0/45089079 shutdown_connections 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 -- 172.21.15.86:0/45089079 wait complete. 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.290+0000 7f1270a1f740 1 -- start start 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 Processor -- start 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d05a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d05af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d05b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56272d060630 con 0x56272d052820 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56272cf72310 con 0x56272d05da00 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56272d0607b0 con 0x56272d053140 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1270a1f740 10 client.0 _ll_register_callbacks cb 0x56272cf40cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1266a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d05a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1267400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d05b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1266a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d05a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52064/0 (socket says 172.21.15.86:52064) 2024-04-25T19:14:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.291+0000 7f1266a00640 1 -- 172.21.15.86:0/3289172670 learned_addr learned my addr 172.21.15.86:0/3289172670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f125fe00640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d05af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1267400640 1 -- 172.21.15.86:0/3289172670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 msgr2=0x56272d05af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1267400640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d05af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1267400640 1 -- 172.21.15.86:0/3289172670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 msgr2=0x56272d05a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1267400640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d05a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1267400640 1 -- 172.21.15.86:0/3289172670 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56272cd97620 con 0x56272d05da00 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1266a00640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d05a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1267400640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d05b470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f12600eeb90 tx=0x7f12600bd270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96940d0dc97c560c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1254c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1254c00640 1 -- 172.21.15.86:0/3289172670 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600d6060 con 0x56272d05da00 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1254c00640 1 -- 172.21.15.86:0/3289172670 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7f12600306e0 con 0x56272d05da00 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.292+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258009dd0 con 0x56272d05da00 2024-04-25T19:14:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1254c00640 1 -- 172.21.15.86:0/3289172670 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f126002c040 con 0x56272d05da00 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1270a1f740 10 client.26651 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56272d06c8f0 con 0x56272d05da00 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1254c00640 1 -- 172.21.15.86:0/3289172670 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258009dd0 con 0x56272d05da00 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1254c00640 1 -- 172.21.15.86:0/3289172670 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1260004670 con 0x56272d05da00 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1270a1f740 10 client.26651 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.293+0000 7f1270a1f740 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.294+0000 7f1270a1f740 20 client.26651 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.294+0000 7f1270a1f740 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d06de50 0x56272d0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.294+0000 7f1270a1f740 4 client.26651 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.294+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56272cfcc8b0 con 0x56272d06de50 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.294+0000 7f1266a00640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d06de50 0x56272d0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.295+0000 7f1254c00640 10 client.26651 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.295+0000 7f1266a00640 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d06de50 0x56272d0702d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8e77e246895a0f server_cookie=41d9ff15968d06dc in_seq=0 out_seq=0 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.295+0000 7f1254c00640 1 -- 172.21.15.86:0/3289172670 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56272cfcc8b0 con 0x56272d06de50 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.295+0000 7f1254c00640 10 client.26651 handle_command_reply: tid=0 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.295+0000 7f1270a1f740 1 client.26651 shutdown 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 msgr2=0x56272d05b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d05b470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f12600eeb90 tx=0x7f12600bd270 comp rx=0 tx=0).stop 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 shutdown_connections 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d06de50 0x56272d0702d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d05b470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1254c00640 0 client.26651 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d05af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 --2- 172.21.15.86:0/3289172670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d05a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 >> 172.21.15.86:0/3289172670 conn(0x56272d04ed30 msgr2=0x56272d054a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 shutdown_connections 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 1 -- 172.21.15.86:0/3289172670 wait complete. 2024-04-25T19:14:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.296+0000 7f1270a1f740 20 client.26651 trim_cache size 0 max 16384 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 Processor -- start 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 -- start start 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d04de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d0551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12681afe00 con 0x56272d05da00 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12681b1940 con 0x56272d053140 2024-04-25T19:14:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.312+0000 7f1270a1f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12681b04d0 con 0x56272d052820 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1267400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d0551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d0694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47742/0 (socket says 172.21.15.86:47742) 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 -- 172.21.15.86:0/1581337968 learned_addr learned my addr 172.21.15.86:0/1581337968 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f125fe00640 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d04de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 -- 172.21.15.86:0/1581337968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 msgr2=0x56272d04de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d04de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 -- 172.21.15.86:0/1581337968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 msgr2=0x56272d0551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d0551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 -- 172.21.15.86:0/1581337968 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56272d0599a0 con 0x56272d052820 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f125fe00640 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d04de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.313+0000 7f1266a00640 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d0694f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12580097e0 tx=0x7f12580183b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a78a270c224432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1257400640 1 -- 172.21.15.86:0/1581337968 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125802ae60 con 0x56272d052820 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1257400640 1 -- 172.21.15.86:0/1581337968 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258030070 con 0x56272d052820 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1257400640 1 -- 172.21.15.86:0/1581337968 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125801e040 con 0x56272d052820 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 -- 172.21.15.86:0/1581337968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 msgr2=0x56272d0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d0694f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12580097e0 tx=0x7f12580183b0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 -- 172.21.15.86:0/1581337968 shutdown_connections 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d05da00 0x56272d0551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d053140 0x56272d04de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 --2- 172.21.15.86:0/1581337968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d052820 0x56272d0694f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 -- 172.21.15.86:0/1581337968 >> 172.21.15.86:0/1581337968 conn(0x56272d04ed30 msgr2=0x56272d04f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.314+0000 7f1270a1f740 1 -- 172.21.15.86:0/1581337968 shutdown_connections 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 -- 172.21.15.86:0/1581337968 wait complete. 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 -- start start 2024-04-25T19:14:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 Processor -- start 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d065ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d062fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d063510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12681b1270 con 0x56272d052820 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12681b0ba0 con 0x56272d05da00 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.315+0000 7f1270a1f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12681ade00 con 0x56272d053140 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1270a1f740 10 client.0 _ll_register_callbacks cb 0x7f12600b8f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1266a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d065ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f125fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d062fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f125fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d062fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47744/0 (socket says 172.21.15.86:47744) 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f125fe00640 1 -- 172.21.15.86:0/496937665 learned_addr learned my addr 172.21.15.86:0/496937665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1267400640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d063510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1266a00640 1 -- 172.21.15.86:0/496937665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 msgr2=0x56272d062fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1266a00640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d062fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1266a00640 1 -- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 msgr2=0x56272d063510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1266a00640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d063510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1266a00640 1 -- 172.21.15.86:0/496937665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56272d063a50 con 0x56272d052820 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.316+0000 7f1267400640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d063510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1266a00640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d065ad0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f12580097b0 tx=0x7f1258009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3137878624677484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1254c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1254c00640 1 -- 172.21.15.86:0/496937665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258038070 con 0x56272d052820 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1254c00640 1 -- 172.21.15.86:0/496937665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7f1258009dd0 con 0x56272d052820 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258030070 con 0x56272d052820 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1254c00640 1 -- 172.21.15.86:0/496937665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f125802ae60 con 0x56272d052820 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1270a1f740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56272d0d4140 con 0x56272d052820 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.317+0000 7f1254c00640 1 -- 172.21.15.86:0/496937665 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258030070 con 0x56272d052820 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1254c00640 1 -- 172.21.15.86:0/496937665 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1258004610 con 0x56272d052820 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1270a1f740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1270a1f740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1270a1f740 20 client.16554 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1270a1f740 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d0d4660 0x56272d0d6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1270a1f740 4 client.16554 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56272cfc6320 con 0x56272d0d4660 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.318+0000 7f125fe00640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d0d4660 0x56272d0d6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.319+0000 7f125fe00640 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d0d4660 0x56272d0d6a50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56594bc6a7981bb2 server_cookie=a4bb82cf9226b4a5 in_seq=0 out_seq=0 2024-04-25T19:14:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.319+0000 7f1254c00640 10 client.16554 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1254c00640 1 -- 172.21.15.86:0/496937665 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56272cfc6320 con 0x56272d0d4660 2024-04-25T19:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1254c00640 10 client.16554 handle_command_reply: tid=0 2024-04-25T19:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 client.16554 shutdown 2024-04-25T19:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 msgr2=0x56272d065ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d065ad0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f12580097b0 tx=0x7f1258009850 comp rx=0 tx=0).stop 2024-04-25T19:14:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 shutdown_connections 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x56272d0d4660 0x56272d0d6a50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56272d05da00 0x56272d063510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56272d053140 0x56272d062fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 --2- 172.21.15.86:0/496937665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56272d052820 0x56272d065ad0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1254c00640 0 client.16554 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.330+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 >> 172.21.15.86:0/496937665 conn(0x56272d04ed30 msgr2=0x56272d04de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.331+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 shutdown_connections 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.331+0000 7f1270a1f740 1 -- 172.21.15.86:0/496937665 wait complete. 2024-04-25T19:14:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.331+0000 7f1270a1f740 20 client.16554 trim_cache size 0 max 16384 2024-04-25T19:14:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:14:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:14:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:14:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.331+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f124c0a3e40 msgr2=0x7f124c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.331+0000 7f126dc00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f124c0a3e40 0x7f124c0a6300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56272d050150 tx=0x7f1250004730 comp rx=0 tx=0).stop 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 msgr2=0x7f126819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 0x7f126819a710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f126002af40 tx=0x7f1260003c10 comp rx=0 tx=0).stop 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 shutdown_connections 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f124c0a3e40 0x7f124c0a6300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f126810c5d0 0x7f126819a710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268103b20 0x7f12681a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 --2- 172.21.15.86:0/3790797071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1268103200 0x7f12681a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 >> 172.21.15.86:0/3790797071 conn(0x7f12680771d0 msgr2=0x7f1268100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 shutdown_connections 2024-04-25T19:14:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:26.332+0000 7f126dc00640 1 -- 172.21.15.86:0/3790797071 wait complete. 2024-04-25T19:14:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:26 smithi086 ceph-mon[27783]: pgmap v359: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-04-25T19:14:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:26 smithi183 ceph-mon[32344]: pgmap v359: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-04-25T19:14:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:26 smithi194 ceph-mon[32740]: pgmap v359: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-04-25T19:14:27.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:27 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:14:27.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:27 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:14:27.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:27 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:14:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:28 smithi086 ceph-mon[27783]: pgmap v360: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-04-25T19:14:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:28 smithi183 ceph-mon[32344]: pgmap v360: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-04-25T19:14:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:28 smithi194 ceph-mon[32740]: pgmap v360: 97 pgs: 97 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-04-25T19:14:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:29 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:14:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:29 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:14:29.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:29 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:14:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:29 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:14:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:29 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:14:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:29 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:14:29.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:29 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:14:29.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:29 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:14:29.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:29 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:14:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:30 smithi194 ceph-mon[32740]: pgmap v361: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T19:14:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:30 smithi086 ceph-mon[27783]: pgmap v361: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T19:14:31.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:30 smithi183 ceph-mon[32344]: pgmap v361: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T19:14:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:32 smithi194 ceph-mon[32740]: pgmap v362: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T19:14:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:32 smithi086 ceph-mon[27783]: pgmap v362: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T19:14:33.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:32 smithi183 ceph-mon[32344]: pgmap v362: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T19:14:33.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:14:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:14:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:34 smithi194 ceph-mon[32740]: pgmap v363: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-25T19:14:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:34 smithi086 ceph-mon[27783]: pgmap v363: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-25T19:14:35.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:34 smithi183 ceph-mon[32344]: pgmap v363: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 -- 172.21.15.86:0/518359597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109310 msgr2=0x7fce381096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 --2- 172.21.15.86:0/518359597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109310 0x7fce381096f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fce280099c0 tx=0x7fce28009cd0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 -- 172.21.15.86:0/518359597 shutdown_connections 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 --2- 172.21.15.86:0/518359597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce3810e3a0 0x7fce38115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 --2- 172.21.15.86:0/518359597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109310 0x7fce381096f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 --2- 172.21.15.86:0/518359597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce38109cc0 0x7fce3810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.539+0000 7fce3dc00700 1 -- 172.21.15.86:0/518359597 >> 172.21.15.86:0/518359597 conn(0x7fce38076210 msgr2=0x7fce38078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.540+0000 7fce3dc00700 1 -- 172.21.15.86:0/518359597 shutdown_connections 2024-04-25T19:14:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.540+0000 7fce3dc00700 1 -- 172.21.15.86:0/518359597 wait complete. 2024-04-25T19:14:36.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 Processor -- start 2024-04-25T19:14:36.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 -- start start 2024-04-25T19:14:36.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38109310 0x7fce3819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 0x7fce3819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce3810e3a0 0x7fce381a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce38119280 con 0x7fce38109cc0 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce38119100 con 0x7fce38109310 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce3dc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fce38119400 con 0x7fce3810e3a0 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce2fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 0x7fce3819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce2fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 0x7fce3819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43782/0 (socket says 172.21.15.86:43782) 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.541+0000 7fce2fe00700 1 -- 172.21.15.86:0/4230527210 learned_addr learned my addr 172.21.15.86:0/4230527210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce36a00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38109310 0x7fce3819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce37400700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce3810e3a0 0x7fce381a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce2fe00700 1 -- 172.21.15.86:0/4230527210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce3810e3a0 msgr2=0x7fce381a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce2fe00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce3810e3a0 0x7fce381a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce2fe00700 1 -- 172.21.15.86:0/4230527210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38109310 msgr2=0x7fce3819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce2fe00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38109310 0x7fce3819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce2fe00700 1 -- 172.21.15.86:0/4230527210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce381a4750 con 0x7fce38109cc0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce36a00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38109310 0x7fce3819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.542+0000 7fce2fe00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 0x7fce3819fcd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e970 tx=0x7fce2000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.543+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20004a30 con 0x7fce38109cc0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.543+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2000cc90 con 0x7fce38109cc0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.543+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce381a4a40 con 0x7fce38109cc0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.543+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20005610 con 0x7fce38109cc0 2024-04-25T19:14:36.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.543+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce381ac2c0 con 0x7fce38109cc0 2024-04-25T19:14:36.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.544+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce38069730 con 0x7fce38109cc0 2024-04-25T19:14:36.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.546+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce2000c7e0 con 0x7fce38109cc0 2024-04-25T19:14:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.548+0000 7fce2f400700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fce1c0a3e30 0x7fce1c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.549+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fce200d0a00 con 0x7fce38109cc0 2024-04-25T19:14:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.549+0000 7fce36a00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fce1c0a3e30 0x7fce1c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:36.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.550+0000 7fce36a00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fce1c0a3e30 0x7fce1c0a62f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce28003760 tx=0x7fce28005fb0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:36.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.553+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce2009cfd0 con 0x7fce38109cc0 2024-04-25T19:14:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.826+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce3802df80 con 0x7fce38109cc0 2024-04-25T19:14:36.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.828+0000 7fce2f400700 1 -- 172.21.15.86:0/4230527210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fce2009c8d0 con 0x7fce38109cc0 2024-04-25T19:14:36.831 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:14:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fce1c0a3e30 msgr2=0x7fce1c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fce1c0a3e30 0x7fce1c0a62f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fce28003760 tx=0x7fce28005fb0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 msgr2=0x7fce3819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 0x7fce3819fcd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e970 tx=0x7fce2000ec80 comp rx=0 tx=0).stop 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 shutdown_connections 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fce1c0a3e30 0x7fce1c0a62f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce3810e3a0 0x7fce381a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce38109310 0x7fce3819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 --2- 172.21.15.86:0/4230527210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce38109cc0 0x7fce3819fcd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 >> 172.21.15.86:0/4230527210 conn(0x7fce38076210 msgr2=0x7fce380775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.831+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 shutdown_connections 2024-04-25T19:14:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:36.832+0000 7fce3dc00700 1 -- 172.21.15.86:0/4230527210 wait complete. 2024-04-25T19:14:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:14:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:36 smithi194 ceph-mon[32740]: pgmap v364: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-04-25T19:14:37.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:36 smithi086 ceph-mon[27783]: pgmap v364: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-04-25T19:14:37.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:36 smithi183 ceph-mon[32344]: pgmap v364: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-04-25T19:14:37.647 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 4ddc9463 --release --await' 2024-04-25T19:14:37.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 4ddc9463 --release --await 2024-04-25T19:14:38.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:37 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/4230527210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:14:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:14:38.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:37 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/4230527210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:14:38.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:37 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/4230527210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:14:38.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:38 smithi086 ceph-mon[27783]: pgmap v365: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-04-25T19:14:39.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:38 smithi183 ceph-mon[32344]: pgmap v365: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-04-25T19:14:39.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:38 smithi194 ceph-mon[32740]: pgmap v365: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 -- 172.21.15.86:0/2657827730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 msgr2=0x7fa0601025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 --2- 172.21.15.86:0/2657827730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 0x7fa0601025b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa054009860 tx=0x7fa05402a340 comp rx=0 tx=0).stop 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 -- 172.21.15.86:0/2657827730 shutdown_connections 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 --2- 172.21.15.86:0/2657827730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa06010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 --2- 172.21.15.86:0/2657827730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa060102b80 0x7fa060106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 --2- 172.21.15.86:0/2657827730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 0x7fa0601025b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 -- 172.21.15.86:0/2657827730 >> 172.21.15.86:0/2657827730 conn(0x7fa0600793e0 msgr2=0x7fa0600797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 -- 172.21.15.86:0/2657827730 shutdown_connections 2024-04-25T19:14:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.474+0000 7fa068400640 1 -- 172.21.15.86:0/2657827730 wait complete. 2024-04-25T19:14:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 Processor -- start 2024-04-25T19:14:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 -- start start 2024-04-25T19:14:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 0x7fa0601a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa0601a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa06019a750 0x7fa06019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601121e0 con 0x7fa0601072a0 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa060112060 con 0x7fa0601021d0 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.476+0000 7fa068400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa060112360 con 0x7fa06019a750 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa0601a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa0601a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43794/0 (socket says 172.21.15.86:43794) 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 -- 172.21.15.86:0/4195936599 learned_addr learned my addr 172.21.15.86:0/4195936599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa066400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa06019a750 0x7fa06019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065a00640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 0x7fa0601a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 -- 172.21.15.86:0/4195936599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa06019a750 msgr2=0x7fa06019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa06019a750 0x7fa06019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 -- 172.21.15.86:0/4195936599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 msgr2=0x7fa0601a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 0x7fa0601a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 -- 172.21.15.86:0/4195936599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06019b350 con 0x7fa0601072a0 2024-04-25T19:14:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.477+0000 7fa065000640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa0601a0bd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9c0 tx=0x7fa04c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.478+0000 7fa05e000640 1 -- 172.21.15.86:0/4195936599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00cd80 con 0x7fa0601072a0 2024-04-25T19:14:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.478+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0601a7430 con 0x7fa0601072a0 2024-04-25T19:14:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.478+0000 7fa05e000640 1 -- 172.21.15.86:0/4195936599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c0054d0 con 0x7fa0601072a0 2024-04-25T19:14:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.478+0000 7fa05e000640 1 -- 172.21.15.86:0/4195936599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010430 con 0x7fa0601072a0 2024-04-25T19:14:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.478+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a7890 con 0x7fa0601072a0 2024-04-25T19:14:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.481+0000 7fa06b276740 1 Processor -- start 2024-04-25T19:14:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 -- start start 2024-04-25T19:14:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e631c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e63c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa05e000640 1 -- 172.21.15.86:0/4195936599 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa04c004fd0 con 0x7fa0601072a0 2024-04-25T19:14:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e63edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c25e597900 con 0x55c25e63ca00 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c25e53def0 con 0x55c25e631820 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa06b276740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c25e5938c0 con 0x55c25e632140 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa065000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e63edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa066400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e631c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa065000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e63edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43798/0 (socket says 172.21.15.86:43798) 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa065000640 1 -- 172.21.15.86:0/391273840 learned_addr learned my addr 172.21.15.86:0/391273840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.482+0000 7fa065a00640 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e63c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa066400640 1 -- 172.21.15.86:0/391273840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 msgr2=0x55c25e63c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa066400640 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e63c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa066400640 1 -- 172.21.15.86:0/391273840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 msgr2=0x55c25e63edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa066400640 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e63edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa066400640 1 -- 172.21.15.86:0/391273840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c25e2aa210 con 0x55c25e631820 2024-04-25T19:14:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa066400640 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e631c00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa058009fd0 tx=0x7fa0580095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99924ad0c094fc7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa052a00640 1 -- 172.21.15.86:0/391273840 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058004b20 con 0x55c25e631820 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa052a00640 1 -- 172.21.15.86:0/391273840 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058004cc0 con 0x55c25e631820 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.483+0000 7fa052a00640 1 -- 172.21.15.86:0/391273840 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580054b0 con 0x55c25e631820 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa05e000640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa0480a3e40 0x7fa0480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa05e000640 1 -- 172.21.15.86:0/4195936599 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fa04c0d0c20 con 0x7fa0601072a0 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa065a00640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa0480a3e40 0x7fa0480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 -- 172.21.15.86:0/391273840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 msgr2=0x55c25e631c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e631c00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa058009fd0 tx=0x7fa0580095c0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 -- 172.21.15.86:0/391273840 shutdown_connections 2024-04-25T19:14:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e63edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e63c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 --2- 172.21.15.86:0/391273840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e631c00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 -- 172.21.15.86:0/391273840 >> 172.21.15.86:0/391273840 conn(0x55c25e62dd30 msgr2=0x55c25e62e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.484+0000 7fa06b276740 1 -- 172.21.15.86:0/391273840 shutdown_connections 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.485+0000 7fa06b276740 1 -- 172.21.15.86:0/391273840 wait complete. 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.485+0000 7fa065a00640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa0480a3e40 0x7fa0480a6300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa05402a930 tx=0x7fa05402c040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.485+0000 7fa06b276740 1 -- start start 2024-04-25T19:14:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.485+0000 7fa06b276740 1 Processor -- start 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e6399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e639f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e63a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c25e63f630 con 0x55c25e631820 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c25e551310 con 0x55c25e632140 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c25e63f7b0 con 0x55c25e63ca00 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa06b276740 10 client.0 _ll_register_callbacks cb 0x55c25e51fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa065000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e639f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa065000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e639f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46656/0 (socket says 172.21.15.86:46656) 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa065000640 1 -- 172.21.15.86:0/3067631519 learned_addr learned my addr 172.21.15.86:0/3067631519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa066400640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e63a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.486+0000 7fa065a00640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e6399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065000640 1 -- 172.21.15.86:0/3067631519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 msgr2=0x55c25e63a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065000640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e63a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065000640 1 -- 172.21.15.86:0/3067631519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 msgr2=0x55c25e6399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065000640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e6399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065000640 1 -- 172.21.15.86:0/3067631519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c25e376620 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065a00640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e6399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa065000640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e639f00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0994c0 tx=0x7fa04c0d4200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a48503479f35f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa047400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa047400640 1 -- 172.21.15.86:0/3067631519 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c09c8c0 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa058004cc0 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa047400640 1 -- 172.21.15.86:0/3067631519 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fa04c0ce520 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa047400640 1 -- 172.21.15.86:0/3067631519 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa04c0d4030 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa06b276740 10 client.26675 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.487+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c25e64b970 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa047400640 1 -- 172.21.15.86:0/3067631519 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058004cc0 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa047400640 1 -- 172.21.15.86:0/3067631519 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa04c0b8070 con 0x55c25e632140 2024-04-25T19:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa06b276740 10 client.26675 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa06b276740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa06b276740 20 client.26675 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa06b276740 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e64ce30 0x55c25e64f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa06b276740 4 client.26675 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c25e5ab8b0 con 0x55c25e64ce30 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.488+0000 7fa065a00640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e64ce30 0x55c25e64f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.489+0000 7fa047400640 10 client.26675 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.489+0000 7fa065a00640 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e64ce30 0x55c25e64f240 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d015bd38462001c4 server_cookie=1fcb95b65e768fe in_seq=0 out_seq=0 2024-04-25T19:14:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa047400640 1 -- 172.21.15.86:0/3067631519 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c25e5ab8b0 con 0x55c25e64ce30 2024-04-25T19:14:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa047400640 10 client.26675 handle_command_reply: tid=0 2024-04-25T19:14:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 client.26675 shutdown 2024-04-25T19:14:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 msgr2=0x55c25e639f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e639f00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0994c0 tx=0x7fa04c0d4200 comp rx=0 tx=0).stop 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 shutdown_connections 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e64ce30 0x55c25e64f240 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e63a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e639f00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 --2- 172.21.15.86:0/3067631519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e6399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa047400640 0 client.26675 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 >> 172.21.15.86:0/3067631519 conn(0x55c25e62dd30 msgr2=0x55c25e633a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 shutdown_connections 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 1 -- 172.21.15.86:0/3067631519 wait complete. 2024-04-25T19:14:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.490+0000 7fa06b276740 20 client.26675 trim_cache size 0 max 16384 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 Processor -- start 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 -- start start 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e62d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e6338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e633e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601afdf0 con 0x55c25e63ca00 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b1930 con 0x55c25e631820 2024-04-25T19:14:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.507+0000 7fa06b276740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b04c0 con 0x55c25e632140 2024-04-25T19:14:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e62d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa066400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e633e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e6338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e62d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46672/0 (socket says 172.21.15.86:46672) 2024-04-25T19:14:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 -- 172.21.15.86:0/429387299 learned_addr learned my addr 172.21.15.86:0/429387299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065000640 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e6338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:35640/0 (socket says 172.21.15.86:35640) 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 -- 172.21.15.86:0/429387299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 msgr2=0x55c25e6338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e6338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 -- 172.21.15.86:0/429387299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 msgr2=0x55c25e633e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e633e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065a00640 1 -- 172.21.15.86:0/429387299 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c25e637f10 con 0x55c25e631820 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa066400640 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e633e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.508+0000 7fa065000640 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e6338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.509+0000 7fa065a00640 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e62d610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa054000c00 tx=0x7fa05403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777f5ecab8550a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.509+0000 7fa052000640 1 -- 172.21.15.86:0/429387299 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054037040 con 0x55c25e631820 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.509+0000 7fa052000640 1 -- 172.21.15.86:0/429387299 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054054070 con 0x55c25e631820 2024-04-25T19:14:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.509+0000 7fa052000640 1 -- 172.21.15.86:0/429387299 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054003830 con 0x55c25e631820 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.509+0000 7fa06b276740 1 -- 172.21.15.86:0/429387299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 msgr2=0x55c25e62d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.509+0000 7fa06b276740 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e62d610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa054000c00 tx=0x7fa05403e000 comp rx=0 tx=0).stop 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 -- 172.21.15.86:0/429387299 shutdown_connections 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e63ca00 0x55c25e633e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e632140 0x55c25e6338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 --2- 172.21.15.86:0/429387299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e631820 0x55c25e62d610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 -- 172.21.15.86:0/429387299 >> 172.21.15.86:0/429387299 conn(0x55c25e62dd30 msgr2=0x55c25e6331d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 -- 172.21.15.86:0/429387299 shutdown_connections 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 -- 172.21.15.86:0/429387299 wait complete. 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 -- start start 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.510+0000 7fa06b276740 1 Processor -- start 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e642bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e6430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e6b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b1260 con 0x55c25e631820 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601b0b90 con 0x55c25e632140 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601addf0 con 0x55c25e63ca00 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa06b276740 10 client.0 _ll_register_callbacks cb 0x7fa04c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e642bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e642bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43828/0 (socket says 172.21.15.86:43828) 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 -- 172.21.15.86:0/1828146071 learned_addr learned my addr 172.21.15.86:0/1828146071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065000640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e6430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa066400640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e6b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 -- 172.21.15.86:0/1828146071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 msgr2=0x55c25e6b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e6b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 -- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 msgr2=0x55c25e6430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e6430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.511+0000 7fa065a00640 1 -- 172.21.15.86:0/1828146071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c25e6b2960 con 0x55c25e631820 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa065a00640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e642bb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa054005fe0 tx=0x7fa05402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a8304435548ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa047400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa047400640 1 -- 172.21.15.86:0/1828146071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054037040 con 0x55c25e631820 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa047400640 1 -- 172.21.15.86:0/1828146071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fa05404f160 con 0x55c25e631820 2024-04-25T19:14:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa054054070 con 0x55c25e631820 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa047400640 1 -- 172.21.15.86:0/1828146071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa054067050 con 0x55c25e631820 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa06b276740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.512+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c25e6b3c70 con 0x55c25e631820 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa047400640 1 -- 172.21.15.86:0/1828146071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054054070 con 0x55c25e631820 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa047400640 1 -- 172.21.15.86:0/1828146071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa054003ce0 con 0x55c25e631820 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa06b276740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa06b276740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa06b276740 20 client.16578 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa06b276740 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e6b43b0 0x55c25e6b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa06b276740 4 client.16578 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c25e5a5320 con 0x55c25e6b43b0 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.513+0000 7fa065000640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e6b43b0 0x55c25e6b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.514+0000 7fa065000640 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e6b43b0 0x55c25e6b67d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd8365d7a51eec4 server_cookie=dab7d93b4d516cf in_seq=0 out_seq=0 2024-04-25T19:14:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.514+0000 7fa047400640 10 client.16578 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa047400640 1 -- 172.21.15.86:0/1828146071 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c25e5a5320 con 0x55c25e6b43b0 2024-04-25T19:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa047400640 10 client.16578 handle_command_reply: tid=0 2024-04-25T19:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 client.16578 shutdown 2024-04-25T19:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 msgr2=0x55c25e642bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e642bb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa054005fe0 tx=0x7fa05402a340 comp rx=0 tx=0).stop 2024-04-25T19:14:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 shutdown_connections 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55c25e6b43b0 0x55c25e6b67d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c25e63ca00 0x55c25e6b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c25e632140 0x55c25e6430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 --2- 172.21.15.86:0/1828146071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c25e631820 0x55c25e642bb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa047400640 0 client.16578 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.524+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 >> 172.21.15.86:0/1828146071 conn(0x55c25e62dd30 msgr2=0x55c25e633a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 shutdown_connections 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa06b276740 1 -- 172.21.15.86:0/1828146071 wait complete. 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa06b276740 20 client.16578 trim_cache size 0 max 16384 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:14:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa0480a3e40 msgr2=0x7fa0480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa068400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa0480a3e40 0x7fa0480a6300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa05402a930 tx=0x7fa05402c040 comp rx=0 tx=0).stop 2024-04-25T19:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 msgr2=0x7fa0601a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.525+0000 7fa068400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa0601a0bd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9c0 tx=0x7fa04c00ee80 comp rx=0 tx=0).stop 2024-04-25T19:14:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 shutdown_connections 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa0480a3e40 0x7fa0480a6300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa06019a750 0x7fa06019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0601072a0 0x7fa0601a0bd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 --2- 172.21.15.86:0/4195936599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0601021d0 0x7fa0601a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 >> 172.21.15.86:0/4195936599 conn(0x7fa0600793e0 msgr2=0x7fa0600ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 shutdown_connections 2024-04-25T19:14:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:39.526+0000 7fa068400640 1 -- 172.21.15.86:0/4195936599 wait complete. 2024-04-25T19:14:40.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 -- 172.21.15.86:0/2801262949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc06b110 msgr2=0x7fc1fc06b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 --2- 172.21.15.86:0/2801262949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc06b110 0x7fc1fc06b570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009990 tx=0x7fc1ec009ca0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 -- 172.21.15.86:0/2801262949 shutdown_connections 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 --2- 172.21.15.86:0/2801262949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 --2- 172.21.15.86:0/2801262949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc108a50 0x7fc1fc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 --2- 172.21.15.86:0/2801262949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc06b110 0x7fc1fc06b570 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 -- 172.21.15.86:0/2801262949 >> 172.21.15.86:0/2801262949 conn(0x7fc1fc0788f0 msgr2=0x7fc1fc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 -- 172.21.15.86:0/2801262949 shutdown_connections 2024-04-25T19:14:40.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.376+0000 7fc204800700 1 -- 172.21.15.86:0/2801262949 wait complete. 2024-04-25T19:14:40.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 Processor -- start 2024-04-25T19:14:40.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 -- start start 2024-04-25T19:14:40.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06b110 0x7fc1fc101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc108a50 0x7fc1fc102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc112bf0 con 0x7fc1fc06bab0 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc112a70 con 0x7fc1fc108a50 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.377+0000 7fc204800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc112d70 con 0x7fc1fc06b110 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52362/0 (socket says 172.21.15.86:52362) 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 -- 172.21.15.86:0/4025298373 learned_addr learned my addr 172.21.15.86:0/4025298373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201e00700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06b110 0x7fc1fc101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc202800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc108a50 0x7fc1fc102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 -- 172.21.15.86:0/4025298373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06b110 msgr2=0x7fc1fc101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06b110 0x7fc1fc101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 -- 172.21.15.86:0/4025298373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc108a50 msgr2=0x7fc1fc102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc108a50 0x7fc1fc102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201400700 1 -- 172.21.15.86:0/4025298373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc1abad0 con 0x7fc1fc06bab0 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.378+0000 7fc201e00700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06b110 0x7fc1fc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.379+0000 7fc201400700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc102330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0057f0 tx=0x7fc1ec00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:40.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.379+0000 7fc1f6a00700 1 -- 172.21.15.86:0/4025298373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec026070 con 0x7fc1fc06bab0 2024-04-25T19:14:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.379+0000 7fc1f6a00700 1 -- 172.21.15.86:0/4025298373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ec021410 con 0x7fc1fc06bab0 2024-04-25T19:14:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.379+0000 7fc1f6a00700 1 -- 172.21.15.86:0/4025298373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec017650 con 0x7fc1fc06bab0 2024-04-25T19:14:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.379+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc1abd00 con 0x7fc1fc06bab0 2024-04-25T19:14:40.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.379+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1ac190 con 0x7fc1fc06bab0 2024-04-25T19:14:40.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.382+0000 7fc1f6a00700 1 -- 172.21.15.86:0/4025298373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1ec029070 con 0x7fc1fc06bab0 2024-04-25T19:14:40.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.384+0000 7fc20f0d5b80 1 Processor -- start 2024-04-25T19:14:40.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.384+0000 7fc20f0d5b80 1 -- start start 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc1f6a00700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e40a3db0 0x7fc1e40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2c660 0x55f0dae36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc1f6a00700 1 -- 172.21.15.86:0/4025298373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fc1ec0d1510 con 0x7fc1fc06bab0 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae374d0 0x55f0dae398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0dad7ef90 con 0x55f0dae374d0 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f0dad25830 con 0x55f0dae2c660 2024-04-25T19:14:40.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f0dad7af50 con 0x55f0dae2bd40 2024-04-25T19:14:40.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc202800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e40a3db0 0x7fc1e40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc201e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc201e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae2c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:55526/0 (socket says 172.21.15.86:55526) 2024-04-25T19:14:40.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.385+0000 7fc201e00700 1 -- 172.21.15.86:0/717044032 learned_addr learned my addr 172.21.15.86:0/717044032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:40.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc202800700 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae374d0 0x55f0dae398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201400700 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2c660 0x55f0dae36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201e00700 1 -- 172.21.15.86:0/717044032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2c660 msgr2=0x55f0dae36f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201e00700 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2c660 0x55f0dae36f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201e00700 1 -- 172.21.15.86:0/717044032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae374d0 msgr2=0x55f0dae398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201e00700 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae374d0 0x55f0dae398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201e00700 1 -- 172.21.15.86:0/717044032 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0dad1b780 con 0x55f0dae2bd40 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc202800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e40a3db0 0x7fc1e40a6270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800f870 tx=0x7fc1f800fc30 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.386+0000 7fc201e00700 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae2c120 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00115c0 tx=0x7fc1f0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd23c99e7c88391c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.387+0000 7fc1eb400700 1 -- 172.21.15.86:0/717044032 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001e070 con 0x55f0dae2bd40 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.387+0000 7fc1eb400700 1 -- 172.21.15.86:0/717044032 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f0019ae0 con 0x55f0dae2bd40 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.387+0000 7fc1eb400700 1 -- 172.21.15.86:0/717044032 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000eb60 con 0x55f0dae2bd40 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.387+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/717044032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 msgr2=0x55f0dae2c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.387+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae2c120 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00115c0 tx=0x7fc1f0009f90 comp rx=0 tx=0).stop 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.387+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/717044032 shutdown_connections 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae374d0 0x55f0dae398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae2c120 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/717044032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2c660 0x55f0dae36f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/717044032 >> 172.21.15.86:0/717044032 conn(0x55f0dad5b050 msgr2=0x55f0dae28250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:40.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/717044032 shutdown_connections 2024-04-25T19:14:40.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/717044032 wait complete. 2024-04-25T19:14:40.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 -- start start 2024-04-25T19:14:40.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.388+0000 7fc20f0d5b80 1 Processor -- start 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 0x55f0dae36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae374d0 0x55f0dae343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0dae39fc0 con 0x55f0dae2c660 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f0dad38960 con 0x55f0dae2bd40 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f0dae3a140 con 0x55f0dae374d0 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc20f0d5b80 10 client.0 _ll_register_callbacks cb 0x55f0dad17820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc201e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 0x55f0dae36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae33eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52372/0 (socket says 172.21.15.86:52372) 2024-04-25T19:14:40.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc201400700 1 -- 172.21.15.86:0/678980798 learned_addr learned my addr 172.21.15.86:0/678980798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc201e00700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 0x55f0dae36bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:41520/0 (socket says 172.21.15.86:41520) 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.389+0000 7fc202800700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae374d0 0x55f0dae343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc201e00700 1 -- 172.21.15.86:0/678980798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae374d0 msgr2=0x55f0dae343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc201e00700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae374d0 0x55f0dae343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc201e00700 1 -- 172.21.15.86:0/678980798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 msgr2=0x55f0dae33eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc201e00700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae33eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc201e00700 1 -- 172.21.15.86:0/678980798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0daccc9d0 con 0x55f0dae2bd40 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc201e00700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 0x55f0dae36bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009230 tx=0x7fc1f000e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef092930c8d411f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.390+0000 7fc1dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc1dfe00700 1 -- 172.21.15.86:0/678980798 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001e050 con 0x55f0dae2bd40 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f0019ae0 con 0x55f0dae2bd40 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc1dfe00700 1 -- 172.21.15.86:0/678980798 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fc1f00198c0 con 0x55f0dae2bd40 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc1dfe00700 1 -- 172.21.15.86:0/678980798 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1f0033050 con 0x55f0dae2bd40 2024-04-25T19:14:40.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc20f0d5b80 10 client.26693 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:40.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0dad15bf0 con 0x55f0dae2bd40 2024-04-25T19:14:40.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.391+0000 7fc1dfe00700 1 -- 172.21.15.86:0/678980798 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f0019ae0 con 0x55f0dae2bd40 2024-04-25T19:14:40.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc1dfe00700 1 -- 172.21.15.86:0/678980798 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc1f0003680 con 0x55f0dae2bd40 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc20f0d5b80 10 client.26693 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc20f0d5b80 10 client.26693 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc20f0d5b80 20 client.26693 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0dae46c20 0x55f0dae49010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc20f0d5b80 4 client.26693 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0dad8c780 con 0x55f0dae46c20 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.392+0000 7fc201400700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0dae46c20 0x55f0dae49010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.393+0000 7fc201400700 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0dae46c20 0x55f0dae49010 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7910b16689d1533 server_cookie=a876fb1894e4d893 in_seq=0 out_seq=0 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.393+0000 7fc1dfe00700 10 client.26693 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc1dfe00700 1 -- 172.21.15.86:0/678980798 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0dad8c780 con 0x55f0dae46c20 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc1dfe00700 10 client.26693 handle_command_reply: tid=0 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 client.26693 shutdown 2024-04-25T19:14:40.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 msgr2=0x55f0dae36bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 0x55f0dae36bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009230 tx=0x7fc1f000e5e0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 shutdown_connections 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0dae46c20 0x55f0dae49010 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae374d0 0x55f0dae343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae2bd40 0x55f0dae36bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/678980798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 >> 172.21.15.86:0/678980798 conn(0x55f0dad5b050 msgr2=0x55f0dae27e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc1dfe00700 0 client.26693 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 shutdown_connections 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/678980798 wait complete. 2024-04-25T19:14:40.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.394+0000 7fc20f0d5b80 20 client.26693 trim_cache size 0 max 16384 2024-04-25T19:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.415+0000 7fc20f0d5b80 1 Processor -- start 2024-04-25T19:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.415+0000 7fc20f0d5b80 1 -- start start 2024-04-25T19:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.415+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.415+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae42ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.415+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae27b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.415+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1b3a80 con 0x55f0dae2c660 2024-04-25T19:14:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1b55c0 con 0x55f0dae374d0 2024-04-25T19:14:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1b4150 con 0x55f0dae2bd40 2024-04-25T19:14:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc201e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae42ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc201e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae33030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:55534/0 (socket says 172.21.15.86:55534) 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc201e00700 1 -- 172.21.15.86:0/795030242 learned_addr learned my addr 172.21.15.86:0/795030242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.416+0000 7fc202800700 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae27b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.417+0000 7fc201e00700 1 -- 172.21.15.86:0/795030242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 msgr2=0x55f0dae27b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.417+0000 7fc201e00700 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae27b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.417+0000 7fc201e00700 1 -- 172.21.15.86:0/795030242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 msgr2=0x55f0dae42ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.417+0000 7fc201e00700 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae42ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.417+0000 7fc201e00700 1 -- 172.21.15.86:0/795030242 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0dae2d500 con 0x55f0dae2bd40 2024-04-25T19:14:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.417+0000 7fc201e00700 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae33030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009f90 tx=0x7fc1f0009fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7108964c118dd1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc1eaa00700 1 -- 172.21.15.86:0/795030242 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0015020 con 0x55f0dae2bd40 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc1eaa00700 1 -- 172.21.15.86:0/795030242 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f002a070 con 0x55f0dae2bd40 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc1eaa00700 1 -- 172.21.15.86:0/795030242 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001e040 con 0x55f0dae2bd40 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/795030242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 msgr2=0x55f0dae33030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae33030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009f90 tx=0x7fc1f0009fc0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/795030242 shutdown_connections 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae27b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae33030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/795030242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae42ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/795030242 >> 172.21.15.86:0/795030242 conn(0x55f0dad5b050 msgr2=0x55f0dae27400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/795030242 shutdown_connections 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.418+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/795030242 wait complete. 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.419+0000 7fc20f0d5b80 1 -- start start 2024-04-25T19:14:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.419+0000 7fc20f0d5b80 1 Processor -- start 2024-04-25T19:14:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae3bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae3c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae3c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1b4ef0 con 0x55f0dae2c660 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1b4820 con 0x55f0dae374d0 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1b1a40 con 0x55f0dae2bd40 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc20f0d5b80 10 client.0 _ll_register_callbacks cb 0x55f0dad17820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae3c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae3c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52392/0 (socket says 172.21.15.86:52392) 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 -- 172.21.15.86:0/3018502848 learned_addr learned my addr 172.21.15.86:0/3018502848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc202800700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae3c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201e00700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae3bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 -- 172.21.15.86:0/3018502848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 msgr2=0x55f0dae3bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae3bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 -- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 msgr2=0x55f0dae3c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae3c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.420+0000 7fc201400700 1 -- 172.21.15.86:0/3018502848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0daead000 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc201400700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae3c300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec09d200 tx=0x7fc1ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1d82cd047922f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc1dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc1dfe00700 1 -- 172.21.15.86:0/3018502848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec09d500 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc1dfe00700 1 -- 172.21.15.86:0/3018502848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fc1ec0cee10 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f002a070 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc1dfe00700 1 -- 172.21.15.86:0/3018502848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1ec0bb050 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc20f0d5b80 10 client.16596 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0daeae350 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.421+0000 7fc1dfe00700 1 -- 172.21.15.86:0/3018502848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f002a070 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc1dfe00700 1 -- 172.21.15.86:0/3018502848 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc1ec0218c0 con 0x55f0dae2c660 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc20f0d5b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc20f0d5b80 10 client.16596 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:14:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc20f0d5b80 20 client.16596 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0daeae9f0 0x55f0daeb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc20f0d5b80 4 client.16596 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4ddc9463", "release": true, "await": true}] 2024-04-25T19:14:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4ddc9463", "release": true, "await": true}) -- 0x55f0dad86820 con 0x55f0daeae9f0 2024-04-25T19:14:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.422+0000 7fc201e00700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0daeae9f0 0x55f0daeb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.423+0000 7fc201e00700 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0daeae9f0 0x55f0daeb0e10 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f5e703844d4151 server_cookie=6e3c7b13d078891a in_seq=0 out_seq=0 2024-04-25T19:14:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.423+0000 7fc1dfe00700 10 client.16596 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc1dfe00700 1 -- 172.21.15.86:0/3018502848 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f0dad86820 con 0x55f0daeae9f0 2024-04-25T19:14:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc1dfe00700 10 client.16596 handle_command_reply: tid=0 2024-04-25T19:14:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 client.16596 shutdown 2024-04-25T19:14:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 msgr2=0x55f0dae3c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae3c300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec09d200 tx=0x7fc1ec009310 comp rx=0 tx=0).stop 2024-04-25T19:14:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 shutdown_connections 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55f0daeae9f0 0x55f0daeb0e10 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0dae374d0 0x55f0dae3c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0dae2bd40 0x55f0dae3bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 --2- 172.21.15.86:0/3018502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0dae2c660 0x55f0dae3c300 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 >> 172.21.15.86:0/3018502848 conn(0x55f0dad5b050 msgr2=0x55f0dae27400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc1dfe00700 0 client.16596 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 shutdown_connections 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.426+0000 7fc20f0d5b80 1 -- 172.21.15.86:0/3018502848 wait complete. 2024-04-25T19:14:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc20f0d5b80 20 client.16596 trim_cache size 0 max 16384 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 36, 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "4ddc9463": { 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 36, 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:14:40.431 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 89.2, 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:14:40.432 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e40a3db0 msgr2=0x7fc1e40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e40a3db0 0x7fc1e40a6270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800f870 tx=0x7fc1f800fc30 comp rx=0 tx=0).stop 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 msgr2=0x7fc1fc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc102330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0057f0 tx=0x7fc1ec00bfd0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 shutdown_connections 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc1fc108a50 0x7fc1fc102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc1e40a3db0 0x7fc1e40a6270 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1fc06b110 0x7fc1fc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 --2- 172.21.15.86:0/4025298373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1fc06bab0 0x7fc1fc102330 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.427+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 >> 172.21.15.86:0/4025298373 conn(0x7fc1fc0788f0 msgr2=0x7fc1fc0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.428+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 shutdown_connections 2024-04-25T19:14:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:14:40.428+0000 7fc204800700 1 -- 172.21.15.86:0/4025298373 wait complete. 2024-04-25T19:14:41.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:40 smithi086 ceph-mon[27783]: pgmap v366: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-25T19:14:41.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:40 smithi183 ceph-mon[32344]: pgmap v366: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-25T19:14:41.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:40 smithi194 ceph-mon[32740]: pgmap v366: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-25T19:14:41.376 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4ddc9463', seconds elapsed: 44.4 2024-04-25T19:14:43.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:42 smithi086 ceph-mon[27783]: pgmap v367: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T19:14:43.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:42 smithi183 ceph-mon[32344]: pgmap v367: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T19:14:43.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:42 smithi194 ceph-mon[32740]: pgmap v367: 97 pgs: 97 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T19:14:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:44 smithi086 ceph-mon[27783]: pgmap v368: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T19:14:45.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:44 smithi183 ceph-mon[32344]: pgmap v368: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T19:14:45.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:44 smithi194 ceph-mon[32740]: pgmap v368: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T19:14:47.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:46 smithi086 ceph-mon[27783]: pgmap v369: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T19:14:47.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:46 smithi183 ceph-mon[32344]: pgmap v369: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T19:14:47.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:46 smithi194 ceph-mon[32740]: pgmap v369: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T19:14:48.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:48 smithi086 ceph-mon[27783]: pgmap v370: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T19:14:49.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:48 smithi183 ceph-mon[32344]: pgmap v370: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T19:14:49.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:48 smithi194 ceph-mon[32740]: pgmap v370: 97 pgs: 97 active+clean; 3.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T19:14:49.535 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.535 INFO:tasks.workunit.client.0.smithi086.stdout:313/600: open fd = 3 2024-04-25T19:14:49.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.535 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.535 INFO:tasks.workunit.client.0.smithi086.stdout:314/600: open fd = 3 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:315/600: open fd = 3 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:316/600: open fd = 3 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:317/600: open fd = 3 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:318/600: open fd = 3 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:319/600: open fd = 3 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.538 INFO:tasks.workunit.client.0.smithi086.stdout:320/600: open fd = 3 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:321/600: open fd = 3 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:322/600: open fd = 3 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:323/600: open fd = 3 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:324/600: open fd = 3 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:325/600: open fd = 3 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:326/600: open fd = 3 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:327/600: open fd = 3 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:328/600: open fd = 3 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:329/600: open fd = 3 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:330/600: open fd = 3 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:331/600: open fd = 3 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.541 INFO:tasks.workunit.client.0.smithi086.stdout:332/600: open fd = 3 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:333/600: open fd = 3 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:334/600: open fd = 3 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:335/600: open fd = 3 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:336/600: open fd = 3 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:337/600: open fd = 3 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:338/600: open fd = 3 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:339/600: open fd = 3 2024-04-25T19:14:49.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:340/600: open fd = 3 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:341/600: open fd = 3 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:342/600: open fd = 3 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.544 INFO:tasks.workunit.client.0.smithi086.stdout:343/600: open fd = 3 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:344/600: open fd = 3 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:345/600: open fd = 3 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:346/600: open fd = 3 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:347/600: open fd = 3 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:348/600: open fd = 3 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:349/600: open fd = 3 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:350/600: open fd = 3 2024-04-25T19:14:49.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:351/600: open fd = 3 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:352/600: open fd = 3 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:353/600: open fd = 3 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.547 INFO:tasks.workunit.client.0.smithi086.stdout:354/600: open fd = 3 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:355/600: open fd = 3 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:356/600: open fd = 3 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:357/600: open fd = 3 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:358/600: open fd = 3 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:359/600: open fd = 3 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:360/600: open fd = 3 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:361/600: open fd = 3 2024-04-25T19:14:49.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:362/600: open fd = 3 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:363/600: open fd = 3 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:364/600: open fd = 3 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.550 INFO:tasks.workunit.client.0.smithi086.stdout:365/600: open fd = 3 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:366/600: open fd = 3 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:367/600: open fd = 3 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:368/600: open fd = 3 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:369/600: open fd = 3 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:370/600: open fd = 3 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:371/600: open fd = 3 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:372/600: open fd = 3 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:373/600: open fd = 3 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:374/600: open fd = 3 2024-04-25T19:14:49.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:14:49.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:14:51.034 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:50 smithi194 ceph-mon[32740]: pgmap v371: 97 pgs: 97 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-25T19:14:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:50 smithi086 ceph-mon[27783]: pgmap v371: 97 pgs: 97 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-25T19:14:51.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:50 smithi183 ceph-mon[32344]: pgmap v371: 97 pgs: 97 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-25T19:14:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 -- 172.21.15.86:0/3091559863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c109480 msgr2=0x7fcc3c111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 --2- 172.21.15.86:0/3091559863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c109480 0x7fcc3c111050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009900 tx=0x7fcc2c02a410 comp rx=0 tx=0).stop 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 -- 172.21.15.86:0/3091559863 shutdown_connections 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 --2- 172.21.15.86:0/3091559863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c109480 0x7fcc3c111050 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 --2- 172.21.15.86:0/3091559863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc3c104d60 0x7fcc3c108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 --2- 172.21.15.86:0/3091559863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 -- 172.21.15.86:0/3091559863 >> 172.21.15.86:0/3091559863 conn(0x7fcc3c0ffc50 msgr2=0x7fcc3c102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 -- 172.21.15.86:0/3091559863 shutdown_connections 2024-04-25T19:14:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.675+0000 7fcc42200640 1 -- 172.21.15.86:0/3091559863 wait complete. 2024-04-25T19:14:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 Processor -- start 2024-04-25T19:14:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 -- start start 2024-04-25T19:14:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc3c109480 0x7fcc3c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc3c19a6a0 0x7fcc3c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c114420 con 0x7fcc3c1043b0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1142a0 con 0x7fcc3c19a6a0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc42200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1145a0 con 0x7fcc3c109480 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.676+0000 7fcc3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50760/0 (socket says 172.21.15.86:50760) 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 -- 172.21.15.86:0/2668713263 learned_addr learned my addr 172.21.15.86:0/2668713263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc3c109480 0x7fcc3c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3be00640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc3c19a6a0 0x7fcc3c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 -- 172.21.15.86:0/2668713263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc3c109480 msgr2=0x7fcc3c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc3c109480 0x7fcc3c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 -- 172.21.15.86:0/2668713263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc3c19a6a0 msgr2=0x7fcc3c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc3c19a6a0 0x7fcc3c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 -- 172.21.15.86:0/2668713263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c19b2a0 con 0x7fcc3c1043b0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc3b400640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c1a05e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400ebc0 tx=0x7fcc3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.677+0000 7fcc33e00640 1 -- 172.21.15.86:0/2668713263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34010630 con 0x7fcc3c1043b0 2024-04-25T19:14:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.678+0000 7fcc33e00640 1 -- 172.21.15.86:0/2668713263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc34010c80 con 0x7fcc3c1043b0 2024-04-25T19:14:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.678+0000 7fcc33e00640 1 -- 172.21.15.86:0/2668713263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34018b60 con 0x7fcc3c1043b0 2024-04-25T19:14:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.678+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c1a7380 con 0x7fcc3c1043b0 2024-04-25T19:14:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.678+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c1a7730 con 0x7fcc3c1043b0 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 Processor -- start 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc33e00640 1 -- 172.21.15.86:0/2668713263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc34018d00 con 0x7fcc3c1043b0 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 -- start start 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f261c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f26c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f26edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55897f1c7900 con 0x55897f26ca00 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55897f16def0 con 0x55897f262140 2024-04-25T19:14:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.681+0000 7fcc451da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55897f1c38c0 con 0x55897f261820 2024-04-25T19:14:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f26edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f26c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f261c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f26edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50768/0 (socket says 172.21.15.86:50768) 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f261c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:40858/0 (socket says 172.21.15.86:40858) 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3aa00640 1 -- 172.21.15.86:0/1113099549 learned_addr learned my addr 172.21.15.86:0/1113099549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 -- 172.21.15.86:0/1113099549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 msgr2=0x55897f26c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f26c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 -- 172.21.15.86:0/1113099549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 msgr2=0x55897f26edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f26edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 -- 172.21.15.86:0/1113099549 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897eeda210 con 0x55897f261820 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc33e00640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcc200a3e40 0x7fcc200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.682+0000 7fcc3be00640 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f261c00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c02a930 tx=0x7fcc2c005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd086ca4742f519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.683+0000 7fcc33e00640 1 -- 172.21.15.86:0/2668713263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fcc340d0e20 con 0x7fcc3c1043b0 2024-04-25T19:14:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.683+0000 7fcc27e00640 1 -- 172.21.15.86:0/1113099549 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c040070 con 0x55897f261820 2024-04-25T19:14:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.683+0000 7fcc3b400640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcc200a3e40 0x7fcc200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.683+0000 7fcc27e00640 1 -- 172.21.15.86:0/1113099549 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2c0044c0 con 0x55897f261820 2024-04-25T19:14:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.683+0000 7fcc27e00640 1 -- 172.21.15.86:0/1113099549 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c035040 con 0x55897f261820 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 -- 172.21.15.86:0/1113099549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 msgr2=0x55897f261c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc3b400640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcc200a3e40 0x7fcc200a6300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55897f262dc0 tx=0x7fcc340d48d0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f261c00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c02a930 tx=0x7fcc2c005be0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 -- 172.21.15.86:0/1113099549 shutdown_connections 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f26edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f26c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 --2- 172.21.15.86:0/1113099549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f261c00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 -- 172.21.15.86:0/1113099549 >> 172.21.15.86:0/1113099549 conn(0x55897f25dd30 msgr2=0x55897f25e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 -- 172.21.15.86:0/1113099549 shutdown_connections 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 -- 172.21.15.86:0/1113099549 wait complete. 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.684+0000 7fcc451da740 1 -- start start 2024-04-25T19:14:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 Processor -- start 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 0x55897f269a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f262140 0x55897f269f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f26a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55897f26f5d0 con 0x55897f261820 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55897f181310 con 0x55897f26ca00 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55897f26f750 con 0x55897f262140 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc451da740 10 client.0 _ll_register_callbacks cb 0x55897f14fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 0x55897f269a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 0x55897f269a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50776/0 (socket says 172.21.15.86:50776) 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc3be00640 1 -- 172.21.15.86:0/3868129060 learned_addr learned my addr 172.21.15.86:0/3868129060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.685+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f262140 0x55897f269f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f26a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 -- 172.21.15.86:0/3868129060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f262140 msgr2=0x55897f269f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f262140 0x55897f269f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 -- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 msgr2=0x55897f26a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f26a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 -- 172.21.15.86:0/3868129060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897efa6620 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 0x55897f269a10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009850 tx=0x7fcc2c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe060dfc06ebd58b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc24c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc24c00640 1 -- 172.21.15.86:0/3868129060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c048070 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc24c00640 1 -- 172.21.15.86:0/3868129060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fcc2c003a00 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.686+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55897f1db8b0 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.687+0000 7fcc24c00640 1 -- 172.21.15.86:0/3868129060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc2c040070 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.687+0000 7fcc451da740 10 client.16608 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.687+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55897f27b8c0 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.687+0000 7fcc24c00640 1 -- 172.21.15.86:0/3868129060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55897f1db8b0 con 0x55897f261820 2024-04-25T19:14:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.687+0000 7fcc24c00640 1 -- 172.21.15.86:0/3868129060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcc2c0314a0 con 0x55897f261820 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc451da740 10 client.16608 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc451da740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc451da740 20 client.16608 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc451da740 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f27cc10 0x55897f27f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc451da740 4 client.16608 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897f1d5320 con 0x55897f27cc10 2024-04-25T19:14:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f27cc10 0x55897f27f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f27cc10 0x55897f27f000 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947f19f9f70ad421 server_cookie=54587fb5aaf02814 in_seq=0 out_seq=0 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.688+0000 7fcc24c00640 10 client.16608 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.689+0000 7fcc24c00640 1 -- 172.21.15.86:0/3868129060 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55897f1d5320 con 0x55897f27cc10 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.689+0000 7fcc24c00640 10 client.16608 handle_command_reply: tid=0 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.689+0000 7fcc451da740 1 client.16608 shutdown 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.689+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 msgr2=0x55897f269a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.689+0000 7fcc451da740 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 0x55897f269a10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009850 tx=0x7fcc2c004b50 comp rx=0 tx=0).stop 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 shutdown_connections 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f27cc10 0x55897f27f000 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f26a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f262140 0x55897f269f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc24c00640 0 client.16608 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 --2- 172.21.15.86:0/3868129060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f261820 0x55897f269a10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 >> 172.21.15.86:0/3868129060 conn(0x55897f25dd30 msgr2=0x55897f263a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 shutdown_connections 2024-04-25T19:14:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 1 -- 172.21.15.86:0/3868129060 wait complete. 2024-04-25T19:14:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.690+0000 7fcc451da740 20 client.16608 trim_cache size 0 max 16384 2024-04-25T19:14:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 Processor -- start 2024-04-25T19:14:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 -- start start 2024-04-25T19:14:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f278490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f2845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f284ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1afea0 con 0x55897f26ca00 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b19e0 con 0x55897f262140 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.706+0000 7fcc451da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b0570 con 0x55897f261820 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f2845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f278490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f2845a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37128/0 (socket says 172.21.15.86:37128) 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 -- 172.21.15.86:0/3389395351 learned_addr learned my addr 172.21.15.86:0/3389395351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f284ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 -- 172.21.15.86:0/3389395351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 msgr2=0x55897f278490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f278490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 -- 172.21.15.86:0/3389395351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 msgr2=0x55897f284ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f284ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3aa00640 1 -- 172.21.15.86:0/3389395351 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897f267f60 con 0x55897f262140 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.707+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f284ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f2845a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55897f25f150 tx=0x7fcc28005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b984f776ab17d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc27400640 1 -- 172.21.15.86:0/3389395351 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28003d20 con 0x55897f262140 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc27400640 1 -- 172.21.15.86:0/3389395351 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2801f500 con 0x55897f262140 2024-04-25T19:14:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc27400640 1 -- 172.21.15.86:0/3389395351 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28010900 con 0x55897f262140 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 -- 172.21.15.86:0/3389395351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 msgr2=0x55897f2845a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f2845a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55897f25f150 tx=0x7fcc28005f90 comp rx=0 tx=0).stop 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 -- 172.21.15.86:0/3389395351 shutdown_connections 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f26ca00 0x55897f284ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f262140 0x55897f2845a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 --2- 172.21.15.86:0/3389395351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f278490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.708+0000 7fcc451da740 1 -- 172.21.15.86:0/3389395351 >> 172.21.15.86:0/3389395351 conn(0x55897f25dd30 msgr2=0x55897f268a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.709+0000 7fcc451da740 1 -- 172.21.15.86:0/3389395351 shutdown_connections 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.709+0000 7fcc451da740 1 -- 172.21.15.86:0/3389395351 wait complete. 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.709+0000 7fcc451da740 1 -- start start 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.709+0000 7fcc451da740 1 Processor -- start 2024-04-25T19:14:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f274d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 0x55897f2720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f2725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b1310 con 0x55897f262140 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b0c40 con 0x55897f26ca00 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1adea0 con 0x55897f261820 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc451da740 10 client.0 _ll_register_callbacks cb 0x7fcc2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 0x55897f2720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 0x55897f2720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50792/0 (socket says 172.21.15.86:50792) 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 -- 172.21.15.86:0/692020708 learned_addr learned my addr 172.21.15.86:0/692020708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3be00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f274d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3be00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f2725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 -- 172.21.15.86:0/692020708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 msgr2=0x55897f274d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f274d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 -- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 msgr2=0x55897f2725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:14:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f2725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3aa00640 1 -- 172.21.15.86:0/692020708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897f272e10 con 0x55897f262140 2024-04-25T19:14:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.710+0000 7fcc3be00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f2725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:14:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 0x55897f2720a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55897f269170 tx=0x7fcc28009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58511703063a1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:14:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc24c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:14:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc24c00640 1 -- 172.21.15.86:0/692020708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801f250 con 0x55897f262140 2024-04-25T19:14:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc24c00640 1 -- 172.21.15.86:0/692020708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fcc28003b20 con 0x55897f262140 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2801f500 con 0x55897f262140 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc24c00640 1 -- 172.21.15.86:0/692020708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc28003d20 con 0x55897f262140 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc451da740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.711+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55897f2e2e00 con 0x55897f262140 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc24c00640 1 -- 172.21.15.86:0/692020708 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2801f500 con 0x55897f262140 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc24c00640 1 -- 172.21.15.86:0/692020708 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcc280364e0 con 0x55897f262140 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc451da740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc451da740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc451da740 20 client.16614 populate_metadata read hostname 'smithi086' 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc451da740 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f2e3470 0x55897f2e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc451da740 4 client.16614 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55897f1cf190 con 0x55897f2e3470 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.712+0000 7fcc3be00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f2e3470 0x55897f2e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.713+0000 7fcc3be00640 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f2e3470 0x55897f2e5890 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af55e1e3a17e228 server_cookie=354c4e5474249be5 in_seq=0 out_seq=0 2024-04-25T19:14:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.713+0000 7fcc24c00640 10 client.16614 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc24c00640 1 -- 172.21.15.86:0/692020708 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55897f1cf190 con 0x55897f2e3470 2024-04-25T19:14:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc24c00640 10 client.16614 handle_command_reply: tid=0 2024-04-25T19:14:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 client.16614 shutdown 2024-04-25T19:14:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 msgr2=0x55897f2720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 0x55897f2720a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55897f269170 tx=0x7fcc28009fd0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 shutdown_connections 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55897f2e3470 0x55897f2e5890 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55897f26ca00 0x55897f2725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55897f262140 0x55897f2720a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 --2- 172.21.15.86:0/692020708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55897f261820 0x55897f274d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc24c00640 0 client.16614 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 >> 172.21.15.86:0/692020708 conn(0x55897f25dd30 msgr2=0x55897f25c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 shutdown_connections 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 1 -- 172.21.15.86:0/692020708 wait complete. 2024-04-25T19:14:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.727+0000 7fcc451da740 20 client.16614 trim_cache size 0 max 16384 2024-04-25T19:14:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:14:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:14:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:14:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcc200a3e40 msgr2=0x7fcc200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcc200a3e40 0x7fcc200a6300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55897f262dc0 tx=0x7fcc340d48d0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 msgr2=0x7fcc3c1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c1a05e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400ebc0 tx=0x7fcc3400c6a0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 shutdown_connections 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fcc200a3e40 0x7fcc200a6300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc3c19a6a0 0x7fcc3c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc3c109480 0x7fcc3c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 --2- 172.21.15.86:0/2668713263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc3c1043b0 0x7fcc3c1a05e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.728+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 >> 172.21.15.86:0/2668713263 conn(0x7fcc3c0ffc50 msgr2=0x7fcc3c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.729+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 shutdown_connections 2024-04-25T19:14:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:14:52.729+0000 7fcc42200640 1 -- 172.21.15.86:0/2668713263 wait complete. 2024-04-25T19:14:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:52 smithi086 ceph-mon[27783]: pgmap v372: 97 pgs: 97 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-04-25T19:14:53.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:52 smithi183 ceph-mon[32344]: pgmap v372: 97 pgs: 97 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-04-25T19:14:53.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:52 smithi194 ceph-mon[32740]: pgmap v372: 97 pgs: 97 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-04-25T19:14:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:54 smithi086 ceph-mon[27783]: pgmap v373: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-25T19:14:55.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:54 smithi183 ceph-mon[32344]: pgmap v373: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-25T19:14:55.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:54 smithi194 ceph-mon[32740]: pgmap v373: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-25T19:14:57.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:56 smithi086 ceph-mon[27783]: pgmap v374: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:14:57.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:56 smithi183 ceph-mon[32344]: pgmap v374: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:14:57.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:56 smithi194 ceph-mon[32740]: pgmap v374: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:14:59.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:14:58 smithi086 ceph-mon[27783]: pgmap v375: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:14:59.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:14:58 smithi183 ceph-mon[32344]: pgmap v375: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:14:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:14:58 smithi194 ceph-mon[32740]: pgmap v375: 97 pgs: 97 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:01.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:00 smithi086 ceph-mon[27783]: pgmap v376: 97 pgs: 97 active+clean; 4.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:15:01.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:00 smithi183 ceph-mon[32344]: pgmap v376: 97 pgs: 97 active+clean; 4.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:15:01.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:00 smithi194 ceph-mon[32740]: pgmap v376: 97 pgs: 97 active+clean; 4.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:15:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:02 smithi086 ceph-mon[27783]: pgmap v377: 97 pgs: 97 active+clean; 4.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:02 smithi183 ceph-mon[32344]: pgmap v377: 97 pgs: 97 active+clean; 4.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:03.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:02 smithi194 ceph-mon[32740]: pgmap v377: 97 pgs: 97 active+clean; 4.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:05.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:04 smithi086 ceph-mon[27783]: pgmap v378: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T19:15:05.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:04 smithi183 ceph-mon[32344]: pgmap v378: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T19:15:05.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:04 smithi194 ceph-mon[32740]: pgmap v378: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 -- 172.21.15.86:0/4265374064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108560 msgr2=0x7fe5d81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 --2- 172.21.15.86:0/4265374064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108560 0x7fe5d81089e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009900 tx=0x7fe5c402a410 comp rx=0 tx=0).stop 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 -- 172.21.15.86:0/4265374064 shutdown_connections 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 --2- 172.21.15.86:0/4265374064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5d8108f20 0x7fe5d81153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 --2- 172.21.15.86:0/4265374064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108560 0x7fe5d81089e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 --2- 172.21.15.86:0/4265374064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d810e620 0x7fe5d810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 -- 172.21.15.86:0/4265374064 >> 172.21.15.86:0/4265374064 conn(0x7fe5d80764e0 msgr2=0x7fe5d8078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 -- 172.21.15.86:0/4265374064 shutdown_connections 2024-04-25T19:15:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.871+0000 7fe5dd000640 1 -- 172.21.15.86:0/4265374064 wait complete. 2024-04-25T19:15:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.872+0000 7fe5dd000640 1 Processor -- start 2024-04-25T19:15:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.872+0000 7fe5dd000640 1 -- start start 2024-04-25T19:15:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5dd000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d8108560 0x7fe5d81a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 0x7fe5d81a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5dd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5d810e620 0x7fe5d819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5dd000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8118800 con 0x7fe5d8108f20 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5dd000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8118680 con 0x7fe5d810e620 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5dd000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d8118980 con 0x7fe5d8108560 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 0x7fe5d81a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d8108560 0x7fe5d81a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 0x7fe5d81a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40028/0 (socket says 172.21.15.86:40028) 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5400640 1 -- 172.21.15.86:0/2606165159 learned_addr learned my addr 172.21.15.86:0/2606165159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d6800640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5d810e620 0x7fe5d819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2606165159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5d810e620 msgr2=0x7fe5d819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5d810e620 0x7fe5d819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 msgr2=0x7fe5d81a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 0x7fe5d81a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2606165159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d819f240 con 0x7fe5d8108560 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.873+0000 7fe5d5400640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 0x7fe5d81a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.874+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d8108560 0x7fe5d81a4910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800e9f0 tx=0x7fe5c800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.874+0000 7fe5cea00640 1 -- 172.21.15.86:0/2606165159 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800ccc0 con 0x7fe5d8108560 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.874+0000 7fe5cea00640 1 -- 172.21.15.86:0/2606165159 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c8005020 con 0x7fe5d8108560 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.874+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d819f530 con 0x7fe5d8108560 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.874+0000 7fe5cea00640 1 -- 172.21.15.86:0/2606165159 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800ce60 con 0x7fe5d8108560 2024-04-25T19:15:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.874+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d81ab7d0 con 0x7fe5d8108560 2024-04-25T19:15:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.877+0000 7fe5dfd6a740 1 Processor -- start 2024-04-25T19:15:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.877+0000 7fe5cea00640 1 -- 172.21.15.86:0/2606165159 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe5c80051d0 con 0x7fe5d8108560 2024-04-25T19:15:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.877+0000 7fe5dfd6a740 1 -- start start 2024-04-25T19:15:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f261c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f26c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 0x55ff9f26edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f1c7900 con 0x55ff9f26ca00 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f16def0 con 0x55ff9f262140 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f1c38c0 con 0x55ff9f261820 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f26c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 0x55ff9f26edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5d6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f261c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 0x55ff9f26edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40040/0 (socket says 172.21.15.86:40040) 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.878+0000 7fe5d5400640 1 -- 172.21.15.86:0/1627316837 learned_addr learned my addr 172.21.15.86:0/1627316837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5d5400640 1 -- 172.21.15.86:0/1627316837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 msgr2=0x55ff9f261c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5d5400640 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f261c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5d5400640 1 -- 172.21.15.86:0/1627316837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 msgr2=0x55ff9f26c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5d5400640 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f26c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5d5400640 1 -- 172.21.15.86:0/1627316837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9eeda210 con 0x55ff9f26ca00 2024-04-25T19:15:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5d5400640 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 0x55ff9f26edf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006010 tx=0x7fe5c4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fc0311bc6c75f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5cea00640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe5bc0a3dc0 0x7fe5bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5cea00640 1 -- 172.21.15.86:0/2606165159 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fe5c80d0820 con 0x7fe5d8108560 2024-04-25T19:15:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.879+0000 7fe5c3400640 1 -- 172.21.15.86:0/1627316837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4040070 con 0x55ff9f26ca00 2024-04-25T19:15:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.880+0000 7fe5d6800640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe5bc0a3dc0 0x7fe5bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.880+0000 7fe5c3400640 1 -- 172.21.15.86:0/1627316837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c40047a0 con 0x55ff9f26ca00 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.880+0000 7fe5c3400640 1 -- 172.21.15.86:0/1627316837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4035040 con 0x55ff9f26ca00 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.880+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/1627316837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 msgr2=0x55ff9f26edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.880+0000 7fe5d6800640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe5bc0a3dc0 0x7fe5bc0a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0005fd0 tx=0x7fe5d0009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.880+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 0x55ff9f26edf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006010 tx=0x7fe5c4005c00 comp rx=0 tx=0).stop 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/1627316837 shutdown_connections 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f26ca00 0x55ff9f26edf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f26c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/1627316837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f261c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/1627316837 >> 172.21.15.86:0/1627316837 conn(0x55ff9f25dd30 msgr2=0x55ff9f25e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/1627316837 shutdown_connections 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/1627316837 wait complete. 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 -- start start 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.881+0000 7fe5dfd6a740 1 Processor -- start 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f269a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f262140 0x55ff9f269f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f26a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f26f5d0 con 0x55ff9f261820 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f181310 con 0x55ff9f26ca00 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f26f750 con 0x55ff9f262140 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5dfd6a740 10 client.0 _ll_register_callbacks cb 0x55ff9f14fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f269a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f269a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40054/0 (socket says 172.21.15.86:40054) 2024-04-25T19:15:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2824213530 learned_addr learned my addr 172.21.15.86:0/2824213530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d6800640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f26a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5400640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f262140 0x55ff9f269f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2824213530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f262140 msgr2=0x55ff9f269f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f262140 0x55ff9f269f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.882+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 msgr2=0x55ff9f26a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f26a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2824213530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9efa6620 con 0x55ff9f261820 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f269a00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8004610 tx=0x7fe5c8098a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba664730b801738e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5b7e00640 1 -- 172.21.15.86:0/2824213530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c801c030 con 0x55ff9f261820 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5b7e00640 1 -- 172.21.15.86:0/2824213530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fe5c80054f0 con 0x55ff9f261820 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c40047a0 con 0x55ff9f261820 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5b7e00640 1 -- 172.21.15.86:0/2824213530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5c809cdf0 con 0x55ff9f261820 2024-04-25T19:15:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5dfd6a740 10 client.16626 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff9f27b8f0 con 0x55ff9f261820 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.883+0000 7fe5b7e00640 1 -- 172.21.15.86:0/2824213530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c40047a0 con 0x55ff9f261820 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5b7e00640 1 -- 172.21.15.86:0/2824213530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe5c809c950 con 0x55ff9f261820 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5dfd6a740 10 client.16626 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5dfd6a740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5dfd6a740 20 client.16626 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f27cd60 0x55ff9f27f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5dfd6a740 4 client.16626 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:05.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff9f1db8b0 con 0x55ff9f27cd60 2024-04-25T19:15:05.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.884+0000 7fe5d5400640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f27cd60 0x55ff9f27f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.885+0000 7fe5d5400640 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f27cd60 0x55ff9f27f150 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5d0fa4b1ea0e92 server_cookie=69b40e47b15c6bd6 in_seq=0 out_seq=0 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.885+0000 7fe5b7e00640 10 client.16626 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5b7e00640 1 -- 172.21.15.86:0/2824213530 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff9f1db8b0 con 0x55ff9f27cd60 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5b7e00640 10 client.16626 handle_command_reply: tid=0 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 client.16626 shutdown 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 msgr2=0x55ff9f269a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f269a00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8004610 tx=0x7fe5c8098a50 comp rx=0 tx=0).stop 2024-04-25T19:15:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 shutdown_connections 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f27cd60 0x55ff9f27f150 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f26a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f262140 0x55ff9f269f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2824213530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f269a00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5b7e00640 0 client.16626 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 >> 172.21.15.86:0/2824213530 conn(0x55ff9f25dd30 msgr2=0x55ff9f263a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 shutdown_connections 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.886+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2824213530 wait complete. 2024-04-25T19:15:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.887+0000 7fe5dfd6a740 20 client.16626 trim_cache size 0 max 16384 2024-04-25T19:15:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 Processor -- start 2024-04-25T19:15:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 -- start start 2024-04-25T19:15:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f278490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f2641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f26ca00 0x55ff9f2849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81b4310 con 0x55ff9f261820 2024-04-25T19:15:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81b5e50 con 0x55ff9f262140 2024-04-25T19:15:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.903+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81b49e0 con 0x55ff9f26ca00 2024-04-25T19:15:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f2641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f278490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f2641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:41056/0 (socket says 172.21.15.86:41056) 2024-04-25T19:15:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5400640 1 -- 172.21.15.86:0/2182715640 learned_addr learned my addr 172.21.15.86:0/2182715640 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d6800640 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f26ca00 0x55ff9f2849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2182715640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f26ca00 msgr2=0x55ff9f2849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f26ca00 0x55ff9f2849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2182715640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 msgr2=0x55ff9f2641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f2641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 -- 172.21.15.86:0/2182715640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9f262900 con 0x55ff9f261820 2024-04-25T19:15:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5400640 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f2641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f278490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8099070 tx=0x7fe5c8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98991e5bf6ce42aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5c2a00640 1 -- 172.21.15.86:0/2182715640 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c801c070 con 0x55ff9f261820 2024-04-25T19:15:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5c2a00640 1 -- 172.21.15.86:0/2182715640 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c80a5360 con 0x55ff9f261820 2024-04-25T19:15:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.904+0000 7fe5c2a00640 1 -- 172.21.15.86:0/2182715640 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c809c230 con 0x55ff9f261820 2024-04-25T19:15:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2182715640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 msgr2=0x55ff9f278490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f278490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8099070 tx=0x7fe5c8009f00 comp rx=0 tx=0).stop 2024-04-25T19:15:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2182715640 shutdown_connections 2024-04-25T19:15:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f26ca00 0x55ff9f2849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f262140 0x55ff9f2641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/2182715640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f261820 0x55ff9f278490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2182715640 >> 172.21.15.86:0/2182715640 conn(0x55ff9f25dd30 msgr2=0x55ff9f25ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2182715640 shutdown_connections 2024-04-25T19:15:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.905+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/2182715640 wait complete. 2024-04-25T19:15:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 -- start start 2024-04-25T19:15:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 Processor -- start 2024-04-25T19:15:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f272510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 0x55ff9f272a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f272f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81b5780 con 0x55ff9f262140 2024-04-25T19:15:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81b50b0 con 0x55ff9f26ca00 2024-04-25T19:15:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d81b2310 con 0x55ff9f261820 2024-04-25T19:15:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.906+0000 7fe5dfd6a740 10 client.0 _ll_register_callbacks cb 0x7fe5c8005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 0x55ff9f272a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f272510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f272f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f272510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:53166/0 (socket says 172.21.15.86:53166) 2024-04-25T19:15:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f272f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:41068/0 (socket says 172.21.15.86:41068) 2024-04-25T19:15:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5e00640 1 -- 172.21.15.86:0/4055965730 learned_addr learned my addr 172.21.15.86:0/4055965730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 0x55ff9f272a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40074/0 (socket says 172.21.15.86:40074) 2024-04-25T19:15:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 -- 172.21.15.86:0/4055965730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 msgr2=0x55ff9f272510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f272510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 -- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 msgr2=0x55ff9f272f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f272f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 -- 172.21.15.86:0/4055965730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9f2e23a0 con 0x55ff9f262140 2024-04-25T19:15:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d6800640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f272f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f272510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5d5400640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 0x55ff9f272a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4004700 tx=0x7fe5c4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed1730ff66dca12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:15:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.907+0000 7fe5b7e00640 1 -- 172.21.15.86:0/4055965730 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4040040 con 0x55ff9f262140 2024-04-25T19:15:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5b7e00640 1 -- 172.21.15.86:0/4055965730 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8637+0+0 (secure 0 0 0) 0x7fe5c4003bc0 con 0x55ff9f262140 2024-04-25T19:15:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c80a5360 con 0x55ff9f262140 2024-04-25T19:15:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5b7e00640 1 -- 172.21.15.86:0/4055965730 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5c405f050 con 0x55ff9f262140 2024-04-25T19:15:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff9f2e3690 con 0x55ff9f262140 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5b7e00640 1 -- 172.21.15.86:0/4055965730 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c80a5360 con 0x55ff9f262140 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5b7e00640 1 -- 172.21.15.86:0/4055965730 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe5c404e070 con 0x55ff9f262140 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 20 client.16632 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f2e3a20 0x55ff9f2e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 4 client.16632 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:15:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.908+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff9f1d5320 con 0x55ff9f2e3a20 2024-04-25T19:15:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.909+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f2e3a20 0x55ff9f2e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.909+0000 7fe5d5e00640 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f2e3a20 0x55ff9f2e5e10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e523b83a839ced96 server_cookie=d9fe8d94c60b9c2b in_seq=0 out_seq=0 2024-04-25T19:15:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.909+0000 7fe5b7e00640 10 client.16632 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5b7e00640 1 -- 172.21.15.86:0/4055965730 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff9f1d5320 con 0x55ff9f2e3a20 2024-04-25T19:15:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5b7e00640 10 client.16632 handle_command_reply: tid=0 2024-04-25T19:15:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 client.16632 shutdown 2024-04-25T19:15:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 msgr2=0x55ff9f272a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 0x55ff9f272a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4004700 tx=0x7fe5c4005de0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 shutdown_connections 2024-04-25T19:15:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ff9f2e3a20 0x55ff9f2e5e10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5b7e00640 0 client.16632 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ff9f26ca00 0x55ff9f272f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff9f262140 0x55ff9f272a50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 --2- 172.21.15.86:0/4055965730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9f261820 0x55ff9f272510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.925+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 >> 172.21.15.86:0/4055965730 conn(0x55ff9f25dd30 msgr2=0x55ff9f2699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.926+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 shutdown_connections 2024-04-25T19:15:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.926+0000 7fe5dfd6a740 1 -- 172.21.15.86:0/4055965730 wait complete. 2024-04-25T19:15:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.926+0000 7fe5dfd6a740 20 client.16632 trim_cache size 0 max 16384 2024-04-25T19:15:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:15:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:15:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:15:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:15:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe5bc0a3dc0 msgr2=0x7fe5bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe5bc0a3dc0 0x7fe5bc0a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0005fd0 tx=0x7fe5d0009500 comp rx=0 tx=0).stop 2024-04-25T19:15:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d8108560 msgr2=0x7fe5d81a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d8108560 0x7fe5d81a4910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800e9f0 tx=0x7fe5c800edf0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 shutdown_connections 2024-04-25T19:15:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe5bc0a3dc0 0x7fe5bc0a6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5d810e620 0x7fe5d819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5d8108f20 0x7fe5d81a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 --2- 172.21.15.86:0/2606165159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d8108560 0x7fe5d81a4910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 >> 172.21.15.86:0/2606165159 conn(0x7fe5d80764e0 msgr2=0x7fe5d8077c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 shutdown_connections 2024-04-25T19:15:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:05.927+0000 7fe5dd000640 1 -- 172.21.15.86:0/2606165159 wait complete. 2024-04-25T19:15:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:06 smithi086 ceph-mon[27783]: pgmap v379: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 80 KiB/s wr, 29 op/s 2024-04-25T19:15:07.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:06 smithi183 ceph-mon[32344]: pgmap v379: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 80 KiB/s wr, 29 op/s 2024-04-25T19:15:07.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:06 smithi194 ceph-mon[32740]: pgmap v379: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 80 KiB/s wr, 29 op/s 2024-04-25T19:15:08.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:07 smithi183 ceph-mon[32344]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T19:15:08.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:07 smithi194 ceph-mon[32740]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T19:15:08.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:07 smithi086 ceph-mon[27783]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T19:15:09.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:08 smithi183 ceph-mon[32344]: pgmap v381: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-04-25T19:15:09.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:08 smithi194 ceph-mon[32740]: pgmap v381: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-04-25T19:15:09.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:08 smithi086 ceph-mon[27783]: pgmap v381: 97 pgs: 97 active+clean; 4.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-04-25T19:15:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:10 smithi183 ceph-mon[32344]: pgmap v382: 97 pgs: 97 active+clean; 4.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-25T19:15:11.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:10 smithi194 ceph-mon[32740]: pgmap v382: 97 pgs: 97 active+clean; 4.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-25T19:15:11.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:10 smithi086 ceph-mon[27783]: pgmap v382: 97 pgs: 97 active+clean; 4.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-25T19:15:12.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:11 smithi183 ceph-mon[32344]: pgmap v383: 97 pgs: 97 active+clean; 4.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-25T19:15:12.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:11 smithi194 ceph-mon[32740]: pgmap v383: 97 pgs: 97 active+clean; 4.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-25T19:15:12.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:11 smithi086 ceph-mon[27783]: pgmap v383: 97 pgs: 97 active+clean; 4.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 108 KiB/s wr, 38 op/s 2024-04-25T19:15:14.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:14 smithi086 ceph-mon[27783]: pgmap v384: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T19:15:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:14 smithi086 ceph-mon[27783]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T19:15:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:14 smithi183 ceph-mon[32344]: pgmap v384: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T19:15:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:14 smithi183 ceph-mon[32344]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T19:15:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:14 smithi194 ceph-mon[32740]: pgmap v384: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T19:15:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:14 smithi194 ceph-mon[32740]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T19:15:16.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:16 smithi086 ceph-mon[27783]: pgmap v386: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-04-25T19:15:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:16 smithi183 ceph-mon[32344]: pgmap v386: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-04-25T19:15:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:16 smithi194 ceph-mon[32740]: pgmap v386: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-04-25T19:15:18.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:18 smithi086 ceph-mon[27783]: pgmap v387: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T19:15:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:18 smithi183 ceph-mon[32344]: pgmap v387: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T19:15:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:18 smithi194 ceph-mon[32740]: pgmap v387: 97 pgs: 97 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 -- 172.21.15.86:0/171014540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108f20 msgr2=0x7f20901153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 --2- 172.21.15.86:0/171014540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108f20 0x7f20901153e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2088009c10 tx=0x7f208802a6e0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 -- 172.21.15.86:0/171014540 shutdown_connections 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 --2- 172.21.15.86:0/171014540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108f20 0x7f20901153e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 --2- 172.21.15.86:0/171014540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090108560 0x7f20901089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 --2- 172.21.15.86:0/171014540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209010e620 0x7f209010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 -- 172.21.15.86:0/171014540 >> 172.21.15.86:0/171014540 conn(0x7f20900764e0 msgr2=0x7f2090078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 -- 172.21.15.86:0/171014540 shutdown_connections 2024-04-25T19:15:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.071+0000 7f2095800640 1 -- 172.21.15.86:0/171014540 wait complete. 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.072+0000 7f2095800640 1 Processor -- start 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.072+0000 7f2095800640 1 -- start start 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2095800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108560 0x7f20901a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2095800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2090108f20 0x7f20901a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2095800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209010e620 0x7f209019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2095800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2090118790 con 0x7f209010e620 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2095800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2090118610 con 0x7f2090108f20 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2095800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2090118910 con 0x7f2090108560 2024-04-25T19:15:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209010e620 0x7f209019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108560 0x7f20901a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209010e620 0x7f209019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50632/0 (socket says 172.21.15.86:50632) 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208f400640 1 -- 172.21.15.86:0/3943874091 learned_addr learned my addr 172.21.15.86:0/3943874091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f2086000640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2090108f20 0x7f20901a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208ea00640 1 -- 172.21.15.86:0/3943874091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2090108f20 msgr2=0x7f20901a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208ea00640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2090108f20 0x7f20901a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208ea00640 1 -- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209010e620 msgr2=0x7f209019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208ea00640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209010e620 0x7f209019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.073+0000 7f208ea00640 1 -- 172.21.15.86:0/3943874091 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209019f1d0 con 0x7f2090108560 2024-04-25T19:15:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.074+0000 7f208ea00640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108560 0x7f20901a4850 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f208000e9f0 tx=0x7f208000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.074+0000 7f2087e00640 1 -- 172.21.15.86:0/3943874091 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000ccc0 con 0x7f2090108560 2024-04-25T19:15:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.074+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209019f4c0 con 0x7f2090108560 2024-04-25T19:15:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.074+0000 7f2087e00640 1 -- 172.21.15.86:0/3943874091 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20800108e0 con 0x7f2090108560 2024-04-25T19:15:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.074+0000 7f2087e00640 1 -- 172.21.15.86:0/3943874091 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080018840 con 0x7f2090108560 2024-04-25T19:15:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.074+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901ab710 con 0x7f2090108560 2024-04-25T19:15:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.078+0000 7f209866a740 1 Processor -- start 2024-04-25T19:15:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.078+0000 7f209866a740 1 -- start start 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f209866a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febd5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f209866a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febe04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f209866a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febe2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f209866a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5556feb3b900 con 0x5556febe0a00 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f209866a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5556feae1ef0 con 0x5556febd6140 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f209866a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556feb378c0 con 0x5556febd5820 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f2087e00640 1 -- 172.21.15.86:0/3943874091 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2080010430 con 0x7f2090108560 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febd5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f2086000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febe2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febd5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:49858/0 (socket says 172.21.15.86:49858) 2024-04-25T19:15:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 -- 172.21.15.86:0/3916027884 learned_addr learned my addr 172.21.15.86:0/3916027884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 -- 172.21.15.86:0/3916027884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 msgr2=0x5556febe04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febe04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 -- 172.21.15.86:0/3916027884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 msgr2=0x5556febe2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.079+0000 7f208f400640 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febe2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.080+0000 7f208f400640 1 -- 172.21.15.86:0/3916027884 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556fe84e210 con 0x5556febd5820 2024-04-25T19:15:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.080+0000 7f2086000640 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febe2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.080+0000 7f208f400640 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febd5c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2088034040 tx=0x7f2088030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09cda1e30f81f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.080+0000 7f207fe00640 1 -- 172.21.15.86:0/3916027884 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088041030 con 0x5556febd5820 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.080+0000 7f207fe00640 1 -- 172.21.15.86:0/3916027884 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088003e20 con 0x5556febd5820 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.080+0000 7f207fe00640 1 -- 172.21.15.86:0/3916027884 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088036040 con 0x5556febd5820 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f2087e00640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f20700a3e40 0x7f20700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f2087e00640 1 -- 172.21.15.86:0/3943874091 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f20800d0a90 con 0x7f2090108560 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f208ea00640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f20700a3e40 0x7f20700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 -- 172.21.15.86:0/3916027884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 msgr2=0x5556febd5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febd5c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2088034040 tx=0x7f2088030430 comp rx=0 tx=0).stop 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 -- 172.21.15.86:0/3916027884 shutdown_connections 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febe2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febe04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 --2- 172.21.15.86:0/3916027884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febd5c00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 -- 172.21.15.86:0/3916027884 >> 172.21.15.86:0/3916027884 conn(0x5556febd1d30 msgr2=0x5556febd2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 -- 172.21.15.86:0/3916027884 shutdown_connections 2024-04-25T19:15:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.081+0000 7f209866a740 1 -- 172.21.15.86:0/3916027884 wait complete. 2024-04-25T19:15:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.082+0000 7f208ea00640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f20700a3e40 0x7f20700a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5556febd6dc0 tx=0x7f2080098600 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.082+0000 7f209866a740 1 -- start start 2024-04-25T19:15:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.082+0000 7f209866a740 1 Processor -- start 2024-04-25T19:15:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 0x5556febdd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556febde440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5556febe35d0 con 0x5556febd5820 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5556feaf5310 con 0x5556febd6140 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556febe3750 con 0x5556febe0a00 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f209866a740 10 client.0 _ll_register_callbacks cb 0x5556feac3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f208f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 0x5556febdd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f208f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 0x5556febdd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50652/0 (socket says 172.21.15.86:50652) 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f208f400640 1 -- 172.21.15.86:0/3705089941 learned_addr learned my addr 172.21.15.86:0/3705089941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f2086000640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f208f400640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556febde440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f2086000640 1 -- 172.21.15.86:0/3705089941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 msgr2=0x5556febde440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f2086000640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556febde440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f2086000640 1 -- 172.21.15.86:0/3705089941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 msgr2=0x5556febdd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f2086000640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 0x5556febdd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f2086000640 1 -- 172.21.15.86:0/3705089941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556fe91a620 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.083+0000 7f208f400640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 0x5556febdd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f2086000640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febddf00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f209010dce0 tx=0x7f2078011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9358472843f0ade4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f207cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f207cc00640 1 -- 172.21.15.86:0/3705089941 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207801f060 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f207cc00640 1 -- 172.21.15.86:0/3705089941 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f2078004300 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2088003e20 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f207cc00640 1 -- 172.21.15.86:0/3705089941 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f207800f8f0 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f209866a740 10 client.26744 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.084+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556febef910 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f207cc00640 1 -- 172.21.15.86:0/3705089941 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088003e20 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f207cc00640 1 -- 172.21.15.86:0/3705089941 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2078019030 con 0x5556febd6140 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f209866a740 10 client.26744 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f209866a740 10 client.26744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f209866a740 20 client.26744 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f209866a740 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556febf0d80 0x5556febf3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f209866a740 4 client.26744 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556feb4f8b0 con 0x5556febf0d80 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.085+0000 7f208f400640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556febf0d80 0x5556febf3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.086+0000 7f208f400640 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556febf0d80 0x5556febf3170 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4ffdb4ddf4bd14 server_cookie=2db73a0b79ab5c04 in_seq=0 out_seq=0 2024-04-25T19:15:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.086+0000 7f207cc00640 10 client.26744 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f207cc00640 1 -- 172.21.15.86:0/3705089941 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556feb4f8b0 con 0x5556febf0d80 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f207cc00640 10 client.26744 handle_command_reply: tid=0 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 client.26744 shutdown 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 msgr2=0x5556febddf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febddf00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f209010dce0 tx=0x7f2078011b50 comp rx=0 tx=0).stop 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 shutdown_connections 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556febf0d80 0x5556febf3170 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556febde440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febddf00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f207cc00640 0 client.26744 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 --2- 172.21.15.86:0/3705089941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd5820 0x5556febdd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 >> 172.21.15.86:0/3705089941 conn(0x5556febd1d30 msgr2=0x5556febd7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 shutdown_connections 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.087+0000 7f209866a740 1 -- 172.21.15.86:0/3705089941 wait complete. 2024-04-25T19:15:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.088+0000 7f209866a740 20 client.26744 trim_cache size 0 max 16384 2024-04-25T19:15:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 Processor -- start 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 -- start start 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febde090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febf8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b4210 con 0x5556febe0a00 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b5d50 con 0x5556febd6140 2024-04-25T19:15:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.104+0000 7f209866a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b48e0 con 0x5556febd5820 2024-04-25T19:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f208f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febf8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febde090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f208f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febf8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50664/0 (socket says 172.21.15.86:50664) 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febde090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60706/0 (socket says 172.21.15.86:60706) 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f208f400640 1 -- 172.21.15.86:0/2078869732 learned_addr learned my addr 172.21.15.86:0/2078869732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f208f400640 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 -- 172.21.15.86:0/2078869732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 msgr2=0x5556febec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 -- 172.21.15.86:0/2078869732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 msgr2=0x5556febf8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febf8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 -- 172.21.15.86:0/2078869732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556febf37d0 con 0x5556febd6140 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.105+0000 7f2086000640 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febde090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2078024040 tx=0x7f207801a600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24de45ece608e4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.106+0000 7f207f400640 1 -- 172.21.15.86:0/2078869732 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207802f070 con 0x5556febd6140 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.106+0000 7f207f400640 1 -- 172.21.15.86:0/2078869732 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207800fa90 con 0x5556febd6140 2024-04-25T19:15:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.106+0000 7f207f400640 1 -- 172.21.15.86:0/2078869732 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078003a60 con 0x5556febd6140 2024-04-25T19:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.106+0000 7f209866a740 1 -- 172.21.15.86:0/2078869732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 msgr2=0x5556febde090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.106+0000 7f209866a740 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febde090 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2078024040 tx=0x7f207801a600 comp rx=0 tx=0).stop 2024-04-25T19:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 -- 172.21.15.86:0/2078869732 shutdown_connections 2024-04-25T19:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febe0a00 0x5556febf8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd6140 0x5556febde090 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 --2- 172.21.15.86:0/2078869732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febd5820 0x5556febec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 -- 172.21.15.86:0/2078869732 >> 172.21.15.86:0/2078869732 conn(0x5556febd1d30 msgr2=0x5556febdd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 -- 172.21.15.86:0/2078869732 shutdown_connections 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 -- 172.21.15.86:0/2078869732 wait complete. 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 -- start start 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.107+0000 7f209866a740 1 Processor -- start 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 0x5556febe5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd6140 0x5556febe6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556fec4f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b5680 con 0x5556febd6140 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b4fb0 con 0x5556febd5820 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b2210 con 0x5556febe0a00 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f209866a740 10 client.0 _ll_register_callbacks cb 0x7f2078009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 0x5556febe5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 0x5556febe5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60718/0 (socket says 172.21.15.86:60718) 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 -- 172.21.15.86:0/2872697120 learned_addr learned my addr 172.21.15.86:0/2872697120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f2086000640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd6140 0x5556febe6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556fec4f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 -- 172.21.15.86:0/2872697120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 msgr2=0x5556fec4f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556fec4f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 -- 172.21.15.86:0/2872697120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd6140 msgr2=0x5556febe6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.108+0000 7f208f400640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd6140 0x5556febe6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f208f400640 1 -- 172.21.15.86:0/2872697120 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556fec4f870 con 0x5556febd5820 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f208f400640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 0x5556febe5c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2088009560 tx=0x7f208803f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52cc3a1ef0c8057d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f207cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f207cc00640 1 -- 172.21.15.86:0/2872697120 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088036030 con 0x5556febd5820 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207800fa90 con 0x5556febd5820 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f207cc00640 1 -- 172.21.15.86:0/2872697120 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f2088003970 con 0x5556febd5820 2024-04-25T19:15:19.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f207cc00640 1 -- 172.21.15.86:0/2872697120 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2088050040 con 0x5556febd5820 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f209866a740 10 client.26750 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.109+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556fec56c60 con 0x5556febd5820 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f207cc00640 1 -- 172.21.15.86:0/2872697120 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207800fa90 con 0x5556febd5820 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f207cc00640 1 -- 172.21.15.86:0/2872697120 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f208803d760 con 0x5556febd5820 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f209866a740 10 client.26750 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f209866a740 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f209866a740 20 client.26750 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f209866a740 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556fec56fa0 0x5556fec59390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f209866a740 4 client.26750 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:15:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556feb49320 con 0x5556fec56fa0 2024-04-25T19:15:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.110+0000 7f2086000640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556fec56fa0 0x5556fec59390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.111+0000 7f2086000640 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556fec56fa0 0x5556fec59390 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad6793fc20e9fef server_cookie=accb1fcd64001856 in_seq=0 out_seq=0 2024-04-25T19:15:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.111+0000 7f207cc00640 10 client.26750 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.125+0000 7f207cc00640 1 -- 172.21.15.86:0/2872697120 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556feb49320 con 0x5556fec56fa0 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.125+0000 7f207cc00640 10 client.26750 handle_command_reply: tid=0 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.125+0000 7f209866a740 1 client.26750 shutdown 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 msgr2=0x5556febe5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 0x5556febe5c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2088009560 tx=0x7f208803f040 comp rx=0 tx=0).stop 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 shutdown_connections 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5556fec56fa0 0x5556fec59390 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556febe0a00 0x5556fec4f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556febd6140 0x5556febe6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 --2- 172.21.15.86:0/2872697120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5556febd5820 0x5556febe5c20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f207cc00640 0 client.26750 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 >> 172.21.15.86:0/2872697120 conn(0x5556febd1d30 msgr2=0x5556febdbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 shutdown_connections 2024-04-25T19:15:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 1 -- 172.21.15.86:0/2872697120 wait complete. 2024-04-25T19:15:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.126+0000 7f209866a740 20 client.26750 trim_cache size 0 max 16384 2024-04-25T19:15:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:15:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:15:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:15:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f20700a3e40 msgr2=0x7f20700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f20700a3e40 0x7f20700a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5556febd6dc0 tx=0x7f2080098600 comp rx=0 tx=0).stop 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108560 msgr2=0x7f20901a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108560 0x7f20901a4850 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f208000e9f0 tx=0x7f208000edf0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 shutdown_connections 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f20700a3e40 0x7f20700a6300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209010e620 0x7f209019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2090108f20 0x7f20901a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 --2- 172.21.15.86:0/3943874091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2090108560 0x7f20901a4850 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 >> 172.21.15.86:0/3943874091 conn(0x7f20900764e0 msgr2=0x7f2090077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 shutdown_connections 2024-04-25T19:15:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:19.127+0000 7f2095800640 1 -- 172.21.15.86:0/3943874091 wait complete. 2024-04-25T19:15:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:20 smithi086 ceph-mon[27783]: pgmap v388: 97 pgs: 97 active+clean; 4.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:15:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:20 smithi183 ceph-mon[32344]: pgmap v388: 97 pgs: 97 active+clean; 4.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:15:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:20 smithi194 ceph-mon[32740]: pgmap v388: 97 pgs: 97 active+clean; 4.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:15:22.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:22 smithi086 ceph-mon[27783]: pgmap v389: 97 pgs: 97 active+clean; 4.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:15:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:22 smithi183 ceph-mon[32344]: pgmap v389: 97 pgs: 97 active+clean; 4.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:15:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:22 smithi194 ceph-mon[32740]: pgmap v389: 97 pgs: 97 active+clean; 4.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:15:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:24 smithi086 ceph-mon[27783]: pgmap v390: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-25T19:15:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:24 smithi183 ceph-mon[32344]: pgmap v390: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-25T19:15:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:24 smithi194 ceph-mon[32740]: pgmap v390: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-25T19:15:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:26 smithi086 ceph-mon[27783]: pgmap v391: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:15:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:26 smithi183 ceph-mon[32344]: pgmap v391: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:15:26.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:26 smithi194 ceph-mon[32740]: pgmap v391: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:15:28.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:28 smithi086 ceph-mon[27783]: pgmap v392: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:28.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:28 smithi183 ceph-mon[32344]: pgmap v392: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:28 smithi194 ceph-mon[32740]: pgmap v392: 97 pgs: 97 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:15:29.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:29 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:15:29.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:29 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:15:30.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:29 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:15:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:30 smithi183 ceph-mon[32344]: pgmap v393: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:15:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:30 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:15:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:30 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:15:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:30 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:15:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:30 smithi194 ceph-mon[32740]: pgmap v393: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:15:30.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:30 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:15:30.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:30 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:15:30.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:30 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:15:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:30 smithi086 ceph-mon[27783]: pgmap v393: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T19:15:31.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:30 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:15:31.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:30 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:15:31.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:30 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 -- 172.21.15.86:0/877057617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 msgr2=0x7ff7b0074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 --2- 172.21.15.86:0/877057617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b0074630 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009860 tx=0x7ff7a402a3d0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 -- 172.21.15.86:0/877057617 shutdown_connections 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 --2- 172.21.15.86:0/877057617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b010e840 0x7ff7b0110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 --2- 172.21.15.86:0/877057617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b0074b70 0x7ff7b010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 --2- 172.21.15.86:0/877057617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b0074630 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 -- 172.21.15.86:0/877057617 >> 172.21.15.86:0/877057617 conn(0x7ff7b00793e0 msgr2=0x7ff7b00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 -- 172.21.15.86:0/877057617 shutdown_connections 2024-04-25T19:15:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.269+0000 7ff7b7400640 1 -- 172.21.15.86:0/877057617 wait complete. 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.270+0000 7ff7b7400640 1 Processor -- start 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.270+0000 7ff7b7400640 1 -- start start 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 0x7ff7b01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b010e840 0x7ff7b019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0114450 con 0x7ff7b0074b70 2024-04-25T19:15:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b7400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01142d0 con 0x7ff7b0074250 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b7400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01145d0 con 0x7ff7b010e840 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 0x7ff7b01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 0x7ff7b01a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60660/0 (socket says 172.21.15.86:60660) 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7afe00640 1 -- 172.21.15.86:0/2072825326 learned_addr learned my addr 172.21.15.86:0/2072825326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.271+0000 7ff7b5400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b010e840 0x7ff7b019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7afe00640 1 -- 172.21.15.86:0/2072825326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b010e840 msgr2=0x7ff7b019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7afe00640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b010e840 0x7ff7b019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7afe00640 1 -- 172.21.15.86:0/2072825326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 msgr2=0x7ff7b01a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7afe00640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b01a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7afe00640 1 -- 172.21.15.86:0/2072825326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019af70 con 0x7ff7b0074b70 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b01a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7afe00640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 0x7ff7b01a0ae0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7ad600640 1 -- 172.21.15.86:0/2072825326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00cd80 con 0x7ff7b0074b70 2024-04-25T19:15:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019b260 con 0x7ff7b0074b70 2024-04-25T19:15:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7ad600640 1 -- 172.21.15.86:0/2072825326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c0054d0 con 0x7ff7b0074b70 2024-04-25T19:15:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7ad600640 1 -- 172.21.15.86:0/2072825326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c010690 con 0x7ff7b0074b70 2024-04-25T19:15:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.272+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a7520 con 0x7ff7b0074b70 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.275+0000 7ff7ba209740 1 Processor -- start 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 -- start start 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ad600640 1 -- 172.21.15.86:0/2072825326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff79c004e90 con 0x7ff7b0074b70 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0278f820 0x559d0278fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d0279a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0279aa00 0x559d0279cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d026f5900 con 0x559d0278f820 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d0269bef0 con 0x559d0279aa00 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7ba209740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d026f18c0 con 0x559d02790140 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0278f820 0x559d0278fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0278f820 0x559d0278fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60676/0 (socket says 172.21.15.86:60676) 2024-04-25T19:15:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7b5400640 1 -- 172.21.15.86:0/1659538109 learned_addr learned my addr 172.21.15.86:0/1659538109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7afe00640 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0279aa00 0x559d0279cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.276+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d0279a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7b4a00640 1 -- 172.21.15.86:0/1659538109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0279aa00 msgr2=0x559d0279cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0279aa00 0x559d0279cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7b4a00640 1 -- 172.21.15.86:0/1659538109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0278f820 msgr2=0x559d0278fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0278f820 0x559d0278fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7b4a00640 1 -- 172.21.15.86:0/1659538109 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d02408210 con 0x559d02790140 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d0279a4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009830 tx=0x7ff7a4030860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671df754243b2c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.277+0000 7ff7a1c00640 1 -- 172.21.15.86:0/1659538109 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4030c80 con 0x559d02790140 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ad600640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff7980a3dc0 0x7ff7980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7a1c00640 1 -- 172.21.15.86:0/1659538109 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a4041030 con 0x559d02790140 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7a1c00640 1 -- 172.21.15.86:0/1659538109 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40046c0 con 0x559d02790140 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ad600640 1 -- 172.21.15.86:0/2072825326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7ff79c0d0c70 con 0x7ff7b0074b70 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7b5400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff7980a3dc0 0x7ff7980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 -- 172.21.15.86:0/1659538109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 msgr2=0x559d0279a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d0279a4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009830 tx=0x7ff7a4030860 comp rx=0 tx=0).stop 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 -- 172.21.15.86:0/1659538109 shutdown_connections 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0279aa00 0x559d0279cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d0279a4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 --2- 172.21.15.86:0/1659538109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0278f820 0x559d0278fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.278+0000 7ff7ba209740 1 -- 172.21.15.86:0/1659538109 >> 172.21.15.86:0/1659538109 conn(0x559d0278bd30 msgr2=0x559d0278c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.279+0000 7ff7ba209740 1 -- 172.21.15.86:0/1659538109 shutdown_connections 2024-04-25T19:15:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.279+0000 7ff7b5400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff7980a3dc0 0x7ff7980a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009fd0 tx=0x7ff7a8010040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.279+0000 7ff7ba209740 1 -- 172.21.15.86:0/1659538109 wait complete. 2024-04-25T19:15:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.279+0000 7ff7ba209740 1 -- start start 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.279+0000 7ff7ba209740 1 Processor -- start 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d02797910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d02797e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d02798390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d0279d5d0 con 0x559d0279aa00 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d026af310 con 0x559d02790140 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d0279d750 con 0x559d0278f820 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7ba209740 10 client.0 _ll_register_callbacks cb 0x559d0267dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d02798390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d02797910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d02797910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58280/0 (socket says 172.21.15.86:58280) 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d02798390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60678/0 (socket says 172.21.15.86:60678) 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 -- 172.21.15.86:0/2698622266 learned_addr learned my addr 172.21.15.86:0/2698622266 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7afe00640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d02797e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 -- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 msgr2=0x559d02797e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d02797e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 -- 172.21.15.86:0/2698622266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 msgr2=0x559d02798390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.280+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d02798390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff7b4a00640 1 -- 172.21.15.86:0/2698622266 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d024d4620 con 0x559d0278f820 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff7b5400640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d02798390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d02797910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004600 tx=0x7ff7a4030ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca3f8534f7a3a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff796000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff796000640 1 -- 172.21.15.86:0/2698622266 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a403d430 con 0x559d0278f820 2024-04-25T19:15:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff796000640 1 -- 172.21.15.86:0/2698622266 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7ff7a403d5d0 con 0x559d0278f820 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a4041030 con 0x559d0278f820 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff796000640 1 -- 172.21.15.86:0/2698622266 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7a403d7d0 con 0x559d0278f820 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff7ba209740 10 client.26758 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.281+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d0279d5d0 con 0x559d0278f820 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff796000640 1 -- 172.21.15.86:0/2698622266 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a4041030 con 0x559d0278f820 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff796000640 1 -- 172.21.15.86:0/2698622266 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff7a40529c0 con 0x559d0278f820 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff7ba209740 10 client.26758 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff7ba209740 10 client.26758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff7ba209740 20 client.26758 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff7ba209740 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d027aa930 0x559d027acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff7ba209740 4 client.26758 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.282+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d027098b0 con 0x559d027aa930 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.283+0000 7ff7afe00640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d027aa930 0x559d027acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.283+0000 7ff7afe00640 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d027aa930 0x559d027acd20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18fdf0732ba7478 server_cookie=8c0d71425cda1fed in_seq=0 out_seq=0 2024-04-25T19:15:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.283+0000 7ff796000640 10 client.26758 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.284+0000 7ff796000640 1 -- 172.21.15.86:0/2698622266 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d027098b0 con 0x559d027aa930 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.284+0000 7ff796000640 10 client.26758 handle_command_reply: tid=0 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.284+0000 7ff7ba209740 1 client.26758 shutdown 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 msgr2=0x559d02797910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d02797910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004600 tx=0x7ff7a4030ef0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 shutdown_connections 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d027aa930 0x559d027acd20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d02798390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff796000640 0 client.26758 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d02797e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 --2- 172.21.15.86:0/2698622266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d02797910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 >> 172.21.15.86:0/2698622266 conn(0x559d0278bd30 msgr2=0x559d02791ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 shutdown_connections 2024-04-25T19:15:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 1 -- 172.21.15.86:0/2698622266 wait complete. 2024-04-25T19:15:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.285+0000 7ff7ba209740 20 client.26758 trim_cache size 0 max 16384 2024-04-25T19:15:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 Processor -- start 2024-04-25T19:15:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 -- start start 2024-04-25T19:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d027a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d027971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d027921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01afee0 con 0x559d0279aa00 2024-04-25T19:15:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b1a20 con 0x559d02790140 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.301+0000 7ff7ba209740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b05b0 con 0x559d0278f820 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d027921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d027971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d027921e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60694/0 (socket says 172.21.15.86:60694) 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7b5400640 1 -- 172.21.15.86:0/3209163942 learned_addr learned my addr 172.21.15.86:0/3209163942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d027a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 -- 172.21.15.86:0/3209163942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 msgr2=0x559d027a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d027a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 -- 172.21.15.86:0/3209163942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 msgr2=0x559d027921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d027921e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 -- 172.21.15.86:0/3209163942 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d02790ad0 con 0x559d02790140 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7b5400640 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d027921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.302+0000 7ff7afe00640 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d027971f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0ce350 tx=0x7ff79c0bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74812b164664b7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7a1200640 1 -- 172.21.15.86:0/3209163942 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c098e40 con 0x559d02790140 2024-04-25T19:15:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7a1200640 1 -- 172.21.15.86:0/3209163942 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c005680 con 0x559d02790140 2024-04-25T19:15:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7a1200640 1 -- 172.21.15.86:0/3209163942 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c005180 con 0x559d02790140 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 -- 172.21.15.86:0/3209163942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 msgr2=0x559d027971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d027971f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0ce350 tx=0x7ff79c0bd040 comp rx=0 tx=0).stop 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 -- 172.21.15.86:0/3209163942 shutdown_connections 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d027921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d02790140 0x559d027971f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 --2- 172.21.15.86:0/3209163942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d0278f820 0x559d027a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 -- 172.21.15.86:0/3209163942 >> 172.21.15.86:0/3209163942 conn(0x559d0278bd30 msgr2=0x559d027999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.303+0000 7ff7ba209740 1 -- 172.21.15.86:0/3209163942 shutdown_connections 2024-04-25T19:15:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 -- 172.21.15.86:0/3209163942 wait complete. 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 -- start start 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 Processor -- start 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 0x559d0279fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d027a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d0280ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.304+0000 7ff7ba209740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b1350 con 0x559d0279aa00 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7ba209740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b0c80 con 0x559d0278f820 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7ba209740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01adee0 con 0x559d02790140 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7ba209740 10 client.0 _ll_register_callbacks cb 0x7ff7a40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 0x559d0279fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d0280ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d027a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 0x559d0279fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43072/0 (socket says 172.21.15.86:43072) 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7b4a00640 1 -- 172.21.15.86:0/80558756 learned_addr learned my addr 172.21.15.86:0/80558756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 -- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 msgr2=0x559d0279fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 0x559d0279fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 -- 172.21.15.86:0/80558756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 msgr2=0x559d0280ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d0280ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 -- 172.21.15.86:0/80558756 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d0280f240 con 0x559d02790140 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 0x559d0279fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff7afe00640 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d027a00a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0ce380 tx=0x7ff79c0d47d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb57420dce2dfcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.305+0000 7ff796000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff796000640 1 -- 172.21.15.86:0/80558756 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0b5050 con 0x559d02790140 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff796000640 1 -- 172.21.15.86:0/80558756 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7ff79c0bd0b0 con 0x559d02790140 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c005680 con 0x559d02790140 2024-04-25T19:15:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff796000640 1 -- 172.21.15.86:0/80558756 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79c09c500 con 0x559d02790140 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 10 client.26764 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d028103f0 con 0x559d02790140 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff796000640 1 -- 172.21.15.86:0/80558756 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c005680 con 0x559d02790140 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff796000640 1 -- 172.21.15.86:0/80558756 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff79c0ce5d0 con 0x559d02790140 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 10 client.26764 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 20 client.26764 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d02810a80 0x559d02812ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 4 client.26764 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.306+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d02703320 con 0x559d02810a80 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.307+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d02810a80 0x559d02812ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.307+0000 7ff7b4a00640 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d02810a80 0x559d02812ea0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2119fc02dd78c579 server_cookie=225d19a19307b3bb in_seq=0 out_seq=0 2024-04-25T19:15:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.307+0000 7ff796000640 10 client.26764 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.321+0000 7ff796000640 1 -- 172.21.15.86:0/80558756 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d02703320 con 0x559d02810a80 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.321+0000 7ff796000640 10 client.26764 handle_command_reply: tid=0 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.321+0000 7ff7ba209740 1 client.26764 shutdown 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 msgr2=0x559d027a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d027a00a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0ce380 tx=0x7ff79c0d47d0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 shutdown_connections 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559d02810a80 0x559d02812ea0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d0279aa00 0x559d0280ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d02790140 0x559d027a00a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 --2- 172.21.15.86:0/80558756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d0278f820 0x559d0279fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff796000640 0 client.26764 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 >> 172.21.15.86:0/80558756 conn(0x559d0278bd30 msgr2=0x559d027978d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 shutdown_connections 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 1 -- 172.21.15.86:0/80558756 wait complete. 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.322+0000 7ff7ba209740 20 client.26764 trim_cache size 0 max 16384 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:15:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:15:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff7980a3dc0 msgr2=0x7ff7980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff7980a3dc0 0x7ff7980a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009fd0 tx=0x7ff7a8010040 comp rx=0 tx=0).stop 2024-04-25T19:15:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 msgr2=0x7ff7b01a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 0x7ff7b01a0ae0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00edf0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 shutdown_connections 2024-04-25T19:15:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff7980a3dc0 0x7ff7980a6280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7b010e840 0x7ff7b019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7b0074b70 0x7ff7b01a0ae0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 --2- 172.21.15.86:0/2072825326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b0074250 0x7ff7b01a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.323+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 >> 172.21.15.86:0/2072825326 conn(0x7ff7b00793e0 msgr2=0x7ff7b00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.324+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 shutdown_connections 2024-04-25T19:15:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:32.324+0000 7ff7b7400640 1 -- 172.21.15.86:0/2072825326 wait complete. 2024-04-25T19:15:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:32 smithi183 ceph-mon[32344]: pgmap v394: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:15:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:32 smithi194 ceph-mon[32740]: pgmap v394: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:15:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:32 smithi086 ceph-mon[27783]: pgmap v394: 97 pgs: 97 active+clean; 5.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:15:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:34 smithi183 ceph-mon[32344]: pgmap v395: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 62 KiB/s wr, 25 op/s 2024-04-25T19:15:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:34 smithi194 ceph-mon[32740]: pgmap v395: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 62 KiB/s wr, 25 op/s 2024-04-25T19:15:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:34 smithi086 ceph-mon[27783]: pgmap v395: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 62 KiB/s wr, 25 op/s 2024-04-25T19:15:35.606 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.5 2024-04-25T19:15:35.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:15:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:15:36.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:36 smithi086 ceph-mon[27783]: pgmap v396: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:15:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:36 smithi183 ceph-mon[32344]: pgmap v396: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:15:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:36 smithi194 ceph-mon[32740]: pgmap v396: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:15:38.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 -- 172.21.15.86:0/4039617473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 msgr2=0x7ffa6010efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:38.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 --2- 172.21.15.86:0/4039617473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6010efb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0098f0 tx=0x7ffa4c009c00 comp rx=0 tx=0).stop 2024-04-25T19:15:38.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 -- 172.21.15.86:0/4039617473 shutdown_connections 2024-04-25T19:15:38.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 --2- 172.21.15.86:0/4039617473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa6010f4f0 0x7ffa601118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 --2- 172.21.15.86:0/4039617473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 0x7ffa6006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 --2- 172.21.15.86:0/4039617473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6010efb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.396+0000 7ffa65c00700 1 -- 172.21.15.86:0/4039617473 >> 172.21.15.86:0/4039617473 conn(0x7ffa600788f0 msgr2=0x7ffa60078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.397+0000 7ffa65c00700 1 -- 172.21.15.86:0/4039617473 shutdown_connections 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.397+0000 7ffa65c00700 1 -- 172.21.15.86:0/4039617473 wait complete. 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 Processor -- start 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 -- start start 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 0x7ffa6019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa6010f4f0 0x7ffa6019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60114e50 con 0x7ffa6006a750 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60114cd0 con 0x7ffa6010f4f0 2024-04-25T19:15:38.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.398+0000 7ffa65c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60114fd0 con 0x7ffa6006b070 2024-04-25T19:15:38.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 0x7ffa6019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58308/0 (socket says 172.21.15.86:58308) 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 -- 172.21.15.86:0/399449927 learned_addr learned my addr 172.21.15.86:0/399449927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5f400700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa6010f4f0 0x7ffa6019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 -- 172.21.15.86:0/399449927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa6010f4f0 msgr2=0x7ffa6019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa6010f4f0 0x7ffa6019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 -- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 msgr2=0x7ffa6019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 0x7ffa6019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5e000700 1 -- 172.21.15.86:0/399449927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601a03f0 con 0x7ffa6006b070 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.399+0000 7ffa5ea00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 0x7ffa6019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.400+0000 7ffa5e000700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6019b920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009230 tx=0x7ffa4c00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.400+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c005140 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.400+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4c00b560 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.400+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601a0680 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.400+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c003ae0 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.400+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601a7f60 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.402+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa60069730 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.404+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa4c003c80 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.405+0000 7ffa57400700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffa440a3db0 0x7ffa440a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.405+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7ffa4c0d1850 con 0x7ffa6006b070 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.406+0000 7ffa5ea00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffa440a3db0 0x7ffa440a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.406+0000 7ffa5ea00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffa440a3db0 0x7ffa440a6270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009750 tx=0x7ffa50009310 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:38.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.410+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa4c013030 con 0x7ffa6006b070 2024-04-25T19:15:38.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.692+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa6002df80 con 0x7ffa6006b070 2024-04-25T19:15:38.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:38 smithi086 ceph-mon[27783]: pgmap v397: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:15:38.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.693+0000 7ffa57400700 1 -- 172.21.15.86:0/399449927 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7ffa4c09deb0 con 0x7ffa6006b070 2024-04-25T19:15:38.695 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:15:38.695 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:15:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffa440a3db0 msgr2=0x7ffa440a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffa440a3db0 0x7ffa440a6270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009750 tx=0x7ffa50009310 comp rx=0 tx=0).stop 2024-04-25T19:15:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 msgr2=0x7ffa6019b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6019b920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009230 tx=0x7ffa4c00fa70 comp rx=0 tx=0).stop 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 shutdown_connections 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa6010f4f0 0x7ffa6019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa6006a750 0x7ffa6019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ffa440a3db0 0x7ffa440a6270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 --2- 172.21.15.86:0/399449927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa6006b070 0x7ffa6019b920 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.696+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 >> 172.21.15.86:0/399449927 conn(0x7ffa600788f0 msgr2=0x7ffa601109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.697+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 shutdown_connections 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:38.697+0000 7ffa65c00700 1 -- 172.21.15.86:0/399449927 wait complete. 2024-04-25T19:15:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:15:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:38 smithi183 ceph-mon[32344]: pgmap v397: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:15:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:38 smithi194 ceph-mon[32740]: pgmap v397: 97 pgs: 97 active+clean; 1.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T19:15:39.538 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 92.5 --await-for 42.0' 2024-04-25T19:15:39.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 92.5 --await-for 42.0 2024-04-25T19:15:39.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:39 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/399449927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:15:39.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:39 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/399449927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:15:39.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:39 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/399449927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:15:39.994 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:15:40.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:40 smithi086 ceph-mon[27783]: pgmap v398: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T19:15:40.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:40 smithi183 ceph-mon[32344]: pgmap v398: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T19:15:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:40 smithi194 ceph-mon[32740]: pgmap v398: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T19:15:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 -- 172.21.15.86:0/1621971021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288108ab0 msgr2=0x7f9288108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 --2- 172.21.15.86:0/1621971021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288108ab0 0x7f9288108e90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9278009990 tx=0x7f9278009ca0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 -- 172.21.15.86:0/1621971021 shutdown_connections 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 --2- 172.21.15.86:0/1621971021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 0x7f928810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 --2- 172.21.15.86:0/1621971021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288108ab0 0x7f9288108e90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 --2- 172.21.15.86:0/1621971021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9288102ac0 0x7f9288102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 -- 172.21.15.86:0/1621971021 >> 172.21.15.86:0/1621971021 conn(0x7f9288079bf0 msgr2=0x7f928807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 -- 172.21.15.86:0/1621971021 shutdown_connections 2024-04-25T19:15:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.197+0000 7f928ee00700 1 -- 172.21.15.86:0/1621971021 wait complete. 2024-04-25T19:15:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.198+0000 7f928ee00700 1 Processor -- start 2024-04-25T19:15:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 -- start start 2024-04-25T19:15:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288102ac0 0x7f9288074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 0x7f9288075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9288108ab0 0x7f92880757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112c50 con 0x7f9288102ac0 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112ad0 con 0x7f9288108ab0 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ee00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112dd0 con 0x7f9288103480 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f9287e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288102ac0 0x7f9288074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f9287e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288102ac0 0x7f9288074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49090/0 (socket says 172.21.15.86:49090) 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f9287e00700 1 -- 172.21.15.86:0/3440383618 learned_addr learned my addr 172.21.15.86:0/3440383618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f9287400700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 0x7f9288075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.199+0000 7f928ce00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9288108ab0 0x7f92880757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9287400700 1 -- 172.21.15.86:0/3440383618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9288108ab0 msgr2=0x7f92880757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9287400700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9288108ab0 0x7f92880757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9287400700 1 -- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288102ac0 msgr2=0x7f9288074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9287400700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288102ac0 0x7f9288074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9287400700 1 -- 172.21.15.86:0/3440383618 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92881a7610 con 0x7f9288103480 2024-04-25T19:15:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9287400700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 0x7f9288075280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f927400e9f0 tx=0x7f927400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9284c00700 1 -- 172.21.15.86:0/3440383618 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927400c9c0 con 0x7f9288103480 2024-04-25T19:15:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f9284c00700 1 -- 172.21.15.86:0/3440383618 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927400cb60 con 0x7f9288103480 2024-04-25T19:15:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.200+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92881a7900 con 0x7f9288103480 2024-04-25T19:15:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.201+0000 7f9284c00700 1 -- 172.21.15.86:0/3440383618 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274018680 con 0x7f9288103480 2024-04-25T19:15:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.201+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92881a7e40 con 0x7f9288103480 2024-04-25T19:15:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.204+0000 7f9284c00700 1 -- 172.21.15.86:0/3440383618 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9274010430 con 0x7f9288103480 2024-04-25T19:15:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.205+0000 7f92996e5b80 1 Processor -- start 2024-04-25T19:15:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.205+0000 7f92996e5b80 1 -- start start 2024-04-25T19:15:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f9284c00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f926c0a3e50 0x7f926c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c106f0 0x55d384c1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c1b560 0x55d384c1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f92996e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d384b63020 con 0x55d384c106f0 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f92996e5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d384b098c0 con 0x55d384c1b560 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f92996e5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d384b5efe0 con 0x55d384c0fdd0 2024-04-25T19:15:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f9284c00700 1 -- 172.21.15.86:0/3440383618 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f92740d09f0 con 0x7f9288103480 2024-04-25T19:15:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f928ce00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f926c0a3e50 0x7f926c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f9287e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f9287e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c101b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44512/0 (socket says 172.21.15.86:44512) 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f9287e00700 1 -- 172.21.15.86:0/1746697627 learned_addr learned my addr 172.21.15.86:0/1746697627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.206+0000 7f9287400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c106f0 0x55d384c1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f928ce00700 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c1b560 0x55d384c1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f9287e00700 1 -- 172.21.15.86:0/1746697627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c1b560 msgr2=0x55d384c1d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f9287e00700 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c1b560 0x55d384c1d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f9287e00700 1 -- 172.21.15.86:0/1746697627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c106f0 msgr2=0x55d384c1b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f9287e00700 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c106f0 0x55d384c1b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f9287e00700 1 -- 172.21.15.86:0/1746697627 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d384aff810 con 0x55d384c0fdd0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f928ce00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f926c0a3e50 0x7f926c0a6310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9280009e10 tx=0x7f9280011c20 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.207+0000 7f9287e00700 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c101b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9278009740 tx=0x7f9278005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7c59499ba4ca78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f927d600700 1 -- 172.21.15.86:0/1746697627 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278027070 con 0x55d384c0fdd0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f927d600700 1 -- 172.21.15.86:0/1746697627 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92780055d0 con 0x55d384c0fdd0 2024-04-25T19:15:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f927d600700 1 -- 172.21.15.86:0/1746697627 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278003c50 con 0x55d384c0fdd0 2024-04-25T19:15:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f92996e5b80 1 -- 172.21.15.86:0/1746697627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 msgr2=0x55d384c101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c101b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9278009740 tx=0x7f9278005c20 comp rx=0 tx=0).stop 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f92996e5b80 1 -- 172.21.15.86:0/1746697627 shutdown_connections 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.208+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c1b560 0x55d384c1d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c101b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1746697627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c106f0 0x55d384c1b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 -- 172.21.15.86:0/1746697627 >> 172.21.15.86:0/1746697627 conn(0x55d384b3f0e0 msgr2=0x55d384c0c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 -- 172.21.15.86:0/1746697627 shutdown_connections 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 -- 172.21.15.86:0/1746697627 wait complete. 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 -- start start 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.209+0000 7f92996e5b80 1 Processor -- start 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c17f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d384c1dff0 con 0x55d384c1b560 2024-04-25T19:15:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d384b1c9f0 con 0x55d384c0fdd0 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d384c1e170 con 0x55d384c106f0 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f92996e5b80 10 client.0 _ll_register_callbacks cb 0x55d384afb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f928ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f9287400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c17f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f9287400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c17f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44520/0 (socket says 172.21.15.86:44520) 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f9287400700 1 -- 172.21.15.86:0/1488212702 learned_addr learned my addr 172.21.15.86:0/1488212702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.210+0000 7f9287e00700 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9287400700 1 -- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 msgr2=0x55d384c1ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9287400700 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c1ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9287400700 1 -- 172.21.15.86:0/1488212702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 msgr2=0x55d384c184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9287400700 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9287400700 1 -- 172.21.15.86:0/1488212702 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d384ab0a60 con 0x55d384c106f0 2024-04-25T19:15:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9287400700 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c17f70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f927400c5b0 tx=0x7f9274079f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238592f49ed03fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9272000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9272000700 1 -- 172.21.15.86:0/1488212702 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927409cfc0 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f9272000700 1 -- 172.21.15.86:0/1488212702 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f92740870b0 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.211+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92780055d0 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f9272000700 1 -- 172.21.15.86:0/1488212702 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92740872b0 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 10 client.26782 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d384af9c80 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f9272000700 1 -- 172.21.15.86:0/1488212702 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92780055d0 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f9272000700 1 -- 172.21.15.86:0/1488212702 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f927409cb20 con 0x55d384c106f0 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 10 client.26782 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 10 client.26782 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 20 client.26782 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c2ac50 0x55d384c2d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 4 client.26782 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.212+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d384b70810 con 0x55d384c2ac50 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.213+0000 7f9287e00700 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c2ac50 0x55d384c2d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.213+0000 7f9287e00700 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c2ac50 0x55d384c2d070 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606a38c239c15938 server_cookie=1ad682ce86f1d85f in_seq=0 out_seq=0 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.213+0000 7f9272000700 10 client.26782 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.214+0000 7f9272000700 1 -- 172.21.15.86:0/1488212702 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d384b70810 con 0x55d384c2ac50 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.214+0000 7f9272000700 10 client.26782 handle_command_reply: tid=0 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.214+0000 7f92996e5b80 1 client.26782 shutdown 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 msgr2=0x55d384c17f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c17f70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f927400c5b0 tx=0x7f9274079f60 comp rx=0 tx=0).stop 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 shutdown_connections 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c2ac50 0x55d384c2d070 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c1ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 --2- 172.21.15.86:0/1488212702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c17f70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 >> 172.21.15.86:0/1488212702 conn(0x55d384b3f0e0 msgr2=0x55d384c0bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f9272000700 0 client.26782 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 shutdown_connections 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 1 -- 172.21.15.86:0/1488212702 wait complete. 2024-04-25T19:15:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.215+0000 7f92996e5b80 20 client.26782 trim_cache size 0 max 16384 2024-04-25T19:15:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.236+0000 7f92996e5b80 1 Processor -- start 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.236+0000 7f92996e5b80 1 -- start start 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.236+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c16f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.236+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c19e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f92996e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92881af5e0 con 0x55d384c1b560 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f92996e5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92881b1120 con 0x55d384c0fdd0 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f92996e5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92881afcb0 con 0x55d384c106f0 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c0c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49112/0 (socket says 172.21.15.86:49112) 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 -- 172.21.15.86:0/3984379191 learned_addr learned my addr 172.21.15.86:0/3984379191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f9287400700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c19e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f9287e00700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c16f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 -- 172.21.15.86:0/3984379191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 msgr2=0x55d384c19e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c19e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 -- 172.21.15.86:0/3984379191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 msgr2=0x55d384c16f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c16f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f928ce00700 1 -- 172.21.15.86:0/3984379191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d384c2d6d0 con 0x55d384c1b560 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f9287e00700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.237+0000 7f9287400700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c19e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.238+0000 7f928ce00700 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c0c430 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9280011f80 tx=0x7f928000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386c2695270264e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.238+0000 7f927cc00700 1 -- 172.21.15.86:0/3984379191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280029070 con 0x55d384c1b560 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.238+0000 7f927cc00700 1 -- 172.21.15.86:0/3984379191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9280004020 con 0x55d384c1b560 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.238+0000 7f927cc00700 1 -- 172.21.15.86:0/3984379191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280009440 con 0x55d384c1b560 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 -- 172.21.15.86:0/3984379191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 msgr2=0x55d384c0c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c0c430 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9280011f80 tx=0x7f928000b040 comp rx=0 tx=0).stop 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 -- 172.21.15.86:0/3984379191 shutdown_connections 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c0c430 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c0fdd0 0x55d384c16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 --2- 172.21.15.86:0/3984379191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c106f0 0x55d384c19e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 -- 172.21.15.86:0/3984379191 >> 172.21.15.86:0/3984379191 conn(0x55d384b3f0e0 msgr2=0x55d384c0bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 -- 172.21.15.86:0/3984379191 shutdown_connections 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 -- 172.21.15.86:0/3984379191 wait complete. 2024-04-25T19:15:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.239+0000 7f92996e5b80 1 -- start start 2024-04-25T19:15:42.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 Processor -- start 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c8b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c106f0 0x55d384c8bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c207d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92881b0a50 con 0x55d384c1b560 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92881b0380 con 0x55d384c106f0 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92881ad5a0 con 0x55d384c0fdd0 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f92996e5b80 10 client.0 _ll_register_callbacks cb 0x55d384afb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f928ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c207d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f928ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c207d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49114/0 (socket says 172.21.15.86:49114) 2024-04-25T19:15:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f9287400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c106f0 0x55d384c8bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f928ce00700 1 -- 172.21.15.86:0/61549037 learned_addr learned my addr 172.21.15.86:0/61549037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.240+0000 7f9287e00700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c8b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9287400700 1 -- 172.21.15.86:0/61549037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 msgr2=0x55d384c8b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9287400700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c8b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9287400700 1 -- 172.21.15.86:0/61549037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 msgr2=0x55d384c207d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9287400700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c207d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9287400700 1 -- 172.21.15.86:0/61549037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d384c10d70 con 0x55d384c106f0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f928ce00700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c207d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9287400700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c106f0 0x55d384c8bab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f927409ab70 tx=0x7f92740a53c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c96d4d812ba99b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9272000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9272000700 1 -- 172.21.15.86:0/61549037 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740872c0 con 0x55d384c106f0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9280004020 con 0x55d384c106f0 2024-04-25T19:15:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.241+0000 7f9272000700 1 -- 172.21.15.86:0/61549037 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f9274010720 con 0x55d384c106f0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f9272000700 1 -- 172.21.15.86:0/61549037 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92740a0c70 con 0x55d384c106f0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 10 client.26792 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d384c11c40 con 0x55d384c106f0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f9272000700 1 -- 172.21.15.86:0/61549037 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9280004020 con 0x55d384c106f0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f9272000700 1 -- 172.21.15.86:0/61549037 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9274098730 con 0x55d384c106f0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 10 client.26792 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 10 client.26792 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 20 client.26792 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c930b0 0x55d384c954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 4 client.26792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 92.5, "await_for": 42.0}] 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.242+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 92.5, "await_for": 42.0}) -- 0x55d384b6a8b0 con 0x55d384c930b0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.243+0000 7f9287e00700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c930b0 0x55d384c954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.243+0000 7f9287e00700 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c930b0 0x55d384c954d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0b14adc4cccba2 server_cookie=d5f85a7870193c4d in_seq=0 out_seq=0 2024-04-25T19:15:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.243+0000 7f9272000700 10 client.26792 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f9272000700 1 -- 172.21.15.86:0/61549037 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d384b6a8b0 con 0x55d384c930b0 2024-04-25T19:15:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f9272000700 10 client.26792 handle_command_reply: tid=0 2024-04-25T19:15:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 client.26792 shutdown 2024-04-25T19:15:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c106f0 msgr2=0x55d384c8bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c106f0 0x55d384c8bab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f927409ab70 tx=0x7f92740a53c0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 shutdown_connections 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d384c930b0 0x55d384c954d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d384c1b560 0x55d384c207d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d384c0fdd0 0x55d384c8b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 --2- 172.21.15.86:0/61549037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d384c106f0 0x55d384c8bab0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 >> 172.21.15.86:0/61549037 conn(0x55d384b3f0e0 msgr2=0x55d384c0bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f9272000700 0 client.26792 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.258+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 shutdown_connections 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.259+0000 7f92996e5b80 1 -- 172.21.15.86:0/61549037 wait complete. 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.259+0000 7f92996e5b80 20 client.26792 trim_cache size 0 max 16384 2024-04-25T19:15:42.261 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 38, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "c7891a21": { 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 38, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:15:42.262 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 92.5, 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:15:42.263 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f926c0a3e50 msgr2=0x7f926c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f926c0a3e50 0x7f926c0a6310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9280009e10 tx=0x7f9280011c20 comp rx=0 tx=0).stop 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 msgr2=0x7f9288075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 0x7f9288075280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f927400e9f0 tx=0x7f927400ed00 comp rx=0 tx=0).stop 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 shutdown_connections 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9288108ab0 0x7f92880757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9288102ac0 0x7f9288074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f926c0a3e50 0x7f926c0a6310 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 --2- 172.21.15.86:0/3440383618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9288103480 0x7f9288075280 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 >> 172.21.15.86:0/3440383618 conn(0x7f9288079bf0 msgr2=0x7f9288101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 shutdown_connections 2024-04-25T19:15:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:15:42.260+0000 7f928ee00700 1 -- 172.21.15.86:0/3440383618 wait complete. 2024-04-25T19:15:42.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:42 smithi183 ceph-mon[32344]: pgmap v399: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:15:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:42 smithi194 ceph-mon[32740]: pgmap v399: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:15:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:42 smithi086 ceph-mon[27783]: pgmap v399: 97 pgs: 97 active+clean; 1.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:15:43.277 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c7891a21', quiesce took 7.7 seconds. Will release after: 32.5 2024-04-25T19:15:44.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:44 smithi183 ceph-mon[32344]: pgmap v400: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T19:15:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:44 smithi194 ceph-mon[32740]: pgmap v400: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T19:15:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:44 smithi086 ceph-mon[27783]: pgmap v400: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 -- 172.21.15.86:0/3735279826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 msgr2=0x7f1478074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 --2- 172.21.15.86:0/3735279826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f1478074760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f146c009900 tx=0x7f146c02a480 comp rx=0 tx=0).stop 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 -- 172.21.15.86:0/3735279826 shutdown_connections 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 --2- 172.21.15.86:0/3735279826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f147810e5a0 0x7f14781109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 --2- 172.21.15.86:0/3735279826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1478074ca0 0x7f147810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 --2- 172.21.15.86:0/3735279826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f1478074760 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 -- 172.21.15.86:0/3735279826 >> 172.21.15.86:0/3735279826 conn(0x7f14780fd890 msgr2=0x7f14780ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 -- 172.21.15.86:0/3735279826 shutdown_connections 2024-04-25T19:15:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.470+0000 7f147f200640 1 -- 172.21.15.86:0/3735279826 wait complete. 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.471+0000 7f147f200640 1 Processor -- start 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.471+0000 7f147f200640 1 -- start start 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f14781a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 0x7f14781a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f147810e5a0 0x7f147819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1478114220 con 0x7f1478074380 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14781140a0 con 0x7f147810e5a0 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f14781143a0 con 0x7f1478074ca0 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f14781a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 0x7f14781a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f14781a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49126/0 (socket says 172.21.15.86:49126) 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 -- 172.21.15.86:0/3940338596 learned_addr learned my addr 172.21.15.86:0/3940338596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 0x7f14781a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44540/0 (socket says 172.21.15.86:44540) 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147e800640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f147810e5a0 0x7f147819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 -- 172.21.15.86:0/3940338596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 msgr2=0x7f14781a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 0x7f14781a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 -- 172.21.15.86:0/3940338596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f147810e5a0 msgr2=0x7f147819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f147810e5a0 0x7f147819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147de00640 1 -- 172.21.15.86:0/3940338596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147819ada0 con 0x7f1478074380 2024-04-25T19:15:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.472+0000 7f147d400640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 0x7f14781a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.473+0000 7f147de00640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f14781a0420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f146c005ab0 tx=0x7f146c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.473+0000 7f1472a00640 1 -- 172.21.15.86:0/3940338596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c03f070 con 0x7f1478074380 2024-04-25T19:15:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.473+0000 7f1472a00640 1 -- 172.21.15.86:0/3940338596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146c031b60 con 0x7f1478074380 2024-04-25T19:15:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.473+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147819b030 con 0x7f1478074380 2024-04-25T19:15:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.473+0000 7f1472a00640 1 -- 172.21.15.86:0/3940338596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c035030 con 0x7f1478074380 2024-04-25T19:15:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.473+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14781a72e0 con 0x7f1478074380 2024-04-25T19:15:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 Processor -- start 2024-04-25T19:15:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 -- start start 2024-04-25T19:15:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1472a00640 1 -- 172.21.15.86:0/3940338596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f146c042070 con 0x7f1478074380 2024-04-25T19:15:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66828d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e66833650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66835f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e6678e900 con 0x559e66833b90 2024-04-25T19:15:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e66734ef0 con 0x559e668292d0 2024-04-25T19:15:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.478+0000 7f1483576740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559e6678a8c0 con 0x559e668289b0 2024-04-25T19:15:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66828d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66835f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66828d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44556/0 (socket says 172.21.15.86:44556) 2024-04-25T19:15:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 -- 172.21.15.86:0/4225657440 learned_addr learned my addr 172.21.15.86:0/4225657440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147de00640 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e66833650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 -- 172.21.15.86:0/4225657440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 msgr2=0x559e66833650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e66833650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 -- 172.21.15.86:0/4225657440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 msgr2=0x559e66835f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66835f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 -- 172.21.15.86:0/4225657440 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e664a1210 con 0x559e668289b0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.479+0000 7f147e800640 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66828d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1474009f50 tx=0x7f14740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2c5a6a3718bbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.480+0000 7f1472a00640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f14600a3d90 0x7f14600a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.480+0000 7f146b000640 1 -- 172.21.15.86:0/4225657440 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474004d20 con 0x559e668289b0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.480+0000 7f146b000640 1 -- 172.21.15.86:0/4225657440 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1474004050 con 0x559e668289b0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.480+0000 7f146b000640 1 -- 172.21.15.86:0/4225657440 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474005280 con 0x559e668289b0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.480+0000 7f1472a00640 1 -- 172.21.15.86:0/3940338596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f146c0f0eb0 con 0x7f1478074380 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.480+0000 7f147de00640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f14600a3d90 0x7f14600a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 -- 172.21.15.86:0/4225657440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 msgr2=0x559e66828d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f147de00640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f14600a3d90 0x7f14600a6250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f146c0ee640 tx=0x7f146c0f4a00 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66828d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1474009f50 tx=0x7f14740118d0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 -- 172.21.15.86:0/4225657440 shutdown_connections 2024-04-25T19:15:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66835f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e66833650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 --2- 172.21.15.86:0/4225657440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66828d90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 -- 172.21.15.86:0/4225657440 >> 172.21.15.86:0/4225657440 conn(0x559e66824ec0 msgr2=0x559e668252a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 -- 172.21.15.86:0/4225657440 shutdown_connections 2024-04-25T19:15:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 -- 172.21.15.86:0/4225657440 wait complete. 2024-04-25T19:15:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.481+0000 7f1483576740 1 -- start start 2024-04-25T19:15:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 Processor -- start 2024-04-25T19:15:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66830b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 0x559e66831050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e66831590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e668367c0 con 0x559e668292d0 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e66748310 con 0x559e66833b90 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559e66836940 con 0x559e668289b0 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f1483576740 10 client.0 _ll_register_callbacks cb 0x559e66716cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f147e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66830b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f147d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 0x559e66831050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f147e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66830b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44562/0 (socket says 172.21.15.86:44562) 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f147d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 0x559e66831050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49148/0 (socket says 172.21.15.86:49148) 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.482+0000 7f147e800640 1 -- 172.21.15.86:0/890385374 learned_addr learned my addr 172.21.15.86:0/890385374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e66831590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 -- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 msgr2=0x559e66831590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e66831590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 -- 172.21.15.86:0/890385374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 msgr2=0x559e66831050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 0x559e66831050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 -- 172.21.15.86:0/890385374 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6656d620 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147d400640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 0x559e66831050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f147e800640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66830b10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1474009250 tx=0x7f1474005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eea19fb7af8ffa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f145f400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.483+0000 7f145f400640 1 -- 172.21.15.86:0/890385374 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147402b020 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.484+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1474004050 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.484+0000 7f145f400640 1 -- 172.21.15.86:0/890385374 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f1474025480 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.484+0000 7f145f400640 1 -- 172.21.15.86:0/890385374 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1474004510 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.484+0000 7f1483576740 10 client.26800 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.484+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e66842a40 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.484+0000 7f145f400640 1 -- 172.21.15.86:0/890385374 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1474004050 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f145f400640 1 -- 172.21.15.86:0/890385374 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1474027740 con 0x559e668289b0 2024-04-25T19:15:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f1483576740 10 client.26800 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f1483576740 10 client.26800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f1483576740 20 client.26800 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f1483576740 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e66843e50 0x559e66846240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f1483576740 4 client.26800 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e667a28b0 con 0x559e66843e50 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.485+0000 7f147d400640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e66843e50 0x559e66846240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.486+0000 7f147d400640 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e66843e50 0x559e66846240 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9574b08c6a1552f6 server_cookie=1db4d4419ee38134 in_seq=0 out_seq=0 2024-04-25T19:15:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.486+0000 7f145f400640 10 client.26800 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f145f400640 1 -- 172.21.15.86:0/890385374 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e667a28b0 con 0x559e66843e50 2024-04-25T19:15:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f145f400640 10 client.26800 handle_command_reply: tid=0 2024-04-25T19:15:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 client.26800 shutdown 2024-04-25T19:15:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 msgr2=0x559e66830b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66830b10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1474009250 tx=0x7f1474005bd0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 shutdown_connections 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e66843e50 0x559e66846240 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e66831590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668292d0 0x559e66831050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f145f400640 0 client.26800 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 --2- 172.21.15.86:0/890385374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e66830b10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 >> 172.21.15.86:0/890385374 conn(0x559e66824ec0 msgr2=0x559e6682abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 shutdown_connections 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 1 -- 172.21.15.86:0/890385374 wait complete. 2024-04-25T19:15:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.487+0000 7f1483576740 20 client.26800 trim_cache size 0 max 16384 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.503+0000 7f1483576740 1 Processor -- start 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 -- start start 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e6683f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e668313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66823ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14781afed0 con 0x559e66833b90 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14781b1a10 con 0x559e668292d0 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f1483576740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f14781b05a0 con 0x559e668289b0 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f147e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66823ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f147e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66823ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49150/0 (socket says 172.21.15.86:49150) 2024-04-25T19:15:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f147e800640 1 -- 172.21.15.86:0/4054029335 learned_addr learned my addr 172.21.15.86:0/4054029335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f147e800640 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e6683f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.504+0000 7f147d400640 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e668313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147e800640 1 -- 172.21.15.86:0/4054029335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 msgr2=0x559e6683f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147e800640 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e6683f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147e800640 1 -- 172.21.15.86:0/4054029335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 msgr2=0x559e668313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147e800640 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e668313b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147e800640 1 -- 172.21.15.86:0/4054029335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e66820820 con 0x559e66833b90 2024-04-25T19:15:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147d400640 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e668313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f147e800640 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66823ea0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1474005d70 tx=0x7f14740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485ad83866e8baa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f146a600640 1 -- 172.21.15.86:0/4054029335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147402b040 con 0x559e66833b90 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f146a600640 1 -- 172.21.15.86:0/4054029335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1474038040 con 0x559e66833b90 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.505+0000 7f146a600640 1 -- 172.21.15.86:0/4054029335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400fb40 con 0x559e66833b90 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 -- 172.21.15.86:0/4054029335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 msgr2=0x559e66823ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66823ea0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1474005d70 tx=0x7f14740118d0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 -- 172.21.15.86:0/4054029335 shutdown_connections 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e66833b90 0x559e66823ea0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e668292d0 0x559e668313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 --2- 172.21.15.86:0/4054029335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668289b0 0x559e6683f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 -- 172.21.15.86:0/4054029335 >> 172.21.15.86:0/4054029335 conn(0x559e66824ec0 msgr2=0x559e66823790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 -- 172.21.15.86:0/4054029335 shutdown_connections 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.506+0000 7f1483576740 1 -- 172.21.15.86:0/4054029335 wait complete. 2024-04-25T19:15:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 -- start start 2024-04-25T19:15:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 Processor -- start 2024-04-25T19:15:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 0x559e6683bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668292d0 0x559e66839180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e668396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14781b1340 con 0x559e668289b0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14781b0c70 con 0x559e66833b90 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f14781aded0 con 0x559e668292d0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f1483576740 10 client.0 _ll_register_callbacks cb 0x7f1474009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.507+0000 7f147e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 0x559e6683bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 0x559e6683bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49164/0 (socket says 172.21.15.86:49164) 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147e800640 1 -- 172.21.15.86:0/3374886613 learned_addr learned my addr 172.21.15.86:0/3374886613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668292d0 0x559e66839180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147e800640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e668396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 -- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 msgr2=0x559e668396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e668396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 -- 172.21.15.86:0/3374886613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 msgr2=0x559e6683bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 0x559e6683bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 -- 172.21.15.86:0/3374886613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e66839c00 con 0x559e668292d0 2024-04-25T19:15:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147e800640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 0x559e6683bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f147d400640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668292d0 0x559e66839180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x559e66830370 tx=0x7f146401f7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82aff029ceeb7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.508+0000 7f145f400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f145f400640 1 -- 172.21.15.86:0/3374886613 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464005220 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1474038040 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f145f400640 1 -- 172.21.15.86:0/3374886613 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7f14640053c0 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f145f400640 1 -- 172.21.15.86:0/3374886613 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1464030050 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 10 client.26806 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e668a9ef0 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f145f400640 1 -- 172.21.15.86:0/3374886613 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1474038040 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f145f400640 1 -- 172.21.15.86:0/3374886613 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1464022310 con 0x559e668292d0 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 10 client.26806 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 10 client.26806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 20 client.26806 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e668aa660 0x559e668aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 4 client.26806 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.509+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e6679c320 con 0x559e668aa660 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.510+0000 7f147e800640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e668aa660 0x559e668aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.510+0000 7f147e800640 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e668aa660 0x559e668aca80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a517ded6eff9ddc server_cookie=2ee05a33db03626a in_seq=0 out_seq=0 2024-04-25T19:15:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.510+0000 7f145f400640 10 client.26806 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.524+0000 7f145f400640 1 -- 172.21.15.86:0/3374886613 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e6679c320 con 0x559e668aa660 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.524+0000 7f145f400640 10 client.26806 handle_command_reply: tid=0 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.524+0000 7f1483576740 1 client.26806 shutdown 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668292d0 msgr2=0x559e66839180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668292d0 0x559e66839180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x559e66830370 tx=0x7f146401f7c0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 shutdown_connections 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559e668aa660 0x559e668aca80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e66833b90 0x559e668396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559e668292d0 0x559e66839180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 --2- 172.21.15.86:0/3374886613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e668289b0 0x559e6683bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 >> 172.21.15.86:0/3374886613 conn(0x559e66824ec0 msgr2=0x559e66824020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f145f400640 0 client.26806 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 shutdown_connections 2024-04-25T19:15:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 1 -- 172.21.15.86:0/3374886613 wait complete. 2024-04-25T19:15:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.525+0000 7f1483576740 20 client.26806 trim_cache size 0 max 16384 2024-04-25T19:15:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:15:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:15:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:15:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f14600a3d90 msgr2=0x7f14600a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f14600a3d90 0x7f14600a6250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f146c0ee640 tx=0x7f146c0f4a00 comp rx=0 tx=0).stop 2024-04-25T19:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 msgr2=0x7f14781a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f14781a0420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f146c005ab0 tx=0x7f146c003c10 comp rx=0 tx=0).stop 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 shutdown_connections 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f14600a3d90 0x7f14600a6250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f147810e5a0 0x7f147819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1478074ca0 0x7f14781a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 --2- 172.21.15.86:0/3940338596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1478074380 0x7f14781a0420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.526+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 >> 172.21.15.86:0/3940338596 conn(0x7f14780fd890 msgr2=0x7f14780ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.527+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 shutdown_connections 2024-04-25T19:15:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:45.527+0000 7f147f200640 1 -- 172.21.15.86:0/3940338596 wait complete. 2024-04-25T19:15:46.892 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:46 smithi086 ceph-mon[27783]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:15:46.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:46 smithi183 ceph-mon[32344]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:15:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:46 smithi194 ceph-mon[32740]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:15:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:48 smithi183 ceph-mon[32344]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:15:48.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:48 smithi194 ceph-mon[32740]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:15:49.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:48 smithi086 ceph-mon[27783]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:15:50.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:50 smithi183 ceph-mon[32344]: pgmap v403: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T19:15:50.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:50 smithi194 ceph-mon[32740]: pgmap v403: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T19:15:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:50 smithi086 ceph-mon[27783]: pgmap v403: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T19:15:52.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:52 smithi183 ceph-mon[32344]: pgmap v404: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T19:15:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:52 smithi194 ceph-mon[32740]: pgmap v404: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T19:15:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:52 smithi086 ceph-mon[27783]: pgmap v404: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-25T19:15:54.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:54 smithi183 ceph-mon[32344]: pgmap v405: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-25T19:15:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:54 smithi194 ceph-mon[32740]: pgmap v405: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-25T19:15:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:54 smithi086 ceph-mon[27783]: pgmap v405: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-25T19:15:56.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:56 smithi183 ceph-mon[32344]: pgmap v406: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-25T19:15:56.944 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:56 smithi086 ceph-mon[27783]: pgmap v406: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-25T19:15:56.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:56 smithi194 ceph-mon[32740]: pgmap v406: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-25T19:15:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 -- 172.21.15.86:0/1067492408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb8010c590 msgr2=0x7fdb8010e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 --2- 172.21.15.86:0/1067492408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb8010c590 0x7fdb8010e9e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009ca0 tx=0x7fdb7802ed60 comp rx=0 tx=0).stop 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 -- 172.21.15.86:0/1067492408 shutdown_connections 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 --2- 172.21.15.86:0/1067492408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb8010c590 0x7fdb8010e9e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 --2- 172.21.15.86:0/1067492408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb80103ae0 0x7fdb8010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 --2- 172.21.15.86:0/1067492408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb801031c0 0x7fdb801035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 -- 172.21.15.86:0/1067492408 >> 172.21.15.86:0/1067492408 conn(0x7fdb80077fd0 msgr2=0x7fdb800783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 -- 172.21.15.86:0/1067492408 shutdown_connections 2024-04-25T19:15:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.670+0000 7fdb84a00640 1 -- 172.21.15.86:0/1067492408 wait complete. 2024-04-25T19:15:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 Processor -- start 2024-04-25T19:15:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 -- start start 2024-04-25T19:15:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb801031c0 0x7fdb80196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb80103ae0 0x7fdb801969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 0x7fdb8019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010ffc0 con 0x7fdb8010c590 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010fe40 con 0x7fdb80103ae0 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.671+0000 7fdb84a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80110140 con 0x7fdb801031c0 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 0x7fdb8019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 0x7fdb8019acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51356/0 (socket says 172.21.15.86:51356) 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 -- 172.21.15.86:0/3030130691 learned_addr learned my addr 172.21.15.86:0/3030130691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7de00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb801031c0 0x7fdb80196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7d400640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb80103ae0 0x7fdb801969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 -- 172.21.15.86:0/3030130691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb801031c0 msgr2=0x7fdb80196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb801031c0 0x7fdb80196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 -- 172.21.15.86:0/3030130691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb80103ae0 msgr2=0x7fdb801969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb80103ae0 0x7fdb801969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7e800640 1 -- 172.21.15.86:0/3030130691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8019b420 con 0x7fdb8010c590 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.672+0000 7fdb7de00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb801031c0 0x7fdb80196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.673+0000 7fdb7e800640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 0x7fdb8019acf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb7802f350 tx=0x7fdb7802fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.673+0000 7fdb76a00640 1 -- 172.21.15.86:0/3030130691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78044020 con 0x7fdb8010c590 2024-04-25T19:15:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.673+0000 7fdb76a00640 1 -- 172.21.15.86:0/3030130691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb780046f0 con 0x7fdb8010c590 2024-04-25T19:15:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.673+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8019b6b0 con 0x7fdb8010c590 2024-04-25T19:15:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.673+0000 7fdb76a00640 1 -- 172.21.15.86:0/3030130691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7803c040 con 0x7fdb8010c590 2024-04-25T19:15:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.673+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a2fa0 con 0x7fdb8010c590 2024-04-25T19:15:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.676+0000 7fdb76a00640 1 -- 172.21.15.86:0/3030130691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb78047020 con 0x7fdb8010c590 2024-04-25T19:15:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.676+0000 7fdb8783d740 1 Processor -- start 2024-04-25T19:15:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 -- start start 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221495ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x56221496bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622148c4900 con 0x56221495e820 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56221486aef0 con 0x56221495f140 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb8783d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5622148c08c0 con 0x562214969a00 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221495ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221495ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51368/0 (socket says 172.21.15.86:51368) 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb7d400640 1 -- 172.21.15.86:0/3687994862 learned_addr learned my addr 172.21.15.86:0/3687994862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb7de00640 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.677+0000 7fdb7d400640 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x56221496bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb76a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdb640a3dc0 0x7fdb640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7de00640 1 -- 172.21.15.86:0/3687994862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 msgr2=0x56221496bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7de00640 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x56221496bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7de00640 1 -- 172.21.15.86:0/3687994862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 msgr2=0x56221495ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7de00640 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221495ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7de00640 1 -- 172.21.15.86:0/3687994862 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622145d7210 con 0x56221495f140 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb76a00640 1 -- 172.21.15.86:0/3030130691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7fdb780f4fb0 con 0x7fdb8010c590 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7e800640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdb640a3dc0 0x7fdb640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.678+0000 7fdb7de00640 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149694c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009fd0 tx=0x7fdb700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6dad64f36ef182d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb6b000640 1 -- 172.21.15.86:0/3687994862 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70004310 con 0x56221495f140 2024-04-25T19:15:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb6b000640 1 -- 172.21.15.86:0/3687994862 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb700044b0 con 0x56221495f140 2024-04-25T19:15:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb6b000640 1 -- 172.21.15.86:0/3687994862 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700035f0 con 0x56221495f140 2024-04-25T19:15:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb8783d740 1 -- 172.21.15.86:0/3687994862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 msgr2=0x5622149694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb8783d740 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149694c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009fd0 tx=0x7fdb700118d0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb7e800640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdb640a3dc0 0x7fdb640a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb780bd3f0 tx=0x7fdb78003f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb8783d740 1 -- 172.21.15.86:0/3687994862 shutdown_connections 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb8783d740 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x56221496bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb8783d740 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149694c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.679+0000 7fdb8783d740 1 --2- 172.21.15.86:0/3687994862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221495ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.680+0000 7fdb8783d740 1 -- 172.21.15.86:0/3687994862 >> 172.21.15.86:0/3687994862 conn(0x56221495ad30 msgr2=0x56221495b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.680+0000 7fdb8783d740 1 -- 172.21.15.86:0/3687994862 shutdown_connections 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.680+0000 7fdb8783d740 1 -- 172.21.15.86:0/3687994862 wait complete. 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.680+0000 7fdb8783d740 1 -- start start 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.680+0000 7fdb8783d740 1 Processor -- start 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495e820 0x562214966970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 0x562214966eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x5622149673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56221496c630 con 0x56221495f140 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56221487e310 con 0x562214969a00 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56221496c7b0 con 0x56221495e820 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb8783d740 10 client.0 _ll_register_callbacks cb 0x56221484ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 0x562214966eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 0x562214966eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51380/0 (socket says 172.21.15.86:51380) 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 -- 172.21.15.86:0/1190724737 learned_addr learned my addr 172.21.15.86:0/1190724737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7d400640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495e820 0x562214966970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7d400640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x5622149673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 -- 172.21.15.86:0/1190724737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495e820 msgr2=0x562214966970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495e820 0x562214966970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 -- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 msgr2=0x5622149673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x5622149673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.681+0000 7fdb7de00640 1 -- 172.21.15.86:0/1190724737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622146a3620 con 0x56221495f140 2024-04-25T19:15:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb7de00640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 0x562214966eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdb7001b040 tx=0x7fdb700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f4ed2a5a2b0cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb5f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb5f400640 1 -- 172.21.15.86:0/1190724737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70029070 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb5f400640 1 -- 172.21.15.86:0/1190724737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7fdb70003a00 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb700044b0 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb5f400640 1 -- 172.21.15.86:0/1190724737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb70035050 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb8783d740 10 client.16722 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.682+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622149788b0 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb5f400640 1 -- 172.21.15.86:0/1190724737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb700044b0 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb5f400640 1 -- 172.21.15.86:0/1190724737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdb70023430 con 0x56221495f140 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb8783d740 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb8783d740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb8783d740 20 client.16722 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb8783d740 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562214979c60 0x56221497c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb8783d740 4 client.16722 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.683+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622148d88b0 con 0x562214979c60 2024-04-25T19:15:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.684+0000 7fdb7d400640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562214979c60 0x56221497c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.684+0000 7fdb7d400640 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562214979c60 0x56221497c050 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e27c5a2f1a3e77a server_cookie=5f8816ea411207e0 in_seq=0 out_seq=0 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.684+0000 7fdb5f400640 10 client.16722 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb5f400640 1 -- 172.21.15.86:0/1190724737 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622148d88b0 con 0x562214979c60 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb5f400640 10 client.16722 handle_command_reply: tid=0 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 client.16722 shutdown 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 msgr2=0x562214966eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 0x562214966eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdb7001b040 tx=0x7fdb700048f0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 shutdown_connections 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x562214979c60 0x56221497c050 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x5622149673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495f140 0x562214966eb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 --2- 172.21.15.86:0/1190724737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495e820 0x562214966970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb5f400640 0 client.16722 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 >> 172.21.15.86:0/1190724737 conn(0x56221495ad30 msgr2=0x562214960ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 shutdown_connections 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.685+0000 7fdb8783d740 1 -- 172.21.15.86:0/1190724737 wait complete. 2024-04-25T19:15:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.686+0000 7fdb8783d740 20 client.16722 trim_cache size 0 max 16384 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 Processor -- start 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 -- start start 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x5622149754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x562214981800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801aba40 con 0x56221495e820 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ad580 con 0x56221495f140 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb8783d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ac110 con 0x562214969a00 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x5622149754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x5622149754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51392/0 (socket says 172.21.15.86:51392) 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb7d400640 1 -- 172.21.15.86:0/504277823 learned_addr learned my addr 172.21.15.86:0/504277823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.702+0000 7fdb7de00640 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x562214981800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 -- 172.21.15.86:0/504277823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 msgr2=0x562214981800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x562214981800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 -- 172.21.15.86:0/504277823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 msgr2=0x5622149608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 -- 172.21.15.86:0/504277823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562214964ec0 con 0x56221495e820 2024-04-25T19:15:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb7d400640 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x5622149754f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0095f0 tx=0x7fdb6c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda8cbfa132febe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb6a600640 1 -- 172.21.15.86:0/504277823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c027170 con 0x56221495e820 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb6a600640 1 -- 172.21.15.86:0/504277823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c0277c0 con 0x56221495e820 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.703+0000 7fdb6a600640 1 -- 172.21.15.86:0/504277823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c003e30 con 0x56221495e820 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 -- 172.21.15.86:0/504277823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 msgr2=0x5622149754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x5622149754f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0095f0 tx=0x7fdb6c005f90 comp rx=0 tx=0).stop 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 -- 172.21.15.86:0/504277823 shutdown_connections 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562214969a00 0x562214981800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56221495f140 0x5622149608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 --2- 172.21.15.86:0/504277823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x5622149754f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 -- 172.21.15.86:0/504277823 >> 172.21.15.86:0/504277823 conn(0x56221495ad30 msgr2=0x5622149601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 -- 172.21.15.86:0/504277823 shutdown_connections 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.704+0000 7fdb8783d740 1 -- 172.21.15.86:0/504277823 wait complete. 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 -- start start 2024-04-25T19:15:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 Processor -- start 2024-04-25T19:15:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221496f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495f140 0x56221496fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x562214970000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801aceb0 con 0x56221495e820 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ac7e0 con 0x562214969a00 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801a9a40 con 0x56221495f140 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.705+0000 7fdb8783d740 10 client.0 _ll_register_callbacks cb 0x7fdb70009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221496f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221496f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51400/0 (socket says 172.21.15.86:51400) 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 -- 172.21.15.86:0/2460368427 learned_addr learned my addr 172.21.15.86:0/2460368427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7de00640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495f140 0x56221496fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x562214970000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 -- 172.21.15.86:0/2460368427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495f140 msgr2=0x56221496fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495f140 0x56221496fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 -- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 msgr2=0x562214970000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x562214970000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 -- 172.21.15.86:0/2460368427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622149df1e0 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb7d400640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221496f580 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0099a0 tx=0x7fdb6c009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9fd138d8430584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.706+0000 7fdb5f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb5f400640 1 -- 172.21.15.86:0/2460368427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c027e60 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb5f400640 1 -- 172.21.15.86:0/2460368427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8637+0+0 (secure 0 0 0) 0x7fdb6c0036f0 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6c0277c0 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb5f400640 1 -- 172.21.15.86:0/2460368427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb6c003bb0 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622149e04d0 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb5f400640 1 -- 172.21.15.86:0/2460368427 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c0277c0 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb5f400640 1 -- 172.21.15.86:0/2460368427 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdb6c005370 con 0x56221495e820 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:15:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 20 client.16728 populate_metadata read hostname 'smithi086' 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5622149e0b40 0x5622149e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 4 client.16728 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.707+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622148d2320 con 0x5622149e0b40 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.708+0000 7fdb7de00640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5622149e0b40 0x5622149e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.708+0000 7fdb7de00640 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5622149e0b40 0x5622149e2f30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5303c31eee3ce6 server_cookie=500fb3b72d8587c in_seq=0 out_seq=0 2024-04-25T19:15:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.708+0000 7fdb5f400640 10 client.16728 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb5f400640 1 -- 172.21.15.86:0/2460368427 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622148d2320 con 0x5622149e0b40 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb5f400640 10 client.16728 handle_command_reply: tid=0 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 client.16728 shutdown 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 msgr2=0x56221496f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221496f580 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0099a0 tx=0x7fdb6c009930 comp rx=0 tx=0).stop 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 shutdown_connections 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5622149e0b40 0x5622149e2f30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562214969a00 0x562214970000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb5f400640 0 client.16728 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56221495f140 0x56221496fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 --2- 172.21.15.86:0/2460368427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56221495e820 0x56221496f580 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 >> 172.21.15.86:0/2460368427 conn(0x56221495ad30 msgr2=0x562214960a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 shutdown_connections 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 1 -- 172.21.15.86:0/2460368427 wait complete. 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.721+0000 7fdb8783d740 20 client.16728 trim_cache size 0 max 16384 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:15:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:15:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:15:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdb640a3dc0 msgr2=0x7fdb640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdb640a3dc0 0x7fdb640a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb780bd3f0 tx=0x7fdb78003f90 comp rx=0 tx=0).stop 2024-04-25T19:15:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 msgr2=0x7fdb8019acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 0x7fdb8019acf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb7802f350 tx=0x7fdb7802fe20 comp rx=0 tx=0).stop 2024-04-25T19:15:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 shutdown_connections 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fdb640a3dc0 0x7fdb640a6280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb8010c590 0x7fdb8019acf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb80103ae0 0x7fdb801969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 --2- 172.21.15.86:0/3030130691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb801031c0 0x7fdb80196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 >> 172.21.15.86:0/3030130691 conn(0x7fdb80077fd0 msgr2=0x7fdb800fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 shutdown_connections 2024-04-25T19:15:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:15:58.722+0000 7fdb84a00640 1 -- 172.21.15.86:0/3030130691 wait complete. 2024-04-25T19:15:58.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:15:58 smithi183 ceph-mon[32344]: pgmap v407: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-25T19:15:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:15:58 smithi194 ceph-mon[32740]: pgmap v407: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-25T19:15:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:15:58 smithi086 ceph-mon[27783]: pgmap v407: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-04-25T19:16:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:00 smithi194 ceph-mon[32740]: pgmap v408: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-04-25T19:16:01.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:00 smithi086 ceph-mon[27783]: pgmap v408: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-04-25T19:16:01.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:00 smithi183 ceph-mon[32344]: pgmap v408: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-04-25T19:16:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:02 smithi194 ceph-mon[32740]: pgmap v409: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T19:16:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:02 smithi086 ceph-mon[27783]: pgmap v409: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T19:16:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:02 smithi183 ceph-mon[32344]: pgmap v409: 97 pgs: 97 active+clean; 2.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T19:16:04.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:04 smithi194 ceph-mon[32740]: pgmap v410: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-04-25T19:16:05.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:04 smithi086 ceph-mon[27783]: pgmap v410: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-04-25T19:16:05.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:04 smithi183 ceph-mon[32344]: pgmap v410: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-04-25T19:16:06.969 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:06 smithi183 ceph-mon[32344]: pgmap v411: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-04-25T19:16:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:06 smithi194 ceph-mon[32740]: pgmap v411: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-04-25T19:16:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:06 smithi086 ceph-mon[27783]: pgmap v411: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-04-25T19:16:07.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:07 smithi194 ceph-mon[32740]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T19:16:08.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:07 smithi086 ceph-mon[27783]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T19:16:08.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:07 smithi183 ceph-mon[32344]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T19:16:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:08 smithi194 ceph-mon[32740]: pgmap v413: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-25T19:16:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:08 smithi086 ceph-mon[27783]: pgmap v413: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-25T19:16:09.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:08 smithi183 ceph-mon[32344]: pgmap v413: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-25T19:16:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:10 smithi194 ceph-mon[32740]: pgmap v414: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-25T19:16:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:10 smithi086 ceph-mon[27783]: pgmap v414: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-25T19:16:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:10 smithi183 ceph-mon[32344]: pgmap v414: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-25T19:16:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 -- 172.21.15.86:0/1461343131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 msgr2=0x7febf0103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 --2- 172.21.15.86:0/1461343131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf0103310 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7febe0009ca0 tx=0x7febe002a950 comp rx=0 tx=0).stop 2024-04-25T19:16:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 -- 172.21.15.86:0/1461343131 shutdown_connections 2024-04-25T19:16:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 --2- 172.21.15.86:0/1461343131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 0x7febf010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 --2- 172.21.15.86:0/1461343131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf0103850 0x7febf010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 --2- 172.21.15.86:0/1461343131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf0103310 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 -- 172.21.15.86:0/1461343131 >> 172.21.15.86:0/1461343131 conn(0x7febf00fc770 msgr2=0x7febf00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 -- 172.21.15.86:0/1461343131 shutdown_connections 2024-04-25T19:16:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.863+0000 7febefe00640 1 -- 172.21.15.86:0/1461343131 wait complete. 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.864+0000 7febefe00640 1 Processor -- start 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.864+0000 7febefe00640 1 -- start start 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febefe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf0103850 0x7febf01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febefe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 0x7febf019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febefe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112010 con 0x7febf0102f30 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febefe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febf0111e90 con 0x7febf0103850 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febefe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112190 con 0x7febf010c350 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febef400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 0x7febf019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febee000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf0103850 0x7febf01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febef400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 0x7febf019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34128/0 (socket says 172.21.15.86:34128) 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febef400640 1 -- 172.21.15.86:0/3775875707 learned_addr learned my addr 172.21.15.86:0/3775875707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febeea00640 1 -- 172.21.15.86:0/3775875707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 msgr2=0x7febf019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febeea00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 0x7febf019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febeea00640 1 -- 172.21.15.86:0/3775875707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf0103850 msgr2=0x7febf01a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febeea00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf0103850 0x7febf01a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.865+0000 7febeea00640 1 -- 172.21.15.86:0/3775875707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf019ad50 con 0x7febf0102f30 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.866+0000 7febeea00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf01a0420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7febe0005e30 tx=0x7febe0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.866+0000 7febe7400640 1 -- 172.21.15.86:0/3775875707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0034040 con 0x7febf0102f30 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.866+0000 7febe7400640 1 -- 172.21.15.86:0/3775875707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe0030850 con 0x7febf0102f30 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.866+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf019afe0 con 0x7febf0102f30 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.866+0000 7febe7400640 1 -- 172.21.15.86:0/3775875707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0004760 con 0x7febf0102f30 2024-04-25T19:16:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.866+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf01a72e0 con 0x7febf0102f30 2024-04-25T19:16:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 Processor -- start 2024-04-25T19:16:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 -- start start 2024-04-25T19:16:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febe7400640 1 -- 172.21.15.86:0/3775875707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febe0037020 con 0x7febf0102f30 2024-04-25T19:16:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d59b0 0x5624c55d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 0x5624c55e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55e0b90 0x5624c55e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5624c553b900 con 0x5624c55d62d0 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624c54e1ef0 con 0x5624c55e0b90 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febf8b94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5624c55378c0 con 0x5624c55d59b0 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 0x5624c55e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 0x5624c55e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45230/0 (socket says 172.21.15.86:45230) 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.870+0000 7febeea00640 1 -- 172.21.15.86:0/2492610515 learned_addr learned my addr 172.21.15.86:0/2492610515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febef400640 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d59b0 0x5624c55d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febee000640 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55e0b90 0x5624c55e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febeea00640 1 -- 172.21.15.86:0/2492610515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d59b0 msgr2=0x5624c55d5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febeea00640 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d59b0 0x5624c55d5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febeea00640 1 -- 172.21.15.86:0/2492610515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55e0b90 msgr2=0x5624c55e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febeea00640 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55e0b90 0x5624c55e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febeea00640 1 -- 172.21.15.86:0/2492610515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c524e210 con 0x5624c55d62d0 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.871+0000 7febeea00640 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 0x5624c55e0650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7febe00eebf0 tx=0x7febe00f7730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8429a63f041e9e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febe7400640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7febd40a3d90 0x7febd40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febdfe00640 1 -- 172.21.15.86:0/2492610515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00f1d20 con 0x5624c55d62d0 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febdfe00640 1 -- 172.21.15.86:0/2492610515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe00fe030 con 0x5624c55d62d0 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febdfe00640 1 -- 172.21.15.86:0/2492610515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00a1ae0 con 0x5624c55d62d0 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febe7400640 1 -- 172.21.15.86:0/3775875707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8681+0+0 (secure 0 0 0) 0x7febe00f1520 con 0x7febf0102f30 2024-04-25T19:16:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febef400640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7febd40a3d90 0x7febd40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febf8b94740 1 -- 172.21.15.86:0/2492610515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 msgr2=0x5624c55e0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.872+0000 7febf8b94740 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 0x5624c55e0650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7febe00eebf0 tx=0x7febe00f7730 comp rx=0 tx=0).stop 2024-04-25T19:16:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febef400640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7febd40a3d90 0x7febd40a6250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7febe8005e00 tx=0x7febe800b420 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 -- 172.21.15.86:0/2492610515 shutdown_connections 2024-04-25T19:16:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55e0b90 0x5624c55e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d62d0 0x5624c55e0650 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 --2- 172.21.15.86:0/2492610515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d59b0 0x5624c55d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 -- 172.21.15.86:0/2492610515 >> 172.21.15.86:0/2492610515 conn(0x5624c55d1ec0 msgr2=0x5624c55d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 -- 172.21.15.86:0/2492610515 shutdown_connections 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 -- 172.21.15.86:0/2492610515 wait complete. 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 -- start start 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.873+0000 7febf8b94740 1 Processor -- start 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55de490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5624c55e3580 con 0x5624c55d59b0 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624c54f5310 con 0x5624c55d62d0 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5624c55e3700 con 0x5624c55e0b90 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febf8b94740 10 client.0 _ll_register_callbacks cb 0x5624c54c3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febeea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55dda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45242/0 (socket says 172.21.15.86:45242) 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febeea00640 1 -- 172.21.15.86:0/2584749008 learned_addr learned my addr 172.21.15.86:0/2584749008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febee000640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febef400640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55de490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febeea00640 1 -- 172.21.15.86:0/2584749008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 msgr2=0x5624c55de490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.874+0000 7febeea00640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55de490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febeea00640 1 -- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 msgr2=0x5624c55ddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febeea00640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55ddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febeea00640 1 -- 172.21.15.86:0/2584749008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c531a620 con 0x5624c55d59b0 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febef400640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55de490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febeea00640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55dda10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7febe00a6a10 tx=0x7febe0030a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809db527e14be363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febdcc00640 1 -- 172.21.15.86:0/2584749008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00a9720 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febdcc00640 1 -- 172.21.15.86:0/2584749008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8681+0+0 (secure 0 0 0) 0x7febe00bb7d0 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe00fe030 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febdcc00640 1 -- 172.21.15.86:0/2584749008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7febe00a99b0 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febf8b94740 10 client.16740 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.875+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624c55e3580 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febdcc00640 1 -- 172.21.15.86:0/2584749008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe00fe030 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febdcc00640 1 -- 172.21.15.86:0/2584749008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7febe004d030 con 0x5624c55d59b0 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febf8b94740 10 client.16740 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febf8b94740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febf8b94740 20 client.16740 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febf8b94740 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c55f0960 0x5624c55f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febf8b94740 4 client.16740 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c554f8b0 con 0x5624c55f0960 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.876+0000 7febee000640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c55f0960 0x5624c55f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.877+0000 7febdcc00640 10 client.16740 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.877+0000 7febee000640 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c55f0960 0x5624c55f2d50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e82d6f5c5d4733f server_cookie=873def784973f6f4 in_seq=0 out_seq=0 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febdcc00640 1 -- 172.21.15.86:0/2584749008 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624c554f8b0 con 0x5624c55f0960 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febdcc00640 10 client.16740 handle_command_reply: tid=0 2024-04-25T19:16:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 client.16740 shutdown 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 msgr2=0x5624c55dda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55dda10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7febe00a6a10 tx=0x7febe0030a00 comp rx=0 tx=0).stop 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 shutdown_connections 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c55f0960 0x5624c55f2d50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55de490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55ddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 --2- 172.21.15.86:0/2584749008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55dda10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febdcc00640 0 client.16740 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 >> 172.21.15.86:0/2584749008 conn(0x5624c55d1ec0 msgr2=0x5624c55d7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 shutdown_connections 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.878+0000 7febf8b94740 1 -- 172.21.15.86:0/2584749008 wait complete. 2024-04-25T19:16:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.879+0000 7febf8b94740 20 client.16740 trim_cache size 0 max 16384 2024-04-25T19:16:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.894+0000 7febf8b94740 1 Processor -- start 2024-04-25T19:16:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 -- start start 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febf01afd60 con 0x5624c55d59b0 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febf01b18a0 con 0x5624c55d62d0 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febf8b94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febf01b0430 con 0x5624c55e0b90 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febef400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febef400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34166/0 (socket says 172.21.15.86:34166) 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febee000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.895+0000 7febef400640 1 -- 172.21.15.86:0/3637612389 learned_addr learned my addr 172.21.15.86:0/3637612389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febeea00640 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febef400640 1 -- 172.21.15.86:0/3637612389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 msgr2=0x5624c55dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febef400640 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55dd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febef400640 1 -- 172.21.15.86:0/3637612389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 msgr2=0x5624c55ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febef400640 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febef400640 1 -- 172.21.15.86:0/3637612389 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c55cd820 con 0x5624c55e0b90 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febee000640 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febeea00640 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.896+0000 7febef400640 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55d0ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7febe800bc10 tx=0x7febe8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=507fc2ada58e67d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.897+0000 7febdf400640 1 -- 172.21.15.86:0/3637612389 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe802d070 con 0x5624c55e0b90 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.897+0000 7febdf400640 1 -- 172.21.15.86:0/3637612389 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe801da80 con 0x5624c55e0b90 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.897+0000 7febdf400640 1 -- 172.21.15.86:0/3637612389 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe80288c0 con 0x5624c55e0b90 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.897+0000 7febf8b94740 1 -- 172.21.15.86:0/3637612389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 msgr2=0x5624c55d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.897+0000 7febf8b94740 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55d0ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7febe800bc10 tx=0x7febe8016000 comp rx=0 tx=0).stop 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 -- 172.21.15.86:0/3637612389 shutdown_connections 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55e0b90 0x5624c55d0ea0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d62d0 0x5624c55dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 --2- 172.21.15.86:0/3637612389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55d59b0 0x5624c55ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 -- 172.21.15.86:0/3637612389 >> 172.21.15.86:0/3637612389 conn(0x5624c55d1ec0 msgr2=0x5624c55d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 -- 172.21.15.86:0/3637612389 shutdown_connections 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 -- 172.21.15.86:0/3637612389 wait complete. 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 -- start start 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.898+0000 7febf8b94740 1 Processor -- start 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d59b0 0x5624c55e6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d62d0 0x5624c55e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 0x5624c55e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febf01b11d0 con 0x5624c55e0b90 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febf01b0b00 con 0x5624c55d59b0 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febf01add60 con 0x5624c55d62d0 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febf8b94740 10 client.0 _ll_register_callbacks cb 0x7febe00f74f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 0x5624c55e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febeea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d59b0 0x5624c55e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 0x5624c55e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45250/0 (socket says 172.21.15.86:45250) 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 -- 172.21.15.86:0/260499187 learned_addr learned my addr 172.21.15.86:0/260499187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febee000640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d62d0 0x5624c55e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 -- 172.21.15.86:0/260499187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d62d0 msgr2=0x5624c55e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d62d0 0x5624c55e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 -- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d59b0 msgr2=0x5624c55e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d59b0 0x5624c55e6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febef400640 1 -- 172.21.15.86:0/260499187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c5656100 con 0x5624c55e0b90 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febee000640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d62d0 0x5624c55e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.899+0000 7febeea00640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d59b0 0x5624c55e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febef400640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 0x5624c55e6fb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7febe8018040 tx=0x7febe8028560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dec85b4aee27f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febdcc00640 1 -- 172.21.15.86:0/260499187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8036060 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febdcc00640 1 -- 172.21.15.86:0/260499187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8681+0+0 (secure 0 0 0) 0x7febe801d7e0 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe801da80 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febdcc00640 1 -- 172.21.15.86:0/260499187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7febe802d030 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febf8b94740 10 client.16746 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.900+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624c56573f0 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febdcc00640 1 -- 172.21.15.86:0/260499187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe801da80 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febdcc00640 1 -- 172.21.15.86:0/260499187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7febe8027070 con 0x5624c55e0b90 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febf8b94740 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febf8b94740 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febf8b94740 20 client.16746 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febf8b94740 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c5657a90 0x5624c5659eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febf8b94740 4 client.16746 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:16:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624c5549320 con 0x5624c5657a90 2024-04-25T19:16:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.901+0000 7febeea00640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c5657a90 0x5624c5659eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.902+0000 7febeea00640 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c5657a90 0x5624c5659eb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb89fdcdb2cae0c server_cookie=67dffe76a5f7e48c in_seq=0 out_seq=0 2024-04-25T19:16:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.902+0000 7febdcc00640 10 client.16746 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febdcc00640 1 -- 172.21.15.86:0/260499187 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624c5549320 con 0x5624c5657a90 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febdcc00640 10 client.16746 handle_command_reply: tid=0 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 client.16746 shutdown 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 msgr2=0x5624c55e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 0x5624c55e6fb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7febe8018040 tx=0x7febe8028560 comp rx=0 tx=0).stop 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 shutdown_connections 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5624c5657a90 0x5624c5659eb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624c55e0b90 0x5624c55e6fb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5624c55d62d0 0x5624c55e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febdcc00640 0 client.16746 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 --2- 172.21.15.86:0/260499187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624c55d59b0 0x5624c55e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 >> 172.21.15.86:0/260499187 conn(0x5624c55d1ec0 msgr2=0x5624c55de240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 shutdown_connections 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.913+0000 7febf8b94740 1 -- 172.21.15.86:0/260499187 wait complete. 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.914+0000 7febf8b94740 20 client.16746 trim_cache size 0 max 16384 2024-04-25T19:16:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:16:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:16:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:16:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:16:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.914+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7febd40a3d90 msgr2=0x7febd40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.914+0000 7febefe00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7febd40a3d90 0x7febd40a6250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7febe8005e00 tx=0x7febe800b420 comp rx=0 tx=0).stop 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.914+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 msgr2=0x7febf01a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.914+0000 7febefe00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf01a0420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7febe0005e30 tx=0x7febe0003f90 comp rx=0 tx=0).stop 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 shutdown_connections 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7febd40a3d90 0x7febd40a6250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febf010c350 0x7febf019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf0103850 0x7febf01a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 --2- 172.21.15.86:0/3775875707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febf0102f30 0x7febf01a0420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 >> 172.21.15.86:0/3775875707 conn(0x7febf00fc770 msgr2=0x7febf0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 shutdown_connections 2024-04-25T19:16:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:11.915+0000 7febefe00640 1 -- 172.21.15.86:0/3775875707 wait complete. 2024-04-25T19:16:12.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:12 smithi194 ceph-mon[32740]: pgmap v415: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-25T19:16:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:12 smithi086 ceph-mon[27783]: pgmap v415: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-25T19:16:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:12 smithi183 ceph-mon[32344]: pgmap v415: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-25T19:16:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:14 smithi086 ceph-mon[27783]: pgmap v416: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:16:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:14 smithi086 ceph-mon[27783]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T19:16:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:14 smithi183 ceph-mon[32344]: pgmap v416: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:16:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:14 smithi183 ceph-mon[32344]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T19:16:14.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:14 smithi194 ceph-mon[32740]: pgmap v416: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:16:14.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:14 smithi194 ceph-mon[32740]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T19:16:15.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:16:16.221 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:16:16.846 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:16 smithi086 ceph-mon[27783]: pgmap v418: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-25T19:16:16.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:16 smithi183 ceph-mon[32344]: pgmap v418: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-25T19:16:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:16 smithi194 ceph-mon[32740]: pgmap v418: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-04-25T19:16:18.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 -- 172.21.15.86:0/1952045019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c81092d0 msgr2=0x7f43c81096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 --2- 172.21.15.86:0/1952045019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c81092d0 0x7f43c81096b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009990 tx=0x7f43bc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 -- 172.21.15.86:0/1952045019 shutdown_connections 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 --2- 172.21.15.86:0/1952045019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 0x7f43c8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 --2- 172.21.15.86:0/1952045019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c81092d0 0x7f43c81096b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 --2- 172.21.15.86:0/1952045019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c8109c80 0x7f43c810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 -- 172.21.15.86:0/1952045019 >> 172.21.15.86:0/1952045019 conn(0x7f43c8076210 msgr2=0x7f43c8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 -- 172.21.15.86:0/1952045019 shutdown_connections 2024-04-25T19:16:18.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.476+0000 7f43cfa00700 1 -- 172.21.15.86:0/1952045019 wait complete. 2024-04-25T19:16:18.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 Processor -- start 2024-04-25T19:16:18.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 -- start start 2024-04-25T19:16:18.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c81092d0 0x7f43c81a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c8109c80 0x7f43c81a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 0x7f43c819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81192b0 con 0x7f43c810e3b0 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8119130 con 0x7f43c8109c80 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.478+0000 7f43cfa00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8119430 con 0x7f43c81092d0 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cd000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c81092d0 0x7f43c81a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43c7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c8109c80 0x7f43c81a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 0x7f43c819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cd000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c81092d0 0x7f43c81a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34178/0 (socket says 172.21.15.86:34178) 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43c7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c8109c80 0x7f43c81a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55640/0 (socket says 172.21.15.86:55640) 2024-04-25T19:16:18.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cd000700 1 -- 172.21.15.86:0/2644556329 learned_addr learned my addr 172.21.15.86:0/2644556329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cda00700 1 -- 172.21.15.86:0/2644556329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c81092d0 msgr2=0x7f43c81a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cda00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c81092d0 0x7f43c81a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cda00700 1 -- 172.21.15.86:0/2644556329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c8109c80 msgr2=0x7f43c81a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:18.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cda00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c8109c80 0x7f43c81a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.479+0000 7f43cda00700 1 -- 172.21.15.86:0/2644556329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c819fd70 con 0x7f43c810e3b0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.480+0000 7f43cda00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 0x7f43c819f610 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f43c000efb0 tx=0x7f43c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.480+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0003a00 con 0x7f43c810e3b0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.480+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c000ce50 con 0x7f43c810e3b0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.480+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c81a0060 con 0x7f43c810e3b0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.480+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00047c0 con 0x7f43c810e3b0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.480+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81ac2d0 con 0x7f43c810e3b0 2024-04-25T19:16:18.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.481+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c8069730 con 0x7f43c810e3b0 2024-04-25T19:16:18.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.483+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43c0012050 con 0x7f43c810e3b0 2024-04-25T19:16:18.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.484+0000 7f43c5600700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43b00a3db0 0x7f43b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:18.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.485+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7f43c00d0220 con 0x7f43c810e3b0 2024-04-25T19:16:18.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.485+0000 7f43cd000700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43b00a3db0 0x7f43b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:18.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.485+0000 7f43cd000700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43b00a3db0 0x7f43b00a6270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b790 tx=0x7f43bc00b260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.489+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43c0022020 con 0x7f43c810e3b0 2024-04-25T19:16:18.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.751+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43c802df80 con 0x7f43c810e3b0 2024-04-25T19:16:18.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.753+0000 7f43c5600700 1 -- 172.21.15.86:0/2644556329 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f43c0010040 con 0x7f43c810e3b0 2024-04-25T19:16:18.756 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:16:18.756 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:16:18.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.755+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43b00a3db0 msgr2=0x7f43b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.755+0000 7f43cfa00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43b00a3db0 0x7f43b00a6270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b790 tx=0x7f43bc00b260 comp rx=0 tx=0).stop 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.755+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 msgr2=0x7f43c819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.755+0000 7f43cfa00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 0x7f43c819f610 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f43c000efb0 tx=0x7f43c000c3a0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 shutdown_connections 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c810e3b0 0x7f43c819f610 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f43b00a3db0 0x7f43b00a6270 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c81092d0 0x7f43c81a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 --2- 172.21.15.86:0/2644556329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43c8109c80 0x7f43c81a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 >> 172.21.15.86:0/2644556329 conn(0x7f43c8076210 msgr2=0x7f43c8077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 shutdown_connections 2024-04-25T19:16:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:18.756+0000 7f43cfa00700 1 -- 172.21.15.86:0/2644556329 wait complete. 2024-04-25T19:16:18.760 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:16:18.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:18 smithi183 ceph-mon[32344]: pgmap v419: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:16:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:18 smithi194 ceph-mon[32740]: pgmap v419: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:16:19.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:18 smithi086 ceph-mon[27783]: pgmap v419: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T19:16:19.630 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id c7891a21 --release --await' 2024-04-25T19:16:19.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id c7891a21 --release --await 2024-04-25T19:16:19.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:19 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2644556329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:16:19.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:19 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2644556329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:16:20.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:19 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2644556329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:16:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:16:20.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:20 smithi086 ceph-mon[27783]: pgmap v420: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-25T19:16:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:20 smithi183 ceph-mon[32344]: pgmap v420: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-25T19:16:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:20 smithi194 ceph-mon[32740]: pgmap v420: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.376+0000 7f6db0200700 1 -- 172.21.15.86:0/2497320325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8076f70 msgr2=0x7f6da8077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.376+0000 7f6db0200700 1 --2- 172.21.15.86:0/2497320325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8076f70 0x7f6da8077350 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009990 tx=0x7f6d9c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 -- 172.21.15.86:0/2497320325 shutdown_connections 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 --2- 172.21.15.86:0/2497320325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6da8113940 0x7f6da8115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 --2- 172.21.15.86:0/2497320325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8076f70 0x7f6da8077350 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 --2- 172.21.15.86:0/2497320325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6da8077890 0x7f6da8113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 -- 172.21.15.86:0/2497320325 >> 172.21.15.86:0/2497320325 conn(0x7f6da81007b0 msgr2=0x7f6da8102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 -- 172.21.15.86:0/2497320325 shutdown_connections 2024-04-25T19:16:22.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.377+0000 7f6db0200700 1 -- 172.21.15.86:0/2497320325 wait complete. 2024-04-25T19:16:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.378+0000 7f6db0200700 1 Processor -- start 2024-04-25T19:16:22.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.378+0000 7f6db0200700 1 -- start start 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6db0200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6da8076f70 0x7f6da81a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6db0200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 0x7f6da81a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6db0200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6da8113940 0x7f6da819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6db0200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6da8119320 con 0x7f6da8077890 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6db0200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81191a0 con 0x7f6da8076f70 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6db0200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81194a0 con 0x7f6da8113940 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 0x7f6da81a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 0x7f6da81a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41514/0 (socket says 172.21.15.86:41514) 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 -- 172.21.15.86:0/3766986841 learned_addr learned my addr 172.21.15.86:0/3766986841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:22.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dad800700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6da8076f70 0x7f6da81a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 -- 172.21.15.86:0/3766986841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6da8113940 msgr2=0x7f6da819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6da8113940 0x7f6da819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 -- 172.21.15.86:0/3766986841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6da8076f70 msgr2=0x7f6da81a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6da8076f70 0x7f6da81a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.379+0000 7f6dace00700 1 -- 172.21.15.86:0/3766986841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da819fd80 con 0x7f6da8077890 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.380+0000 7f6dace00700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 0x7f6da81a59a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400eb30 tx=0x7f6d9400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.380+0000 7f6da6000700 1 -- 172.21.15.86:0/3766986841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400cc10 con 0x7f6da8077890 2024-04-25T19:16:22.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.380+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81a0070 con 0x7f6da8077890 2024-04-25T19:16:22.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.380+0000 7f6da6000700 1 -- 172.21.15.86:0/3766986841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d940044d0 con 0x7f6da8077890 2024-04-25T19:16:22.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.380+0000 7f6da6000700 1 -- 172.21.15.86:0/3766986841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94003e60 con 0x7f6da8077890 2024-04-25T19:16:22.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.380+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da81ac330 con 0x7f6da8077890 2024-04-25T19:16:22.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.384+0000 7f6da6000700 1 -- 172.21.15.86:0/3766986841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d94004000 con 0x7f6da8077890 2024-04-25T19:16:22.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6dbac0ab80 1 Processor -- start 2024-04-25T19:16:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6dbac0ab80 1 -- start start 2024-04-25T19:16:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6da6000700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d900a3e30 0x7f6d900a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6da6000700 1 -- 172.21.15.86:0/3766986841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7f6d940d0aa0 con 0x7f6da8077890 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337cdcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337ce7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6dae200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d900a3e30 0x7f6d900a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.385+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564337c2ff90 con 0x564337cdc920 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564337bd6830 con 0x564337cdd240 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564337c2bf50 con 0x564337ce80f0 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337cdcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337cdcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41528/0 (socket says 172.21.15.86:41528) 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 -- 172.21.15.86:0/687964905 learned_addr learned my addr 172.21.15.86:0/687964905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dace00700 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337ce7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dae200700 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 -- 172.21.15.86:0/687964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 msgr2=0x564337cea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 -- 172.21.15.86:0/687964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 msgr2=0x564337ce7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337ce7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.386+0000 7f6dad800700 1 -- 172.21.15.86:0/687964905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564337bcc780 con 0x564337cdc920 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.387+0000 7f6dae200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d900a3e30 0x7f6d900a62f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6da806c300 tx=0x7f6da000b040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.387+0000 7f6dad800700 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337cdcd00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0096a0 tx=0x7f6d9c0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b00a7ca4432399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.387+0000 7f6d9aa00700 1 -- 172.21.15.86:0/687964905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c027070 con 0x564337cdc920 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.387+0000 7f6d9aa00700 1 -- 172.21.15.86:0/687964905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c0056d0 con 0x564337cdc920 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.387+0000 7f6d9aa00700 1 -- 172.21.15.86:0/687964905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c003b10 con 0x564337cdc920 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/687964905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 msgr2=0x564337cdcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337cdcd00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0096a0 tx=0x7f6d9c0096d0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/687964905 shutdown_connections 2024-04-25T19:16:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337cdcd00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/687964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337ce7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/687964905 >> 172.21.15.86:0/687964905 conn(0x564337c0b6a0 msgr2=0x564337cd8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/687964905 shutdown_connections 2024-04-25T19:16:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.388+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/687964905 wait complete. 2024-04-25T19:16:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 -- start start 2024-04-25T19:16:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 Processor -- start 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337ce7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337ce4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337ce5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564337ceabb0 con 0x564337cdd240 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564337be9960 con 0x564337cdc920 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.389+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564337cead30 con 0x564337ce80f0 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dbac0ab80 10 client.0 _ll_register_callbacks cb 0x564337bc8820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337ce4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dad800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337ce7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dad800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337ce7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:42200/0 (socket says 172.21.15.86:42200) 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dad800700 1 -- 172.21.15.86:0/2507763743 learned_addr learned my addr 172.21.15.86:0/2507763743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dae200700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337ce5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dace00700 1 -- 172.21.15.86:0/2507763743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 msgr2=0x564337ce5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dace00700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337ce5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dace00700 1 -- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 msgr2=0x564337ce7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dace00700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337ce7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dace00700 1 -- 172.21.15.86:0/2507763743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564337b7d9d0 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.390+0000 7f6dad800700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337ce7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6dace00700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337ce4b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6d940ce120 tx=0x7f6d9409c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1976a5a613102fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6d8f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6d8f400700 1 -- 172.21.15.86:0/2507763743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9409cdd0 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6d8f400700 1 -- 172.21.15.86:0/2507763743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7f6d940a5150 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9c0056d0 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6d8f400700 1 -- 172.21.15.86:0/2507763743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d9409d070 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6dbac0ab80 10 client.16764 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.391+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564337bc6bf0 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6d8f400700 1 -- 172.21.15.86:0/2507763743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c0056d0 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6d8f400700 1 -- 172.21.15.86:0/2507763743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6d940870c0 con 0x564337cdd240 2024-04-25T19:16:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dbac0ab80 10 client.16764 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dbac0ab80 10 client.16764 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:16:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dbac0ab80 20 client.16764 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337cf7830 0x564337cf9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dbac0ab80 4 client.16764 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:16:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564337c3d780 con 0x564337cf7830 2024-04-25T19:16:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.392+0000 7f6dad800700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337cf7830 0x564337cf9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.393+0000 7f6dad800700 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337cf7830 0x564337cf9c20 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c277c81582062eae server_cookie=a2d40dea146d77da in_seq=0 out_seq=0 2024-04-25T19:16:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.393+0000 7f6d8f400700 10 client.16764 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6d8f400700 1 -- 172.21.15.86:0/2507763743 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564337c3d780 con 0x564337cf7830 2024-04-25T19:16:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6d8f400700 10 client.16764 handle_command_reply: tid=0 2024-04-25T19:16:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 client.16764 shutdown 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 msgr2=0x564337ce4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337ce4b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6d940ce120 tx=0x7f6d9409c3c0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 shutdown_connections 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337cf7830 0x564337cf9c20 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337ce5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337ce7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2507763743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337ce4b20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 >> 172.21.15.86:0/2507763743 conn(0x564337c0b6a0 msgr2=0x564337cd89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6d8f400700 0 client.16764 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.394+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 shutdown_connections 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.395+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2507763743 wait complete. 2024-04-25T19:16:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.395+0000 7f6dbac0ab80 20 client.16764 trim_cache size 0 max 16384 2024-04-25T19:16:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.415+0000 7f6dbac0ab80 1 Processor -- start 2024-04-25T19:16:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 -- start start 2024-04-25T19:16:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337cd8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337cf3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b3bd0 con 0x564337cdd240 2024-04-25T19:16:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b5710 con 0x564337cdc920 2024-04-25T19:16:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b42a0 con 0x564337ce80f0 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dad800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337cd8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dae200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337cf3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dae200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cff450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36190/0 (socket says 172.21.15.86:36190) 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dad800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337cd8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:42212/0 (socket says 172.21.15.86:42212) 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.416+0000 7f6dae200700 1 -- 172.21.15.86:0/2952739868 learned_addr learned my addr 172.21.15.86:0/2952739868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dace00700 1 -- 172.21.15.86:0/2952739868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 msgr2=0x564337cff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dace00700 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dace00700 1 -- 172.21.15.86:0/2952739868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 msgr2=0x564337cd8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dace00700 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337cd8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dace00700 1 -- 172.21.15.86:0/2952739868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564337ce30f0 con 0x564337cdd240 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dad800700 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337cd8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6dace00700 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337cf3ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6d94098dd0 tx=0x7f6d94003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc4fd3968891317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6d9a000700 1 -- 172.21.15.86:0/2952739868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94009500 con 0x564337cdd240 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6d9a000700 1 -- 172.21.15.86:0/2952739868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9409c970 con 0x564337cdd240 2024-04-25T19:16:22.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.417+0000 7f6d9a000700 1 -- 172.21.15.86:0/2952739868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940a5150 con 0x564337cdd240 2024-04-25T19:16:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2952739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 msgr2=0x564337cf3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337cf3ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6d94098dd0 tx=0x7f6d94003900 comp rx=0 tx=0).stop 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2952739868 shutdown_connections 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdc920 0x564337cd8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2952739868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdd240 0x564337cf3ab0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2952739868 >> 172.21.15.86:0/2952739868 conn(0x564337c0b6a0 msgr2=0x564337cd8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2952739868 shutdown_connections 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.418+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2952739868 wait complete. 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.419+0000 7f6dbac0ab80 1 -- start start 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.419+0000 7f6dbac0ab80 1 Processor -- start 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.419+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337ceffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337cf0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dbac0ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b5040 con 0x564337cdc920 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b4970 con 0x564337cdd240 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dbac0ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b1b90 con 0x564337ce80f0 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dbac0ab80 10 client.0 _ll_register_callbacks cb 0x564337bc8820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337ceffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337ceffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41556/0 (socket says 172.21.15.86:41556) 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 -- 172.21.15.86:0/2161659922 learned_addr learned my addr 172.21.15.86:0/2161659922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dae200700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dace00700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337cf0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 -- 172.21.15.86:0/2161659922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 msgr2=0x564337cecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 -- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 msgr2=0x564337cf0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337cf0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.420+0000 7f6dad800700 1 -- 172.21.15.86:0/2161659922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564337ced7d0 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6dad800700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337ceffc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0096a0 tx=0x7f6d9c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6549babf01e1685a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6d8f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6d8f400700 1 -- 172.21.15.86:0/2161659922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c027040 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6d8f400700 1 -- 172.21.15.86:0/2161659922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7f6d9c003ca0 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9409c970 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6d8f400700 1 -- 172.21.15.86:0/2161659922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d9c034270 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6dbac0ab80 10 client.16770 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.421+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564337d5e850 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6d8f400700 1 -- 172.21.15.86:0/2161659922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9409c970 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6d8f400700 1 -- 172.21.15.86:0/2161659922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6d9c02d040 con 0x564337cdc920 2024-04-25T19:16:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dbac0ab80 10 client.16770 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dbac0ab80 10 client.16770 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dbac0ab80 20 client.16770 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337d5ea80 0x564337d60e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dbac0ab80 4 client.16770 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c7891a21", "release": true, "await": true}] 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c7891a21", "release": true, "await": true}) -- 0x564337c37820 con 0x564337d5ea80 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.422+0000 7f6dace00700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337d5ea80 0x564337d60e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.423+0000 7f6dace00700 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337d5ea80 0x564337d60e70 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d0246486cb5025 server_cookie=ef46683731b20c78 in_seq=0 out_seq=0 2024-04-25T19:16:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.423+0000 7f6d8f400700 10 client.16770 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6d8f400700 1 -- 172.21.15.86:0/2161659922 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564337c37820 con 0x564337d5ea80 2024-04-25T19:16:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6d8f400700 10 client.16770 handle_command_reply: tid=0 2024-04-25T19:16:22.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 client.16770 shutdown 2024-04-25T19:16:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 msgr2=0x564337ceffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337ceffc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0096a0 tx=0x7f6d9c005c70 comp rx=0 tx=0).stop 2024-04-25T19:16:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 shutdown_connections 2024-04-25T19:16:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564337d5ea80 0x564337d60e70 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564337ce80f0 0x564337cecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564337cdc920 0x564337ceffc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 --2- 172.21.15.86:0/2161659922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564337cdd240 0x564337cf0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 >> 172.21.15.86:0/2161659922 conn(0x564337c0b6a0 msgr2=0x564337cde820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.426+0000 7f6d8f400700 0 client.16770 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.427+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 shutdown_connections 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.427+0000 7f6dbac0ab80 1 -- 172.21.15.86:0/2161659922 wait complete. 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.427+0000 7f6dbac0ab80 20 client.16770 trim_cache size 0 max 16384 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:16:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 40, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "c7891a21": { 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 40, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 92.5, 2024-04-25T19:16:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:16:22.431 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d900a3e30 msgr2=0x7f6d900a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d900a3e30 0x7f6d900a62f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6da806c300 tx=0x7f6da000b040 comp rx=0 tx=0).stop 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 msgr2=0x7f6da81a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 0x7f6da81a59a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400eb30 tx=0x7f6d9400ee40 comp rx=0 tx=0).stop 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 shutdown_connections 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d900a3e30 0x7f6d900a62f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6da8113940 0x7f6da819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6da8076f70 0x7f6da81a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 --2- 172.21.15.86:0/3766986841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da8077890 0x7f6da81a59a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:22.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 >> 172.21.15.86:0/3766986841 conn(0x7f6da81007b0 msgr2=0x7f6da81145b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:22.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 shutdown_connections 2024-04-25T19:16:22.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:16:22.428+0000 7f6db0200700 1 -- 172.21.15.86:0/3766986841 wait complete. 2024-04-25T19:16:22.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:22 smithi183 ceph-mon[32344]: pgmap v421: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-25T19:16:22.951 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:22 smithi086 ceph-mon[27783]: pgmap v421: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-25T19:16:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:22 smithi194 ceph-mon[32740]: pgmap v421: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-25T19:16:23.468 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c7891a21', seconds elapsed: 47.9 2024-04-25T19:16:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:24 smithi183 ceph-mon[32344]: pgmap v422: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-04-25T19:16:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:24 smithi194 ceph-mon[32740]: pgmap v422: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-04-25T19:16:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.059+0000 7fa5a8400640 1 -- 172.21.15.86:0/3000594960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0104d60 msgr2=0x7fa5a0108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 --2- 172.21.15.86:0/3000594960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0104d60 0x7fa5a0108da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009860 tx=0x7fa58c02a340 comp rx=0 tx=0).stop 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 -- 172.21.15.86:0/3000594960 shutdown_connections 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 --2- 172.21.15.86:0/3000594960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a0109480 0x7fa5a0111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 --2- 172.21.15.86:0/3000594960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0104d60 0x7fa5a0108da0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 --2- 172.21.15.86:0/3000594960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a01043b0 0x7fa5a0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 -- 172.21.15.86:0/3000594960 >> 172.21.15.86:0/3000594960 conn(0x7fa5a00ffc50 msgr2=0x7fa5a0102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 -- 172.21.15.86:0/3000594960 shutdown_connections 2024-04-25T19:16:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.060+0000 7fa5a8400640 1 -- 172.21.15.86:0/3000594960 wait complete. 2024-04-25T19:16:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.061+0000 7fa5a8400640 1 Processor -- start 2024-04-25T19:16:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.061+0000 7fa5a8400640 1 -- start start 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a8400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 0x7fa5a01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a8400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0109480 0x7fa5a01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a8400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a019a6e0 0x7fa5a019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a8400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01143c0 con 0x7fa5a019a6e0 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a8400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0114240 con 0x7fa5a0109480 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a8400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0114540 con 0x7fa5a01043b0 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 0x7fa5a01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 0x7fa5a01a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36220/0 (socket says 172.21.15.86:36220) 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 -- 172.21.15.86:0/2134514796 learned_addr learned my addr 172.21.15.86:0/2134514796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a6400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a019a6e0 0x7fa5a019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5000640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0109480 0x7fa5a01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 -- 172.21.15.86:0/2134514796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0109480 msgr2=0x7fa5a01a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0109480 0x7fa5a01a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 -- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a019a6e0 msgr2=0x7fa5a019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a019a6e0 0x7fa5a019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.062+0000 7fa5a5a00640 1 -- 172.21.15.86:0/2134514796 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a019b2e0 con 0x7fa5a01043b0 2024-04-25T19:16:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.063+0000 7fa5a5a00640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 0x7fa5a01a0620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9c0 tx=0x7fa59400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.063+0000 7fa59e000640 1 -- 172.21.15.86:0/2134514796 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400cde0 con 0x7fa5a01043b0 2024-04-25T19:16:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.063+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a01a73c0 con 0x7fa5a01043b0 2024-04-25T19:16:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.063+0000 7fa59e000640 1 -- 172.21.15.86:0/2134514796 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5940035f0 con 0x7fa5a01043b0 2024-04-25T19:16:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.063+0000 7fa59e000640 1 -- 172.21.15.86:0/2134514796 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594010690 con 0x7fa5a01043b0 2024-04-25T19:16:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.063+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a01a77c0 con 0x7fa5a01043b0 2024-04-25T19:16:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.066+0000 7fa5ab1c4740 1 Processor -- start 2024-04-25T19:16:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.066+0000 7fa5ab1c4740 1 -- start start 2024-04-25T19:16:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa59e000640 1 -- 172.21.15.86:0/2134514796 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa594004000 con 0x7fa5a01043b0 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e34c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e41df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ad7d9a900 con 0x560ad7e34820 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ad7d40ef0 con 0x560ad7e3fa00 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ad7d968c0 con 0x560ad7e35140 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5a5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e34c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41602/0 (socket says 172.21.15.86:41602) 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5a6400640 1 -- 172.21.15.86:0/3216816931 learned_addr learned my addr 172.21.15.86:0/3216816931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.067+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e41df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa5a6400640 1 -- 172.21.15.86:0/3216816931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 msgr2=0x560ad7e3f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa5a6400640 1 -- 172.21.15.86:0/3216816931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 msgr2=0x560ad7e41df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e41df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa5a6400640 1 -- 172.21.15.86:0/3216816931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad7aad210 con 0x560ad7e34820 2024-04-25T19:16:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e34c00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa598009f50 tx=0x7fa5980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2897da2c708a834d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa592a00640 1 -- 172.21.15.86:0/3216816931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598004d20 con 0x560ad7e34820 2024-04-25T19:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa592a00640 1 -- 172.21.15.86:0/3216816931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa598004050 con 0x560ad7e34820 2024-04-25T19:16:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.068+0000 7fa592a00640 1 -- 172.21.15.86:0/3216816931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598005280 con 0x560ad7e34820 2024-04-25T19:16:25.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:24 smithi086 ceph-mon[27783]: pgmap v422: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa59e000640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5880a3dc0 0x7fa5880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa59e000640 1 -- 172.21.15.86:0/2134514796 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fa5940d0bd0 con 0x7fa5a01043b0 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5a5a00640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5880a3dc0 0x7fa5880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3216816931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 msgr2=0x560ad7e34c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e34c00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa598009f50 tx=0x7fa5980118d0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3216816931 shutdown_connections 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e41df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3216816931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e34c00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.069+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3216816931 >> 172.21.15.86:0/3216816931 conn(0x560ad7e30d30 msgr2=0x560ad7e31110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.070+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3216816931 shutdown_connections 2024-04-25T19:16:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.070+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3216816931 wait complete. 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.070+0000 7fa5a5a00640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5880a3dc0 0x7fa5880a6280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x560ad7e35dc0 tx=0x7fa5940d4680 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.070+0000 7fa5ab1c4740 1 -- start start 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.070+0000 7fa5ab1c4740 1 Processor -- start 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e34820 0x560ad7e3c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 0x560ad7e3ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ad7e42630 con 0x560ad7e3fa00 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ad7d54310 con 0x560ad7e35140 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ad7e427b0 con 0x560ad7e34820 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5ab1c4740 10 client.0 _ll_register_callbacks cb 0x560ad7d22cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5a5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 0x560ad7e3ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5a5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 0x560ad7e3ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:42264/0 (socket says 172.21.15.86:42264) 2024-04-25T19:16:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5a5000640 1 -- 172.21.15.86:0/3367837796 learned_addr learned my addr 172.21.15.86:0/3367837796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.071+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e34820 0x560ad7e3c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a5000640 1 -- 172.21.15.86:0/3367837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e34820 msgr2=0x560ad7e3c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e34820 0x560ad7e3c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a5000640 1 -- 172.21.15.86:0/3367837796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 msgr2=0x560ad7e3d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e3d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a5000640 1 -- 172.21.15.86:0/3367837796 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad7b79620 con 0x560ad7e35140 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e34820 0x560ad7e3c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 0x560ad7e3ce90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa58c02a930 tx=0x7fa58c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9bcef1a9f2c9b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa587400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa587400640 1 -- 172.21.15.86:0/3367837796 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c037040 con 0x560ad7e35140 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa587400640 1 -- 172.21.15.86:0/3367837796 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fa58c030450 con 0x560ad7e35140 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.072+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa598004050 con 0x560ad7e35140 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa587400640 1 -- 172.21.15.86:0/3367837796 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa58c04e050 con 0x560ad7e35140 2024-04-25T19:16:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 10 client.26882 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ad7e4e840 con 0x560ad7e35140 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa587400640 1 -- 172.21.15.86:0/3367837796 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa598004050 con 0x560ad7e35140 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa587400640 1 -- 172.21.15.86:0/3367837796 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa58c03d450 con 0x560ad7e35140 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 10 client.26882 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 10 client.26882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 20 client.26882 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7e4fd50 0x560ad7e52160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 4 client.26882 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:16:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.073+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad7dae8b0 con 0x560ad7e4fd50 2024-04-25T19:16:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.074+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7e4fd50 0x560ad7e52160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.074+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7e4fd50 0x560ad7e52160 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666196c3085d0ea1 server_cookie=9b530592e1dc405c in_seq=0 out_seq=0 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.074+0000 7fa587400640 10 client.26882 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.075+0000 7fa587400640 1 -- 172.21.15.86:0/3367837796 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ad7dae8b0 con 0x560ad7e4fd50 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.075+0000 7fa587400640 10 client.26882 handle_command_reply: tid=0 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.075+0000 7fa5ab1c4740 1 client.26882 shutdown 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.075+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 msgr2=0x560ad7e3ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.075+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 0x560ad7e3ce90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa58c02a930 tx=0x7fa58c009490 comp rx=0 tx=0).stop 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 shutdown_connections 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7e4fd50 0x560ad7e52160 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e35140 0x560ad7e3ce90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3367837796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e34820 0x560ad7e3c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 >> 172.21.15.86:0/3367837796 conn(0x560ad7e30d30 msgr2=0x560ad7e36a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa587400640 0 client.26882 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 shutdown_connections 2024-04-25T19:16:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3367837796 wait complete. 2024-04-25T19:16:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.076+0000 7fa5ab1c4740 20 client.26882 trim_cache size 0 max 16384 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 Processor -- start 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 -- start start 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e57a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01aff60 con 0x560ad7e34820 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01b1aa0 con 0x560ad7e3fa00 2024-04-25T19:16:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.092+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01b0630 con 0x560ad7e35140 2024-04-25T19:16:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36242/0 (socket says 172.21.15.86:36242) 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 -- 172.21.15.86:0/4085261255 learned_addr learned my addr 172.21.15.86:0/4085261255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a6400640 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e57a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 -- 172.21.15.86:0/4085261255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 msgr2=0x560ad7e57a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e57a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 -- 172.21.15.86:0/4085261255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 msgr2=0x560ad7e4b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e4b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 -- 172.21.15.86:0/4085261255 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad7e34f70 con 0x560ad7e35140 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a6400640 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e4b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.093+0000 7fa5a5000640 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3d1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005b60 tx=0x7fa58c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc3b193d471661f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.094+0000 7fa592000640 1 -- 172.21.15.86:0/4085261255 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c037050 con 0x560ad7e35140 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.094+0000 7fa592000640 1 -- 172.21.15.86:0/4085261255 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa58c030900 con 0x560ad7e35140 2024-04-25T19:16:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.094+0000 7fa592000640 1 -- 172.21.15.86:0/4085261255 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c005040 con 0x560ad7e35140 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/4085261255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 msgr2=0x560ad7e3d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3d1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005b60 tx=0x7fa58c02a340 comp rx=0 tx=0).stop 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/4085261255 shutdown_connections 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e3fa00 0x560ad7e57a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e3d1f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/4085261255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e34820 0x560ad7e4b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/4085261255 >> 172.21.15.86:0/4085261255 conn(0x560ad7e30d30 msgr2=0x560ad7e2f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/4085261255 shutdown_connections 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/4085261255 wait complete. 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 -- start start 2024-04-25T19:16:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.095+0000 7fa5ab1c4740 1 Processor -- start 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e34820 0x560ad7e47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01b13d0 con 0x560ad7e3fa00 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01b0d00 con 0x560ad7e34820 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01adf60 con 0x560ad7e35140 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5ab1c4740 10 client.0 _ll_register_callbacks cb 0x7fa58c005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e45720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5a5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5a5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:36246/0 (socket says 172.21.15.86:36246) 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5a5000640 1 -- 172.21.15.86:0/3025902364 learned_addr learned my addr 172.21.15.86:0/3025902364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.096+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e34820 0x560ad7e47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5a5000640 1 -- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e34820 msgr2=0x560ad7e47e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e34820 0x560ad7e47e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5a5000640 1 -- 172.21.15.86:0/3025902364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 msgr2=0x560ad7e45720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e45720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5a5000640 1 -- 172.21.15.86:0/3025902364 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad7e45f50 con 0x560ad7e35140 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5a5000640 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e451c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005b60 tx=0x7fa58c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb3503b61e56de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa587400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa587400640 1 -- 172.21.15.86:0/3025902364 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c037040 con 0x560ad7e35140 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c030900 con 0x560ad7e35140 2024-04-25T19:16:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.097+0000 7fa587400640 1 -- 172.21.15.86:0/3025902364 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fa58c030450 con 0x560ad7e35140 2024-04-25T19:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa587400640 1 -- 172.21.15.86:0/3025902364 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa58c060050 con 0x560ad7e35140 2024-04-25T19:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 10 client.26881 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ad7eb6390 con 0x560ad7e35140 2024-04-25T19:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa587400640 1 -- 172.21.15.86:0/3025902364 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa58c030900 con 0x560ad7e35140 2024-04-25T19:16:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa587400640 1 -- 172.21.15.86:0/3025902364 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa58c004750 con 0x560ad7e35140 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 10 client.26881 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 20 client.26881 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7eb6a70 0x560ad7eb8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 4 client.26881 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.098+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ad7da8320 con 0x560ad7eb6a70 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.099+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7eb6a70 0x560ad7eb8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.099+0000 7fa5a6400640 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7eb6a70 0x560ad7eb8e90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85950a1961420aca server_cookie=d513812e92c18dfe in_seq=0 out_seq=0 2024-04-25T19:16:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.099+0000 7fa587400640 10 client.26881 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa587400640 1 -- 172.21.15.86:0/3025902364 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ad7da8320 con 0x560ad7eb6a70 2024-04-25T19:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa587400640 10 client.26881 handle_command_reply: tid=0 2024-04-25T19:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 client.26881 shutdown 2024-04-25T19:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 msgr2=0x560ad7e451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e451c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005b60 tx=0x7fa58c005f90 comp rx=0 tx=0).stop 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 shutdown_connections 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560ad7eb6a70 0x560ad7eb8e90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ad7e3fa00 0x560ad7e45720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ad7e35140 0x560ad7e451c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 --2- 172.21.15.86:0/3025902364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ad7e34820 0x560ad7e47e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa587400640 0 client.26881 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.110+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 >> 172.21.15.86:0/3025902364 conn(0x560ad7e30d30 msgr2=0x560ad7e36a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 shutdown_connections 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5ab1c4740 1 -- 172.21.15.86:0/3025902364 wait complete. 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5ab1c4740 20 client.26881 trim_cache size 0 max 16384 2024-04-25T19:16:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5880a3dc0 msgr2=0x7fa5880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5a8400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5880a3dc0 0x7fa5880a6280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x560ad7e35dc0 tx=0x7fa5940d4680 comp rx=0 tx=0).stop 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 msgr2=0x7fa5a01a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.111+0000 7fa5a8400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 0x7fa5a01a0620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9c0 tx=0x7fa59400ee80 comp rx=0 tx=0).stop 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 shutdown_connections 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa5880a3dc0 0x7fa5880a6280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a019a6e0 0x7fa5a019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0109480 0x7fa5a01a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 --2- 172.21.15.86:0/2134514796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5a01043b0 0x7fa5a01a0620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 >> 172.21.15.86:0/2134514796 conn(0x7fa5a00ffc50 msgr2=0x7fa5a0101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 shutdown_connections 2024-04-25T19:16:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:25.112+0000 7fa5a8400640 1 -- 172.21.15.86:0/2134514796 wait complete. 2024-04-25T19:16:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:26 smithi183 ceph-mon[32344]: pgmap v423: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-25T19:16:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:26 smithi194 ceph-mon[32740]: pgmap v423: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-25T19:16:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:26 smithi086 ceph-mon[27783]: pgmap v423: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-25T19:16:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:28 smithi183 ceph-mon[32344]: pgmap v424: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-04-25T19:16:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:28 smithi194 ceph-mon[32740]: pgmap v424: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-04-25T19:16:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:28 smithi086 ceph-mon[27783]: pgmap v424: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-04-25T19:16:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:30 smithi183 ceph-mon[32344]: pgmap v425: 97 pgs: 97 active+clean; 2.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-25T19:16:30.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:30 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:16:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:30 smithi194 ceph-mon[32740]: pgmap v425: 97 pgs: 97 active+clean; 2.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-25T19:16:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:30 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:16:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:30 smithi086 ceph-mon[27783]: pgmap v425: 97 pgs: 97 active+clean; 2.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-25T19:16:31.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:30 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:375/600: open fd = 3 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:376/600: open fd = 3 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.538 INFO:tasks.workunit.client.0.smithi086.stdout:377/600: open fd = 3 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:378/600: open fd = 3 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:379/600: open fd = 3 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:380/600: open fd = 3 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:381/600: open fd = 3 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:382/600: open fd = 3 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:383/600: open fd = 3 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:384/600: open fd = 3 2024-04-25T19:16:31.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:385/600: open fd = 3 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:386/600: open fd = 3 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:387/600: open fd = 3 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.541 INFO:tasks.workunit.client.0.smithi086.stdout:388/600: open fd = 3 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:389/600: open fd = 3 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:390/600: open fd = 3 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:391/600: open fd = 3 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:392/600: open fd = 3 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:393/600: open fd = 3 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:394/600: open fd = 3 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:395/600: open fd = 3 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:396/600: open fd = 3 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:397/600: open fd = 3 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:398/600: open fd = 3 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:399/600: open fd = 3 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:400/600: open fd = 3 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:401/600: open fd = 3 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:402/600: open fd = 3 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:403/600: open fd = 3 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:404/600: open fd = 3 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:405/600: open fd = 3 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:406/600: open fd = 3 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:407/600: open fd = 3 2024-04-25T19:16:31.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:408/600: open fd = 3 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:409/600: open fd = 3 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:410/600: open fd = 3 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:411/600: open fd = 3 2024-04-25T19:16:31.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:412/600: open fd = 3 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:413/600: open fd = 3 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:414/600: open fd = 3 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:415/600: open fd = 3 2024-04-25T19:16:31.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:416/600: open fd = 3 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:417/600: open fd = 3 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:418/600: open fd = 3 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.549 INFO:tasks.workunit.client.0.smithi086.stdout:419/600: open fd = 3 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:420/600: open fd = 3 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:421/600: open fd = 3 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:422/600: open fd = 3 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:423/600: open fd = 3 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:424/600: open fd = 3 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:425/600: open fd = 3 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:426/600: open fd = 3 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:427/600: open fd = 3 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:428/600: open fd = 3 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:429/600: open fd = 3 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:430/600: open fd = 3 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:431/600: open fd = 3 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:432/600: open fd = 3 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:433/600: open fd = 3 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:434/600: open fd = 3 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:435/600: open fd = 3 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:436/600: open fd = 3 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:16:31.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:16:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:32 smithi183 ceph-mon[32344]: pgmap v426: 97 pgs: 97 active+clean; 2.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T19:16:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:32 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:16:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:32 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:16:32.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:32 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:16:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:32 smithi194 ceph-mon[32740]: pgmap v426: 97 pgs: 97 active+clean; 2.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T19:16:32.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:32 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:16:32.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:32 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:16:32.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:32 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:16:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:32 smithi086 ceph-mon[27783]: pgmap v426: 97 pgs: 97 active+clean; 2.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T19:16:33.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:32 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:16:33.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:32 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:16:33.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:32 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:16:34.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:34 smithi183 ceph-mon[32344]: pgmap v427: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T19:16:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:34 smithi194 ceph-mon[32740]: pgmap v427: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T19:16:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:34 smithi086 ceph-mon[27783]: pgmap v427: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T19:16:36.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:36 smithi183 ceph-mon[32344]: pgmap v428: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:16:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:36 smithi194 ceph-mon[32740]: pgmap v428: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:16:37.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:36 smithi086 ceph-mon[27783]: pgmap v428: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:16:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 -- 172.21.15.86:0/1280149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108490 msgr2=0x7fba3c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 --2- 172.21.15.86:0/1280149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108490 0x7fba3c1088d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fba24009930 tx=0x7fba2402a420 comp rx=0 tx=0).stop 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 -- 172.21.15.86:0/1280149070 shutdown_connections 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 --2- 172.21.15.86:0/1280149070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3c108e10 0x7fba3c115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 --2- 172.21.15.86:0/1280149070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108490 0x7fba3c1088d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 --2- 172.21.15.86:0/1280149070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c10e420 0x7fba3c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 -- 172.21.15.86:0/1280149070 >> 172.21.15.86:0/1280149070 conn(0x7fba3c077d30 msgr2=0x7fba3c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 -- 172.21.15.86:0/1280149070 shutdown_connections 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.251+0000 7fba41a00640 1 -- 172.21.15.86:0/1280149070 wait complete. 2024-04-25T19:16:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.252+0000 7fba41a00640 1 Processor -- start 2024-04-25T19:16:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.252+0000 7fba41a00640 1 -- start start 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 0x7fba3c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108e10 0x7fba3c1a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3c10e420 0x7fba3c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1185f0 con 0x7fba3c108e10 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c118470 con 0x7fba3c108490 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c118770 con 0x7fba3c10e420 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 0x7fba3c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 0x7fba3c1a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:34066/0 (socket says 172.21.15.86:34066) 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 -- 172.21.15.86:0/4260202758 learned_addr learned my addr 172.21.15.86:0/4260202758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba33e00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108e10 0x7fba3c1a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba41000640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3c10e420 0x7fba3c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 -- 172.21.15.86:0/4260202758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3c10e420 msgr2=0x7fba3c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3c10e420 0x7fba3c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 -- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108e10 msgr2=0x7fba3c1a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108e10 0x7fba3c1a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.253+0000 7fba3be00640 1 -- 172.21.15.86:0/4260202758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c19f0f0 con 0x7fba3c108490 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.254+0000 7fba3be00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 0x7fba3c1a47c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e9f0 tx=0x7fba2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.254+0000 7fba39600640 1 -- 172.21.15.86:0/4260202758 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00cd80 con 0x7fba3c108490 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.254+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c19f3e0 con 0x7fba3c108490 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.254+0000 7fba39600640 1 -- 172.21.15.86:0/4260202758 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2c0108e0 con 0x7fba3c108490 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.254+0000 7fba39600640 1 -- 172.21.15.86:0/4260202758 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c018980 con 0x7fba3c108490 2024-04-25T19:16:38.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.254+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c1ab680 con 0x7fba3c108490 2024-04-25T19:16:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.258+0000 7fba45e16740 1 Processor -- start 2024-04-25T19:16:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.258+0000 7fba45e16740 1 -- start start 2024-04-25T19:16:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba45e16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba45e16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735cb4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba45e16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba45e16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557735c0f900 con 0x557735cb4b90 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba45e16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557735bb5ef0 con 0x557735caa2d0 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba45e16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557735c0b8c0 con 0x557735ca99b0 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba39600640 1 -- 172.21.15.86:0/4260202758 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba2c010430 con 0x7fba3c108490 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51196/0 (socket says 172.21.15.86:51196) 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba33e00640 1 -- 172.21.15.86:0/490544404 learned_addr learned my addr 172.21.15.86:0/490544404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.259+0000 7fba41000640 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba3be00640 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735cb4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba33e00640 1 -- 172.21.15.86:0/490544404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 msgr2=0x557735ca9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba33e00640 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba33e00640 1 -- 172.21.15.86:0/490544404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 msgr2=0x557735cb4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba33e00640 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735cb4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba33e00640 1 -- 172.21.15.86:0/490544404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557735922210 con 0x557735cb4b90 2024-04-25T19:16:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba3be00640 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735cb4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:16:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba33e00640 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb6f80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fba2402a940 tx=0x7fba24005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933a554b6884ace3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba31600640 1 -- 172.21.15.86:0/490544404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24040070 con 0x557735cb4b90 2024-04-25T19:16:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba31600640 1 -- 172.21.15.86:0/490544404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba24004030 con 0x557735cb4b90 2024-04-25T19:16:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.260+0000 7fba31600640 1 -- 172.21.15.86:0/490544404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24035040 con 0x557735cb4b90 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba39600640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fba200a3d90 0x7fba200a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba39600640 1 -- 172.21.15.86:0/4260202758 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fba2c0d0a70 con 0x7fba3c108490 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 -- 172.21.15.86:0/490544404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 msgr2=0x557735cb6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb6f80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fba2402a940 tx=0x7fba24005be0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba41000640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fba200a3d90 0x7fba200a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 -- 172.21.15.86:0/490544404 shutdown_connections 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb6f80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735cb4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 --2- 172.21.15.86:0/490544404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 -- 172.21.15.86:0/490544404 >> 172.21.15.86:0/490544404 conn(0x557735ca5ec0 msgr2=0x557735ca62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 -- 172.21.15.86:0/490544404 shutdown_connections 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.261+0000 7fba45e16740 1 -- 172.21.15.86:0/490544404 wait complete. 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.262+0000 7fba45e16740 1 -- start start 2024-04-25T19:16:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.262+0000 7fba41000640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fba200a3d90 0x7fba200a6250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba34005fd0 tx=0x7fba34009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.262+0000 7fba45e16740 1 Processor -- start 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735cb1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 0x557735cb1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735cb4b90 0x557735cb2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557735cb7760 con 0x557735caa2d0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557735bc9310 con 0x557735cb4b90 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557735cb78e0 con 0x557735ca99b0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba45e16740 10 client.0 _ll_register_callbacks cb 0x557735b97cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 0x557735cb1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735cb1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 0x557735cb1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51198/0 (socket says 172.21.15.86:51198) 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 -- 172.21.15.86:0/1840213175 learned_addr learned my addr 172.21.15.86:0/1840213175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba41000640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735cb4b90 0x557735cb2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 -- 172.21.15.86:0/1840213175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 msgr2=0x557735cb1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735cb1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 -- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735cb4b90 msgr2=0x557735cb2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735cb4b90 0x557735cb2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba33e00640 1 -- 172.21.15.86:0/1840213175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577359ee620 con 0x557735caa2d0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.263+0000 7fba3be00640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735cb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba33e00640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 0x557735cb1fd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fba24000c00 tx=0x7fba2402ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764e90bf66c99774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba2a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba2a000640 1 -- 172.21.15.86:0/1840213175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24040070 con 0x557735caa2d0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba2a000640 1 -- 172.21.15.86:0/1840213175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fba24005260 con 0x557735caa2d0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba24004030 con 0x557735caa2d0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba2a000640 1 -- 172.21.15.86:0/1840213175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba24035050 con 0x557735caa2d0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba45e16740 10 client.16800 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557735cc3890 con 0x557735caa2d0 2024-04-25T19:16:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.264+0000 7fba2a000640 1 -- 172.21.15.86:0/1840213175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba24004030 con 0x557735caa2d0 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba2a000640 1 -- 172.21.15.86:0/1840213175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fba24004a10 con 0x557735caa2d0 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba45e16740 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba45e16740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba45e16740 20 client.16800 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba45e16740 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735cc4be0 0x557735cc6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba45e16740 4 client.16800 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557735c238b0 con 0x557735cc4be0 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.265+0000 7fba3be00640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735cc4be0 0x557735cc6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.266+0000 7fba3be00640 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735cc4be0 0x557735cc6fd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ae0a0b5772d1b40 server_cookie=4528012af62aa4f7 in_seq=0 out_seq=0 2024-04-25T19:16:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.266+0000 7fba2a000640 10 client.16800 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba2a000640 1 -- 172.21.15.86:0/1840213175 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557735c238b0 con 0x557735cc4be0 2024-04-25T19:16:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba2a000640 10 client.16800 handle_command_reply: tid=0 2024-04-25T19:16:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 client.16800 shutdown 2024-04-25T19:16:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 msgr2=0x557735cb1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 0x557735cb1fd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fba24000c00 tx=0x7fba2402ab70 comp rx=0 tx=0).stop 2024-04-25T19:16:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 shutdown_connections 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735cc4be0 0x557735cc6fd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735cb4b90 0x557735cb2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735caa2d0 0x557735cb1fd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 --2- 172.21.15.86:0/1840213175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735cb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 >> 172.21.15.86:0/1840213175 conn(0x557735ca5ec0 msgr2=0x557735cabbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba2a000640 0 client.16800 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 shutdown_connections 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 1 -- 172.21.15.86:0/1840213175 wait complete. 2024-04-25T19:16:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.267+0000 7fba45e16740 20 client.16800 trim_cache size 0 max 16384 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.283+0000 7fba45e16740 1 Processor -- start 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 -- start start 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735caba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1b4180 con 0x557735cb4b90 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1b5cc0 con 0x557735caa2d0 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba45e16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1b4850 con 0x557735ca99b0 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba41000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba41000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51200/0 (socket says 172.21.15.86:51200) 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba33e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735caba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba41000640 1 -- 172.21.15.86:0/3204495343 learned_addr learned my addr 172.21.15.86:0/3204495343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.284+0000 7fba3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51062/0 (socket says 172.21.15.86:51062) 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba33e00640 1 -- 172.21.15.86:0/3204495343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 msgr2=0x557735ca57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba33e00640 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba33e00640 1 -- 172.21.15.86:0/3204495343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 msgr2=0x557735cb1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba33e00640 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba33e00640 1 -- 172.21.15.86:0/3204495343 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557735caa100 con 0x557735caa2d0 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba41000640 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba33e00640 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735caba70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba2402a460 tx=0x7fba24005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee4ed8913ba6797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba30c00640 1 -- 172.21.15.86:0/3204495343 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24040040 con 0x557735caa2d0 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba30c00640 1 -- 172.21.15.86:0/3204495343 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba240036b0 con 0x557735caa2d0 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.285+0000 7fba30c00640 1 -- 172.21.15.86:0/3204495343 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24005390 con 0x557735caa2d0 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 -- 172.21.15.86:0/3204495343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 msgr2=0x557735caba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735caba70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba2402a460 tx=0x7fba24005fd0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 -- 172.21.15.86:0/3204495343 shutdown_connections 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cb1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735caa2d0 0x557735caba70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 --2- 172.21.15.86:0/3204495343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735ca99b0 0x557735ca57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 -- 172.21.15.86:0/3204495343 >> 172.21.15.86:0/3204495343 conn(0x557735ca5ec0 msgr2=0x557735cab360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 -- 172.21.15.86:0/3204495343 shutdown_connections 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 -- 172.21.15.86:0/3204495343 wait complete. 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.286+0000 7fba45e16740 1 -- start start 2024-04-25T19:16:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 Processor -- start 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735ca99b0 0x557735d236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 0x557735d23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cc9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1b55f0 con 0x557735cb4b90 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1b4f20 con 0x557735ca99b0 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c1b2180 con 0x557735caa2d0 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba45e16740 10 client.0 _ll_register_callbacks cb 0x7fba24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba33e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 0x557735d23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.287+0000 7fba41000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cc9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 0x557735d23c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51068/0 (socket says 172.21.15.86:51068) 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 -- 172.21.15.86:0/2212581105 learned_addr learned my addr 172.21.15.86:0/2212581105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba3be00640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735ca99b0 0x557735d236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 -- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735ca99b0 msgr2=0x557735d236c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735ca99b0 0x557735d236c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 -- 172.21.15.86:0/2212581105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 msgr2=0x557735cc9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cc9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 -- 172.21.15.86:0/2212581105 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557735cc9e60 con 0x557735caa2d0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba41000640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cc9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba33e00640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 0x557735d23c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fba24004e60 tx=0x7fba240050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520d34f479ec8bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.288+0000 7fba2a000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.289+0000 7fba2a000640 1 -- 172.21.15.86:0/2212581105 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24040070 con 0x557735caa2d0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.289+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba240036b0 con 0x557735caa2d0 2024-04-25T19:16:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.289+0000 7fba2a000640 1 -- 172.21.15.86:0/2212581105 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fba24004980 con 0x557735caa2d0 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.289+0000 7fba2a000640 1 -- 172.21.15.86:0/2212581105 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba24004bc0 con 0x557735caa2d0 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.289+0000 7fba45e16740 10 client.26899 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.289+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557735d2ade0 con 0x557735caa2d0 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba2a000640 1 -- 172.21.15.86:0/2212581105 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba240036b0 con 0x557735caa2d0 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba2a000640 1 -- 172.21.15.86:0/2212581105 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fba24003db0 con 0x557735caa2d0 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba45e16740 10 client.26899 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba45e16740 10 client.26899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba45e16740 20 client.26899 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba45e16740 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735d2b360 0x557735d2d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba45e16740 4 client.26899 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.290+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557735c1d320 con 0x557735d2b360 2024-04-25T19:16:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.291+0000 7fba3be00640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735d2b360 0x557735d2d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.291+0000 7fba2a000640 10 client.26899 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.291+0000 7fba3be00640 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735d2b360 0x557735d2d780 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414279ac617b7941 server_cookie=899ce0eb8e73df46 in_seq=0 out_seq=0 2024-04-25T19:16:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.304+0000 7fba2a000640 1 -- 172.21.15.86:0/2212581105 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557735c1d320 con 0x557735d2b360 2024-04-25T19:16:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.304+0000 7fba2a000640 10 client.26899 handle_command_reply: tid=0 2024-04-25T19:16:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.304+0000 7fba45e16740 1 client.26899 shutdown 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.304+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 msgr2=0x557735d23c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.304+0000 7fba45e16740 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 0x557735d23c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fba24004e60 tx=0x7fba240050c0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 shutdown_connections 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557735d2b360 0x557735d2d780 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557735cb4b90 0x557735cc9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557735caa2d0 0x557735d23c00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 --2- 172.21.15.86:0/2212581105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557735ca99b0 0x557735d236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba2a000640 0 client.26899 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 >> 172.21.15.86:0/2212581105 conn(0x557735ca5ec0 msgr2=0x557735cabbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 shutdown_connections 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 1 -- 172.21.15.86:0/2212581105 wait complete. 2024-04-25T19:16:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.305+0000 7fba45e16740 20 client.26899 trim_cache size 0 max 16384 2024-04-25T19:16:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:16:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:16:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:16:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fba200a3d90 msgr2=0x7fba200a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fba200a3d90 0x7fba200a6250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba34005fd0 tx=0x7fba34009500 comp rx=0 tx=0).stop 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 msgr2=0x7fba3c1a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 0x7fba3c1a47c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e9f0 tx=0x7fba2c00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 shutdown_connections 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fba200a3d90 0x7fba200a6250 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba3c10e420 0x7fba3c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba3c108e10 0x7fba3c1a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 --2- 172.21.15.86:0/4260202758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3c108490 0x7fba3c1a47c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 >> 172.21.15.86:0/4260202758 conn(0x7fba3c077d30 msgr2=0x7fba3c078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 shutdown_connections 2024-04-25T19:16:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:38.306+0000 7fba41a00640 1 -- 172.21.15.86:0/4260202758 wait complete. 2024-04-25T19:16:38.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:38 smithi183 ceph-mon[32344]: pgmap v429: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:16:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:38 smithi194 ceph-mon[32740]: pgmap v429: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:16:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:38 smithi086 ceph-mon[27783]: pgmap v429: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:16:40.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:40 smithi183 ceph-mon[32344]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T19:16:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:40 smithi194 ceph-mon[32740]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T19:16:41.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:40 smithi086 ceph-mon[27783]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T19:16:42.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:42 smithi183 ceph-mon[32344]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T19:16:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:42 smithi194 ceph-mon[32740]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T19:16:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:42 smithi086 ceph-mon[27783]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T19:16:44.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:44 smithi183 ceph-mon[32344]: pgmap v432: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:16:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:44 smithi194 ceph-mon[32740]: pgmap v432: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:16:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:44 smithi086 ceph-mon[27783]: pgmap v432: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:16:46.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:46 smithi183 ceph-mon[32344]: pgmap v433: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:46 smithi194 ceph-mon[32740]: pgmap v433: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:47.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:46 smithi086 ceph-mon[27783]: pgmap v433: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:48 smithi086 ceph-mon[27783]: pgmap v434: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:48.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:48 smithi183 ceph-mon[32344]: pgmap v434: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:48.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:48 smithi194 ceph-mon[32740]: pgmap v434: 97 pgs: 97 active+clean; 2.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:50.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:50 smithi194 ceph-mon[32740]: pgmap v435: 97 pgs: 97 active+clean; 3.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:16:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:50 smithi086 ceph-mon[27783]: pgmap v435: 97 pgs: 97 active+clean; 3.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:16:51.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:50 smithi183 ceph-mon[32344]: pgmap v435: 97 pgs: 97 active+clean; 3.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:16:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 -- 172.21.15.86:0/2488037729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c102980 msgr2=0x7fab9c1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 --2- 172.21.15.86:0/2488037729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c102980 0x7fab9c1069c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab88009900 tx=0x7fab8802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 -- 172.21.15.86:0/2488037729 shutdown_connections 2024-04-25T19:16:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 --2- 172.21.15.86:0/2488037729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c1070a0 0x7fab9c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 --2- 172.21.15.86:0/2488037729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c102980 0x7fab9c1069c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 --2- 172.21.15.86:0/2488037729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab9c101fd0 0x7fab9c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 -- 172.21.15.86:0/2488037729 >> 172.21.15.86:0/2488037729 conn(0x7fab9c0fd890 msgr2=0x7fab9c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 -- 172.21.15.86:0/2488037729 shutdown_connections 2024-04-25T19:16:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.449+0000 7faba2e00640 1 -- 172.21.15.86:0/2488037729 wait complete. 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.450+0000 7faba2e00640 1 Processor -- start 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.450+0000 7faba2e00640 1 -- start start 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab9c101fd0 0x7fab9c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 0x7fab9c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c19ce70 0x7fab9c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c111fc0 con 0x7fab9c19ce70 2024-04-25T19:16:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c111e40 con 0x7fab9c101fd0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c112140 con 0x7fab9c1070a0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c19ce70 0x7fab9c19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba1000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 0x7fab9c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba1000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 0x7fab9c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:42084/0 (socket says 172.21.15.86:42084) 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba1000640 1 -- 172.21.15.86:0/2836421826 learned_addr learned my addr 172.21.15.86:0/2836421826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c19ce70 0x7fab9c19d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41516/0 (socket says 172.21.15.86:41516) 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.451+0000 7faba1a00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab9c101fd0 0x7fab9c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba1000640 1 -- 172.21.15.86:0/2836421826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab9c101fd0 msgr2=0x7fab9c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba1000640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab9c101fd0 0x7fab9c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba1000640 1 -- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c19ce70 msgr2=0x7fab9c19d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba1000640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c19ce70 0x7fab9c19d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba1000640 1 -- 172.21.15.86:0/2836421826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c19d960 con 0x7fab9c1070a0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba1000640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 0x7fab9c198b40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab88006010 tx=0x7fab88004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7fab9a000640 1 -- 172.21.15.86:0/2836421826 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8803f030 con 0x7fab9c1070a0 2024-04-25T19:16:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7fab9a000640 1 -- 172.21.15.86:0/2836421826 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab88003aa0 con 0x7fab9c1070a0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c19dbf0 con 0x7fab9c1070a0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7fab9a000640 1 -- 172.21.15.86:0/2836421826 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88035070 con 0x7fab9c1070a0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.452+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c074410 con 0x7fab9c1070a0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.455+0000 7faba717e740 1 Processor -- start 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 -- start start 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7fab9a000640 1 -- 172.21.15.86:0/2836421826 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab88042030 con 0x7fab9c1070a0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fd2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e802f38900 con 0x55e802fd29b0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e802edeef0 con 0x55e802fd32d0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba717e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e802f348c0 con 0x55e802fddb90 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fd2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fd2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41520/0 (socket says 172.21.15.86:41520) 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba1a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba2400640 1 -- 172.21.15.86:0/3627435344 learned_addr learned my addr 172.21.15.86:0/3627435344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.456+0000 7faba1000640 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba2400640 1 -- 172.21.15.86:0/3627435344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 msgr2=0x55e802fdff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba2400640 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba2400640 1 -- 172.21.15.86:0/3627435344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 msgr2=0x55e802fdd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba2400640 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba2400640 1 -- 172.21.15.86:0/3627435344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e802c4b210 con 0x55e802fd29b0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba1a00640 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7faba2400640 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fd2d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fab94009f50 tx=0x7fab94009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1509e1c025f00d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7fab8ea00640 1 -- 172.21.15.86:0/3627435344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94003c30 con 0x55e802fd29b0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7fab8ea00640 1 -- 172.21.15.86:0/3627435344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9401f070 con 0x55e802fd29b0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.457+0000 7fab8ea00640 1 -- 172.21.15.86:0/3627435344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9400b920 con 0x55e802fd29b0 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7fab9a000640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fab840a3d10 0x7fab840a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7fab9a000640 1 -- 172.21.15.86:0/2836421826 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fab880f0c10 con 0x7fab9c1070a0 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba1a00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fab840a3d10 0x7fab840a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 -- 172.21.15.86:0/3627435344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 msgr2=0x55e802fd2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fd2d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fab94009f50 tx=0x7fab94009500 comp rx=0 tx=0).stop 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 -- 172.21.15.86:0/3627435344 shutdown_connections 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 --2- 172.21.15.86:0/3627435344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fd2d90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.458+0000 7faba717e740 1 -- 172.21.15.86:0/3627435344 >> 172.21.15.86:0/3627435344 conn(0x55e802fceec0 msgr2=0x55e802fcf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.459+0000 7faba717e740 1 -- 172.21.15.86:0/3627435344 shutdown_connections 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.459+0000 7faba717e740 1 -- 172.21.15.86:0/3627435344 wait complete. 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.459+0000 7faba1a00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fab840a3d10 0x7fab840a61d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55e802fd3f50 tx=0x7fab9000b390 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.459+0000 7faba717e740 1 -- start start 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.459+0000 7faba717e740 1 Processor -- start 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fdaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e802fe07c0 con 0x55e802fd29b0 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e802ef2310 con 0x55e802fd32d0 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e802fe0940 con 0x55e802fddb90 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba717e740 10 client.0 _ll_register_callbacks cb 0x55e802ec0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fdaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fdaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41524/0 (socket says 172.21.15.86:41524) 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba1a00640 1 -- 172.21.15.86:0/571867772 learned_addr learned my addr 172.21.15.86:0/571867772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba1000640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.460+0000 7faba2400640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1a00640 1 -- 172.21.15.86:0/571867772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 msgr2=0x55e802fdb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1a00640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1a00640 1 -- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 msgr2=0x55e802fdafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1a00640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1a00640 1 -- 172.21.15.86:0/571867772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e802d17620 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba2400640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1000640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba1a00640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fdaaa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fab90003e90 tx=0x7fab90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7fbfb006063170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7fab83400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7fab83400640 1 -- 172.21.15.86:0/571867772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90011650 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7fab83400640 1 -- 172.21.15.86:0/571867772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fab900117f0 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9401f070 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.461+0000 7fab83400640 1 -- 172.21.15.86:0/571867772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab900119f0 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e802feca50 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7fab83400640 1 -- 172.21.15.86:0/571867772 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9401f070 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7fab83400640 1 -- 172.21.15.86:0/571867772 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fab90026070 con 0x55e802fd29b0 2024-04-25T19:16:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 20 client.16818 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e802fedf60 0x55e802ff03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 4 client.16818 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.462+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e802f4c8b0 con 0x55e802fedf60 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.463+0000 7faba1000640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e802fedf60 0x55e802ff03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.463+0000 7faba1000640 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e802fedf60 0x55e802ff03b0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afafb02daff2f70d server_cookie=9fef5f63b9042e81 in_seq=0 out_seq=0 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.463+0000 7fab83400640 10 client.16818 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7fab83400640 1 -- 172.21.15.86:0/571867772 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e802f4c8b0 con 0x55e802fedf60 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7fab83400640 10 client.16818 handle_command_reply: tid=0 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 client.16818 shutdown 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 msgr2=0x55e802fdaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fdaaa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fab90003e90 tx=0x7fab90004730 comp rx=0 tx=0).stop 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 shutdown_connections 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e802fedf60 0x55e802ff03b0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fdb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 --2- 172.21.15.86:0/571867772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd29b0 0x55e802fdaaa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7fab83400640 0 client.16818 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 >> 172.21.15.86:0/571867772 conn(0x55e802fceec0 msgr2=0x55e802fd4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 shutdown_connections 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 1 -- 172.21.15.86:0/571867772 wait complete. 2024-04-25T19:16:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.464+0000 7faba717e740 20 client.16818 trim_cache size 0 max 16384 2024-04-25T19:16:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.480+0000 7faba717e740 1 Processor -- start 2024-04-25T19:16:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 -- start start 2024-04-25T19:16:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fd29b0 0x55e802fe9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fddb90 0x55e802ff5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ab9b0 con 0x55e802fddb90 2024-04-25T19:16:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ad4f0 con 0x55e802fd32d0 2024-04-25T19:16:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba717e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ac080 con 0x55e802fd29b0 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba1a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fd29b0 0x55e802fe9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba1a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fd29b0 0x55e802fe9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:42106/0 (socket says 172.21.15.86:42106) 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba1a00640 1 -- 172.21.15.86:0/675529437 learned_addr learned my addr 172.21.15.86:0/675529437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba2400640 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fddb90 0x55e802ff5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.481+0000 7faba1000640 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7faba2400640 1 -- 172.21.15.86:0/675529437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fd29b0 msgr2=0x55e802fe9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7faba2400640 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fd29b0 0x55e802fe9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7faba2400640 1 -- 172.21.15.86:0/675529437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 msgr2=0x55e802fdc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7faba2400640 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7faba2400640 1 -- 172.21.15.86:0/675529437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e802fd3100 con 0x55e802fddb90 2024-04-25T19:16:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7faba2400640 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fddb90 0x55e802ff5cc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab94009f80 tx=0x7fab94009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b2f782ce66b08d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7fab8e000640 1 -- 172.21.15.86:0/675529437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94028070 con 0x55e802fddb90 2024-04-25T19:16:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7fab8e000640 1 -- 172.21.15.86:0/675529437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9401f040 con 0x55e802fddb90 2024-04-25T19:16:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.482+0000 7fab8e000640 1 -- 172.21.15.86:0/675529437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94004510 con 0x55e802fddb90 2024-04-25T19:16:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 -- 172.21.15.86:0/675529437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fddb90 msgr2=0x55e802ff5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fddb90 0x55e802ff5cc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fab94009f80 tx=0x7fab94009c70 comp rx=0 tx=0).stop 2024-04-25T19:16:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 -- 172.21.15.86:0/675529437 shutdown_connections 2024-04-25T19:16:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fddb90 0x55e802ff5cc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd32d0 0x55e802fdc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 --2- 172.21.15.86:0/675529437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fd29b0 0x55e802fe9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 -- 172.21.15.86:0/675529437 >> 172.21.15.86:0/675529437 conn(0x55e802fceec0 msgr2=0x55e802fdc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 -- 172.21.15.86:0/675529437 shutdown_connections 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.483+0000 7faba717e740 1 -- 172.21.15.86:0/675529437 wait complete. 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.484+0000 7faba717e740 1 -- start start 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.484+0000 7faba717e740 1 Processor -- start 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd29b0 0x55e802fe5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 0x55e802fe3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fe37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ace20 con 0x55e802fd32d0 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1ac750 con 0x55e802fd29b0 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1a99b0 con 0x55e802fddb90 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba717e740 10 client.0 _ll_register_callbacks cb 0x7fab90003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 0x55e802fe3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 0x55e802fe3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41532/0 (socket says 172.21.15.86:41532) 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 -- 172.21.15.86:0/158366645 learned_addr learned my addr 172.21.15.86:0/158366645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1a00640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd29b0 0x55e802fe5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba2400640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fe37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 -- 172.21.15.86:0/158366645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 msgr2=0x55e802fe37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fe37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 -- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd29b0 msgr2=0x55e802fe5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd29b0 0x55e802fe5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.485+0000 7faba1000640 1 -- 172.21.15.86:0/158366645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e802fe3fe0 con 0x55e802fd32d0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7faba1000640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 0x55e802fe3270 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fab880ee8f0 tx=0x7fab8803b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeca4bc68b464f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7fab83400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7fab83400640 1 -- 172.21.15.86:0/158366645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab880d6070 con 0x55e802fd32d0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7fab83400640 1 -- 172.21.15.86:0/158366645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8637+0+0 (secure 0 0 0) 0x7fab88003c50 con 0x55e802fd32d0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9401f040 con 0x55e802fd32d0 2024-04-25T19:16:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7fab83400640 1 -- 172.21.15.86:0/158366645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab88049040 con 0x55e802fd32d0 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7faba717e740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e80305cd90 con 0x55e802fd32d0 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7fab83400640 1 -- 172.21.15.86:0/158366645 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9401f040 con 0x55e802fd32d0 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7fab83400640 1 -- 172.21.15.86:0/158366645 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fab880ee570 con 0x55e802fd32d0 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7faba717e740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.486+0000 7faba717e740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7faba717e740 20 client.16824 populate_metadata read hostname 'smithi086' 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7faba717e740 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e80305d4b0 0x55e80305f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7faba717e740 4 client.16824 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e802f46320 con 0x55e80305d4b0 2024-04-25T19:16:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7faba1a00640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e80305d4b0 0x55e80305f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7fab83400640 10 client.16824 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.487+0000 7faba1a00640 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e80305d4b0 0x55e80305f8a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8109d81b591d8c server_cookie=61d6f07a45319dde in_seq=0 out_seq=0 2024-04-25T19:16:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.501+0000 7fab83400640 1 -- 172.21.15.86:0/158366645 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e802f46320 con 0x55e80305d4b0 2024-04-25T19:16:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.501+0000 7fab83400640 10 client.16824 handle_command_reply: tid=0 2024-04-25T19:16:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.501+0000 7faba717e740 1 client.16824 shutdown 2024-04-25T19:16:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.501+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 msgr2=0x55e802fe3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.501+0000 7faba717e740 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 0x55e802fe3270 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fab880ee8f0 tx=0x7fab8803b640 comp rx=0 tx=0).stop 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 shutdown_connections 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e80305d4b0 0x55e80305f8a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e802fddb90 0x55e802fe37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802fd32d0 0x55e802fe3270 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 --2- 172.21.15.86:0/158366645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e802fd29b0 0x55e802fe5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7fab83400640 0 client.16824 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 >> 172.21.15.86:0/158366645 conn(0x55e802fceec0 msgr2=0x55e802fd4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 shutdown_connections 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 1 -- 172.21.15.86:0/158366645 wait complete. 2024-04-25T19:16:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.502+0000 7faba717e740 20 client.16824 trim_cache size 0 max 16384 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fab840a3d10 msgr2=0x7fab840a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fab840a3d10 0x7fab840a61d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55e802fd3f50 tx=0x7fab9000b390 comp rx=0 tx=0).stop 2024-04-25T19:16:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 msgr2=0x7fab9c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 0x7fab9c198b40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab88006010 tx=0x7fab88004060 comp rx=0 tx=0).stop 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 shutdown_connections 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fab840a3d10 0x7fab840a61d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab9c19ce70 0x7fab9c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab9c1070a0 0x7fab9c198b40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 --2- 172.21.15.86:0/2836421826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab9c101fd0 0x7fab9c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 >> 172.21.15.86:0/2836421826 conn(0x7fab9c0fd890 msgr2=0x7fab9c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 shutdown_connections 2024-04-25T19:16:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:16:51.503+0000 7faba2e00640 1 -- 172.21.15.86:0/2836421826 wait complete. 2024-04-25T19:16:52.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:52 smithi194 ceph-mon[32740]: pgmap v436: 97 pgs: 97 active+clean; 3.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:52 smithi086 ceph-mon[27783]: pgmap v436: 97 pgs: 97 active+clean; 3.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:53.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:52 smithi183 ceph-mon[32344]: pgmap v436: 97 pgs: 97 active+clean; 3.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:54.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:54 smithi194 ceph-mon[32740]: pgmap v437: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-25T19:16:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:54 smithi086 ceph-mon[27783]: pgmap v437: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-25T19:16:55.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:54 smithi183 ceph-mon[32344]: pgmap v437: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-25T19:16:56.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:56 smithi194 ceph-mon[32740]: pgmap v438: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:57.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:56 smithi086 ceph-mon[27783]: pgmap v438: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:57.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:56 smithi183 ceph-mon[32344]: pgmap v438: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:58.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:16:58 smithi194 ceph-mon[32740]: pgmap v439: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:16:58 smithi086 ceph-mon[27783]: pgmap v439: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:16:59.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:16:58 smithi183 ceph-mon[32344]: pgmap v439: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:00.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:00 smithi194 ceph-mon[32740]: pgmap v440: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:17:01.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:00 smithi086 ceph-mon[27783]: pgmap v440: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:17:01.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:00 smithi183 ceph-mon[32344]: pgmap v440: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:17:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:02 smithi194 ceph-mon[32740]: pgmap v441: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:02.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:02 smithi194 ceph-mon[32740]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T19:17:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:02 smithi086 ceph-mon[27783]: pgmap v441: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:02 smithi086 ceph-mon[27783]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T19:17:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:02 smithi183 ceph-mon[32344]: pgmap v441: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:03.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:02 smithi183 ceph-mon[32344]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T19:17:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 -- 172.21.15.86:0/2427748730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0112dd0 msgr2=0x7f65f01151c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2427748730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0112dd0 0x7f65f01151c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009900 tx=0x7f65e002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 -- 172.21.15.86:0/2427748730 shutdown_connections 2024-04-25T19:17:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2427748730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0112dd0 0x7f65f01151c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2427748730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076da0 0x7f65f0112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2427748730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65f0076480 0x7f65f0076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 -- 172.21.15.86:0/2427748730 >> 172.21.15.86:0/2427748730 conn(0x7f65f00ffc40 msgr2=0x7f65f0102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 -- 172.21.15.86:0/2427748730 shutdown_connections 2024-04-25T19:17:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.647+0000 7f65f5c00640 1 -- 172.21.15.86:0/2427748730 wait complete. 2024-04-25T19:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 Processor -- start 2024-04-25T19:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 -- start start 2024-04-25T19:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076480 0x7f65f01a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 0x7f65f01a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65f0112dd0 0x7f65f019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01187f0 con 0x7f65f0076da0 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0118670 con 0x7f65f0112dd0 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.649+0000 7f65f5c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0118970 con 0x7f65f0076480 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 0x7f65f01a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076480 0x7f65f01a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 0x7f65f01a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51322/0 (socket says 172.21.15.86:51322) 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65ee000640 1 -- 172.21.15.86:0/3879645670 learned_addr learned my addr 172.21.15.86:0/3879645670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65ef400640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65f0112dd0 0x7f65f019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 -- 172.21.15.86:0/3879645670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65f0112dd0 msgr2=0x7f65f019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65f0112dd0 0x7f65f019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 -- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 msgr2=0x7f65f01a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 0x7f65f01a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 -- 172.21.15.86:0/3879645670 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f019f1d0 con 0x7f65f0076480 2024-04-25T19:17:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65ee000640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 0x7f65f01a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.650+0000 7f65eea00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076480 0x7f65f01a48a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e9f0 tx=0x7f65e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.651+0000 7f65e7400640 1 -- 172.21.15.86:0/3879645670 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800ccc0 con 0x7f65f0076480 2024-04-25T19:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.651+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f019f4c0 con 0x7f65f0076480 2024-04-25T19:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.651+0000 7f65e7400640 1 -- 172.21.15.86:0/3879645670 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e80108e0 con 0x7f65f0076480 2024-04-25T19:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.651+0000 7f65e7400640 1 -- 172.21.15.86:0/3879645670 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8018880 con 0x7f65f0076480 2024-04-25T19:17:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.651+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f01ab760 con 0x7f65f0076480 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65f8a37740 1 Processor -- start 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65f8a37740 1 -- start start 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65e7400640 1 -- 172.21.15.86:0/3879645670 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65e8010430 con 0x7f65f0076480 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84eb99c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eba44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.654+0000 7f65f8a37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e84eaff900 con 0x55e84eb99820 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65f8a37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e84eaa5ef0 con 0x55e84eba4a00 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65f8a37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e84eafb8c0 con 0x55e84eb9a140 2024-04-25T19:17:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84eb99c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84eb99c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51326/0 (socket says 172.21.15.86:51326) 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 -- 172.21.15.86:0/547645219 learned_addr learned my addr 172.21.15.86:0/547645219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65eea00640 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eba44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ee000640 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 -- 172.21.15.86:0/547645219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 msgr2=0x55e84eba44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eba44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 -- 172.21.15.86:0/547645219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 msgr2=0x55e84eba6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.655+0000 7f65ef400640 1 -- 172.21.15.86:0/547645219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84e812210 con 0x55e84eb99820 2024-04-25T19:17:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65ef400640 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84eb99c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005ec0 tx=0x7f65e0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5ad06d08eab728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65e7400640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f65d40a3e40 0x7f65d40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65dfe00640 1 -- 172.21.15.86:0/547645219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0040070 con 0x55e84eb99820 2024-04-25T19:17:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65dfe00640 1 -- 172.21.15.86:0/547645219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e00043c0 con 0x55e84eb99820 2024-04-25T19:17:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65dfe00640 1 -- 172.21.15.86:0/547645219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0035040 con 0x55e84eb99820 2024-04-25T19:17:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65e7400640 1 -- 172.21.15.86:0/3879645670 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8681+0+0 (secure 0 0 0) 0x7f65e80d0b90 con 0x7f65f0076480 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.656+0000 7f65eea00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f65d40a3e40 0x7f65d40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 -- 172.21.15.86:0/547645219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 msgr2=0x55e84eb99c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84eb99c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005ec0 tx=0x7f65e0005c00 comp rx=0 tx=0).stop 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65eea00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f65d40a3e40 0x7f65d40a6300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55e84eb9adc0 tx=0x7f65e80d4640 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 -- 172.21.15.86:0/547645219 shutdown_connections 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eba44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 --2- 172.21.15.86:0/547645219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84eb99c00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 -- 172.21.15.86:0/547645219 >> 172.21.15.86:0/547645219 conn(0x55e84eb95d30 msgr2=0x55e84eb96110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 -- 172.21.15.86:0/547645219 shutdown_connections 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 -- 172.21.15.86:0/547645219 wait complete. 2024-04-25T19:17:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.657+0000 7f65f8a37740 1 -- start start 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 Processor -- start 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb99820 0x55e84eba19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 0x55e84eba1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e84eba75d0 con 0x55e84eb9a140 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e84eab9310 con 0x55e84eba4a00 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e84eba7750 con 0x55e84eb99820 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65f8a37740 10 client.0 _ll_register_callbacks cb 0x55e84ea87cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.658+0000 7f65ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 0x55e84eba1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 0x55e84eba1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51340/0 (socket says 172.21.15.86:51340) 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 -- 172.21.15.86:0/1325926411 learned_addr learned my addr 172.21.15.86:0/1325926411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ef400640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb99820 0x55e84eba19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ef400640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 -- 172.21.15.86:0/1325926411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb99820 msgr2=0x55e84eba19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb99820 0x55e84eba19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 -- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 msgr2=0x55e84eba2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 -- 172.21.15.86:0/1325926411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84e8de620 con 0x55e84eb9a140 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ef400640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65ee000640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 0x55e84eba1f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f65d8012dd0 tx=0x7f65d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195c1c1977018789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.659+0000 7f65dcc00640 1 -- 172.21.15.86:0/1325926411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8003970 con 0x55e84eb9a140 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65dcc00640 1 -- 172.21.15.86:0/1325926411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8681+0+0 (secure 0 0 0) 0x7f65d8003b10 con 0x55e84eb9a140 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e00043c0 con 0x55e84eb9a140 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65dcc00640 1 -- 172.21.15.86:0/1325926411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65d8003d10 con 0x55e84eb9a140 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65f8a37740 10 client.16836 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e84ebb38e0 con 0x55e84eb9a140 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65dcc00640 1 -- 172.21.15.86:0/1325926411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e00043c0 con 0x55e84eb9a140 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65dcc00640 1 -- 172.21.15.86:0/1325926411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f65d801c460 con 0x55e84eb9a140 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65f8a37740 10 client.16836 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65f8a37740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.660+0000 7f65f8a37740 20 client.16836 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.661+0000 7f65f8a37740 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ebb4df0 0x55e84ebb7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.661+0000 7f65f8a37740 4 client.16836 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.661+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e84eb138b0 con 0x55e84ebb4df0 2024-04-25T19:17:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.661+0000 7f65ef400640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ebb4df0 0x55e84ebb7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.661+0000 7f65ef400640 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ebb4df0 0x55e84ebb7200 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a5c4519563dfb2 server_cookie=fbb299492614c010 in_seq=0 out_seq=0 2024-04-25T19:17:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.661+0000 7f65dcc00640 10 client.16836 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65dcc00640 1 -- 172.21.15.86:0/1325926411 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e84eb138b0 con 0x55e84ebb4df0 2024-04-25T19:17:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65dcc00640 10 client.16836 handle_command_reply: tid=0 2024-04-25T19:17:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65f8a37740 1 client.16836 shutdown 2024-04-25T19:17:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 msgr2=0x55e84eba1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65f8a37740 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 0x55e84eba1f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f65d8012dd0 tx=0x7f65d800f790 comp rx=0 tx=0).stop 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 shutdown_connections 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.662+0000 7f65f8a37740 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ebb4df0 0x55e84ebb7200 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eba4a00 0x55e84eba2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb9a140 0x55e84eba1f10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 1 --2- 172.21.15.86:0/1325926411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb99820 0x55e84eba19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 >> 172.21.15.86:0/1325926411 conn(0x55e84eb95d30 msgr2=0x55e84eb9ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65dcc00640 0 client.16836 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 shutdown_connections 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 1 -- 172.21.15.86:0/1325926411 wait complete. 2024-04-25T19:17:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.663+0000 7f65f8a37740 20 client.16836 trim_cache size 0 max 16384 2024-04-25T19:17:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 Processor -- start 2024-04-25T19:17:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 -- start start 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb99820 0x55e84ebb0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eb9c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eba4a00 0x55e84ebbcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b4410 con 0x55e84eba4a00 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b5f50 con 0x55e84eb99820 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.679+0000 7f65f8a37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b4ae0 con 0x55e84eb9a140 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ef400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb99820 0x55e84ebb0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eb9c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ef400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb99820 0x55e84ebb0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:60326/0 (socket says 172.21.15.86:60326) 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eb9c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:60938/0 (socket says 172.21.15.86:60938) 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ef400640 1 -- 172.21.15.86:0/504559300 learned_addr learned my addr 172.21.15.86:0/504559300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ef400640 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eba4a00 0x55e84ebbcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 -- 172.21.15.86:0/504559300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb99820 msgr2=0x55e84ebb0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb99820 0x55e84ebb0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 -- 172.21.15.86:0/504559300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eba4a00 msgr2=0x55e84ebbcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eba4a00 0x55e84ebbcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 -- 172.21.15.86:0/504559300 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84eb99f70 con 0x55e84eb9a140 2024-04-25T19:17:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.680+0000 7f65ee000640 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eb9c1e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f65d8012da0 tx=0x7f65d8003b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460d9a1d35ccb1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65df400640 1 -- 172.21.15.86:0/504559300 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d801c460 con 0x55e84eb9a140 2024-04-25T19:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65df400640 1 -- 172.21.15.86:0/504559300 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d801c600 con 0x55e84eb9a140 2024-04-25T19:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65df400640 1 -- 172.21.15.86:0/504559300 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8005590 con 0x55e84eb9a140 2024-04-25T19:17:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 -- 172.21.15.86:0/504559300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 msgr2=0x55e84eb9c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eb9c1e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f65d8012da0 tx=0x7f65d8003b00 comp rx=0 tx=0).stop 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 -- 172.21.15.86:0/504559300 shutdown_connections 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eba4a00 0x55e84ebbcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eb9a140 0x55e84eb9c1e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 --2- 172.21.15.86:0/504559300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb99820 0x55e84ebb0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 -- 172.21.15.86:0/504559300 >> 172.21.15.86:0/504559300 conn(0x55e84eb95d30 msgr2=0x55e84eb9cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 -- 172.21.15.86:0/504559300 shutdown_connections 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.681+0000 7f65f8a37740 1 -- 172.21.15.86:0/504559300 wait complete. 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.682+0000 7f65f8a37740 1 -- start start 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.682+0000 7f65f8a37740 1 Processor -- start 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.682+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84ebacde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.682+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb9a140 0x55e84ebaa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.682+0000 7f65f8a37740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eba4a00 0x55e84ebaa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.682+0000 7f65f8a37740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b5880 con 0x55e84eb99820 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65f8a37740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b51b0 con 0x55e84eb9a140 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65f8a37740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b2410 con 0x55e84eba4a00 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65f8a37740 10 client.0 _ll_register_callbacks cb 0x7f65d80129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84ebacde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84ebacde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51358/0 (socket says 172.21.15.86:51358) 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 -- 172.21.15.86:0/4004038922 learned_addr learned my addr 172.21.15.86:0/4004038922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ee000640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb9a140 0x55e84ebaa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eba4a00 0x55e84ebaa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 -- 172.21.15.86:0/4004038922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eba4a00 msgr2=0x55e84ebaa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eba4a00 0x55e84ebaa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 -- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb9a140 msgr2=0x55e84ebaa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb9a140 0x55e84ebaa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 -- 172.21.15.86:0/4004038922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84ebaaf00 con 0x55e84eb99820 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65ef400640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84ebacde0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009850 tx=0x7f65e0048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612e34dd3c6a0acf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.683+0000 7f65dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65dcc00640 1 -- 172.21.15.86:0/4004038922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e005b060 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65dcc00640 1 -- 172.21.15.86:0/4004038922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8681+0+0 (secure 0 0 0) 0x7f65e0050310 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d801c600 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65dcc00640 1 -- 172.21.15.86:0/4004038922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65e0035040 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65f8a37740 10 client.16842 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e84ec1b490 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65dcc00640 1 -- 172.21.15.86:0/4004038922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d801c600 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.684+0000 7f65dcc00640 1 -- 172.21.15.86:0/4004038922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f65e0046b40 con 0x55e84eb99820 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65f8a37740 10 client.16842 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65f8a37740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65f8a37740 20 client.16842 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65f8a37740 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ec1b820 0x55e84ec1dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65f8a37740 4 client.16842 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:17:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e84eb0d320 con 0x55e84ec1b820 2024-04-25T19:17:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65ee000640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ec1b820 0x55e84ec1dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65ee000640 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ec1b820 0x55e84ec1dc10 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfef8847cdfdb301 server_cookie=8f0d2829155cf521 in_seq=0 out_seq=0 2024-04-25T19:17:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.685+0000 7f65dcc00640 10 client.16842 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.700+0000 7f65dcc00640 1 -- 172.21.15.86:0/4004038922 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e84eb0d320 con 0x55e84ec1b820 2024-04-25T19:17:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.700+0000 7f65dcc00640 10 client.16842 handle_command_reply: tid=0 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 client.16842 shutdown 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 msgr2=0x55e84ebacde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84ebacde0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009850 tx=0x7f65e0048040 comp rx=0 tx=0).stop 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 shutdown_connections 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e84ec1b820 0x55e84ec1dc10 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e84eba4a00 0x55e84ebaa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e84eb9a140 0x55e84ebaa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65dcc00640 0 client.16842 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 --2- 172.21.15.86:0/4004038922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e84eb99820 0x55e84ebacde0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 >> 172.21.15.86:0/4004038922 conn(0x55e84eb95d30 msgr2=0x55e84eb94600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 shutdown_connections 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 1 -- 172.21.15.86:0/4004038922 wait complete. 2024-04-25T19:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.701+0000 7f65f8a37740 20 client.16842 trim_cache size 0 max 16384 2024-04-25T19:17:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:17:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:17:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:17:04.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f65d40a3e40 msgr2=0x7f65d40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f65d40a3e40 0x7f65d40a6300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55e84eb9adc0 tx=0x7f65e80d4640 comp rx=0 tx=0).stop 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076480 msgr2=0x7f65f01a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076480 0x7f65f01a48a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e9f0 tx=0x7f65e800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 shutdown_connections 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f65d40a3e40 0x7f65d40a6300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65f0112dd0 0x7f65f019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0076da0 0x7f65f01a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3879645670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f0076480 0x7f65f01a48a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:04.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 >> 172.21.15.86:0/3879645670 conn(0x7f65f00ffc40 msgr2=0x7f65f0102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 shutdown_connections 2024-04-25T19:17:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:04.702+0000 7f65f5c00640 1 -- 172.21.15.86:0/3879645670 wait complete. 2024-04-25T19:17:04.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:04 smithi194 ceph-mon[32740]: pgmap v443: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:05.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:04 smithi086 ceph-mon[27783]: pgmap v443: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:05.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:04 smithi183 ceph-mon[32344]: pgmap v443: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:06.969 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:06 smithi183 ceph-mon[32344]: pgmap v444: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:06.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:06 smithi194 ceph-mon[32740]: pgmap v444: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:06 smithi086 ceph-mon[27783]: pgmap v444: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:08 smithi194 ceph-mon[32740]: pgmap v445: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:08.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:08 smithi194 ceph-mon[32740]: osdmap e112: 12 total, 12 up, 12 in 2024-04-25T19:17:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:08 smithi086 ceph-mon[27783]: pgmap v445: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:08 smithi086 ceph-mon[27783]: osdmap e112: 12 total, 12 up, 12 in 2024-04-25T19:17:09.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:08 smithi183 ceph-mon[32344]: pgmap v445: 97 pgs: 97 active+clean; 3.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-04-25T19:17:09.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:08 smithi183 ceph-mon[32344]: osdmap e112: 12 total, 12 up, 12 in 2024-04-25T19:17:10.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:10 smithi194 ceph-mon[32740]: pgmap v447: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 KiB/s wr, 48 op/s 2024-04-25T19:17:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:10 smithi086 ceph-mon[27783]: pgmap v447: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 KiB/s wr, 48 op/s 2024-04-25T19:17:11.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:10 smithi183 ceph-mon[32344]: pgmap v447: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 KiB/s wr, 48 op/s 2024-04-25T19:17:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:12 smithi086 ceph-mon[27783]: pgmap v448: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 108 KiB/s wr, 40 op/s 2024-04-25T19:17:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:12 smithi183 ceph-mon[32344]: pgmap v448: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 108 KiB/s wr, 40 op/s 2024-04-25T19:17:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:12 smithi194 ceph-mon[32740]: pgmap v448: 97 pgs: 97 active+clean; 3.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 108 KiB/s wr, 40 op/s 2024-04-25T19:17:15.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:14 smithi086 ceph-mon[27783]: pgmap v449: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:15.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:14 smithi183 ceph-mon[32344]: pgmap v449: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:15.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:14 smithi194 ceph-mon[32740]: pgmap v449: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:17.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:16 smithi086 ceph-mon[27783]: pgmap v450: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:17.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:16 smithi183 ceph-mon[32344]: pgmap v450: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:16 smithi194 ceph-mon[32740]: pgmap v450: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 -- 172.21.15.86:0/2144028419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0141043c0 msgr2=0x7ff0141047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 --2- 172.21.15.86:0/2144028419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0141043c0 0x7ff0141047a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff004009ca0 tx=0x7ff00402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 -- 172.21.15.86:0/2144028419 shutdown_connections 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 --2- 172.21.15.86:0/2144028419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff014109490 0x7ff014111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 --2- 172.21.15.86:0/2144028419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014104d70 0x7ff014108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 --2- 172.21.15.86:0/2144028419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0141043c0 0x7ff0141047a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 -- 172.21.15.86:0/2144028419 >> 172.21.15.86:0/2144028419 conn(0x7ff0140ffc20 msgr2=0x7ff014102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 -- 172.21.15.86:0/2144028419 shutdown_connections 2024-04-25T19:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.847+0000 7ff018a00640 1 -- 172.21.15.86:0/2144028419 wait complete. 2024-04-25T19:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 Processor -- start 2024-04-25T19:17:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 -- start start 2024-04-25T19:17:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0141043c0 0x7ff0141a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 0x7ff0141a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01419a6c0 0x7ff01419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff014114420 con 0x7ff014109490 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141142a0 con 0x7ff0141043c0 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff018a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141145a0 con 0x7ff01419a6c0 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.849+0000 7ff009400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 0x7ff0141a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff011e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0141043c0 0x7ff0141a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 0x7ff0141a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39430/0 (socket says 172.21.15.86:39430) 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 -- 172.21.15.86:0/2291336564 learned_addr learned my addr 172.21.15.86:0/2291336564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff012800640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01419a6c0 0x7ff01419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 -- 172.21.15.86:0/2291336564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01419a6c0 msgr2=0x7ff01419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01419a6c0 0x7ff01419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 -- 172.21.15.86:0/2291336564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0141043c0 msgr2=0x7ff0141a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0141043c0 0x7ff0141a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 -- 172.21.15.86:0/2291336564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01419b2c0 con 0x7ff014109490 2024-04-25T19:17:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.850+0000 7ff009400640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 0x7ff0141a0b40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9c0 tx=0x7feffc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.851+0000 7ff00b400640 1 -- 172.21.15.86:0/2291336564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc00cd80 con 0x7ff014109490 2024-04-25T19:17:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.851+0000 7ff00b400640 1 -- 172.21.15.86:0/2291336564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc0054d0 con 0x7ff014109490 2024-04-25T19:17:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.851+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0141a73a0 con 0x7ff014109490 2024-04-25T19:17:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.851+0000 7ff00b400640 1 -- 172.21.15.86:0/2291336564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc010430 con 0x7ff014109490 2024-04-25T19:17:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.851+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a77a0 con 0x7ff014109490 2024-04-25T19:17:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff00b400640 1 -- 172.21.15.86:0/2291336564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feffc004fd0 con 0x7ff014109490 2024-04-25T19:17:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 Processor -- start 2024-04-25T19:17:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 -- start start 2024-04-25T19:17:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 0x55821eed0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eedb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eedba00 0x55821eedddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55821ee36900 con 0x55821eed1140 2024-04-25T19:17:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55821eddcef0 con 0x55821eedba00 2024-04-25T19:17:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.855+0000 7ff01b96f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55821ee328c0 con 0x55821eed0820 2024-04-25T19:17:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff011e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eedb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff009400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eedba00 0x55821eedddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff012800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 0x55821eed0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff012800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 0x55821eed0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:44734/0 (socket says 172.21.15.86:44734) 2024-04-25T19:17:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff011e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eedb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39436/0 (socket says 172.21.15.86:39436) 2024-04-25T19:17:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff012800640 1 -- 172.21.15.86:0/3374964361 learned_addr learned my addr 172.21.15.86:0/3374964361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff009400640 1 -- 172.21.15.86:0/3374964361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 msgr2=0x55821eed0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff009400640 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 0x55821eed0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff009400640 1 -- 172.21.15.86:0/3374964361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 msgr2=0x55821eedb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff009400640 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eedb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff009400640 1 -- 172.21.15.86:0/3374964361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55821eb49210 con 0x55821eedba00 2024-04-25T19:17:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff012800640 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 0x55821eed0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff011e00640 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eedb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.856+0000 7ff00b400640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feff80a3dc0 0x7feff80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.857+0000 7ff009400640 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eedba00 0x55821eedddf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feffc0ce350 tx=0x7feffc0d4db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f050497c8cd3bc20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.857+0000 7ff00b400640 1 -- 172.21.15.86:0/2291336564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7feffc0d0c70 con 0x7ff014109490 2024-04-25T19:17:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.857+0000 7ff012800640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feff80a3dc0 0x7feff80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.857+0000 7ff003000640 1 -- 172.21.15.86:0/3374964361 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc098e40 con 0x55821eedba00 2024-04-25T19:17:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.857+0000 7ff003000640 1 -- 172.21.15.86:0/3374964361 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc0046c0 con 0x55821eedba00 2024-04-25T19:17:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.857+0000 7ff003000640 1 -- 172.21.15.86:0/3374964361 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc09c500 con 0x55821eedba00 2024-04-25T19:17:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 -- 172.21.15.86:0/3374964361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eedba00 msgr2=0x55821eedddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eedba00 0x55821eedddf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feffc0ce350 tx=0x7feffc0d4db0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff012800640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feff80a3dc0 0x7feff80a6280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0124a0 tx=0x7ff00c005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 -- 172.21.15.86:0/3374964361 shutdown_connections 2024-04-25T19:17:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eedba00 0x55821eedddf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eedb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 --2- 172.21.15.86:0/3374964361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed0820 0x55821eed0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 -- 172.21.15.86:0/3374964361 >> 172.21.15.86:0/3374964361 conn(0x55821eeccd30 msgr2=0x55821eecd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 -- 172.21.15.86:0/3374964361 shutdown_connections 2024-04-25T19:17:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 -- 172.21.15.86:0/3374964361 wait complete. 2024-04-25T19:17:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 -- start start 2024-04-25T19:17:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.858+0000 7ff01b96f740 1 Processor -- start 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 0x55821eed89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed1140 0x55821eed8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eed9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55821eede630 con 0x55821eed0820 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55821edf0310 con 0x55821eed1140 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55821eede7b0 con 0x55821eedba00 2024-04-25T19:17:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff01b96f740 10 client.0 _ll_register_callbacks cb 0x55821edbecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff011e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 0x55821eed89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff011e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 0x55821eed89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39450/0 (socket says 172.21.15.86:39450) 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff011e00640 1 -- 172.21.15.86:0/2792747247 learned_addr learned my addr 172.21.15.86:0/2792747247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff009400640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed1140 0x55821eed8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.859+0000 7ff012800640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eed9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff011e00640 1 -- 172.21.15.86:0/2792747247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 msgr2=0x55821eed9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff011e00640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eed9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff011e00640 1 -- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed1140 msgr2=0x55821eed8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff011e00640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed1140 0x55821eed8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff011e00640 1 -- 172.21.15.86:0/2792747247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55821ec15620 con 0x55821eed0820 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff009400640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed1140 0x55821eed8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff011e00640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 0x55821eed89d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff00402ade0 tx=0x7ff004031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb064557e54f742e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7feff7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7feff7400640 1 -- 172.21.15.86:0/2792747247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004031750 con 0x55821eed0820 2024-04-25T19:17:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7feff7400640 1 -- 172.21.15.86:0/2792747247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7ff004003970 con 0x55821eed0820 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.860+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55821ee4a8b0 con 0x55821eed0820 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7feff7400640 1 -- 172.21.15.86:0/2792747247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0040318f0 con 0x55821eed0820 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 10 client.16854 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55821eeea8e0 con 0x55821eed0820 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7feff7400640 1 -- 172.21.15.86:0/2792747247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55821ee4a8b0 con 0x55821eed0820 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7feff7400640 1 -- 172.21.15.86:0/2792747247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff00402f610 con 0x55821eed0820 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 10 client.16854 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 20 client.16854 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821eeebdf0 0x55821eeee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 4 client.16854 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.861+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55821ee44320 con 0x55821eeebdf0 2024-04-25T19:17:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.862+0000 7ff009400640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821eeebdf0 0x55821eeee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.862+0000 7ff009400640 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821eeebdf0 0x55821eeee200 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fc90281ac66847 server_cookie=6d0b97d2f40c9417 in_seq=0 out_seq=0 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.862+0000 7feff7400640 10 client.16854 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7feff7400640 1 -- 172.21.15.86:0/2792747247 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55821ee44320 con 0x55821eeebdf0 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7feff7400640 10 client.16854 handle_command_reply: tid=0 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 client.16854 shutdown 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 msgr2=0x55821eed89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 0x55821eed89d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff00402ade0 tx=0x7ff004031450 comp rx=0 tx=0).stop 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 shutdown_connections 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821eeebdf0 0x55821eeee200 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eed9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed1140 0x55821eed8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7feff7400640 0 client.16854 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 --2- 172.21.15.86:0/2792747247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed0820 0x55821eed89d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.863+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 >> 172.21.15.86:0/2792747247 conn(0x55821eeccd30 msgr2=0x55821eed2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.864+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 shutdown_connections 2024-04-25T19:17:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.864+0000 7ff01b96f740 1 -- 172.21.15.86:0/2792747247 wait complete. 2024-04-25T19:17:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.864+0000 7ff01b96f740 20 client.16854 trim_cache size 0 max 16384 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 Processor -- start 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 -- start start 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eed31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eecbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141afed0 con 0x55821eed1140 2024-04-25T19:17:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141b1a10 con 0x55821eed0820 2024-04-25T19:17:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff01b96f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141b05a0 con 0x55821eedba00 2024-04-25T19:17:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.880+0000 7ff009400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eed31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eed31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39454/0 (socket says 172.21.15.86:39454) 2024-04-25T19:17:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 -- 172.21.15.86:0/4010460936 learned_addr learned my addr 172.21.15.86:0/4010460936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff012800640 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eecbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff011e00640 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 -- 172.21.15.86:0/4010460936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 msgr2=0x55821eecbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eecbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 -- 172.21.15.86:0/4010460936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 msgr2=0x55821eee74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 -- 172.21.15.86:0/4010460936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55821eecdf60 con 0x55821eed1140 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff011e00640 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:17:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff009400640 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eed31e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feffc0d4240 tx=0x7feffc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ad8516e3470dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.881+0000 7ff002600640 1 -- 172.21.15.86:0/4010460936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc09c6e0 con 0x55821eed1140 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff002600640 1 -- 172.21.15.86:0/4010460936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc0d4590 con 0x55821eed1140 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff002600640 1 -- 172.21.15.86:0/4010460936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc09c970 con 0x55821eed1140 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 -- 172.21.15.86:0/4010460936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 msgr2=0x55821eed31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eed31e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feffc0d4240 tx=0x7feffc005880 comp rx=0 tx=0).stop 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 -- 172.21.15.86:0/4010460936 shutdown_connections 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eedba00 0x55821eecbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eed1140 0x55821eed31e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 --2- 172.21.15.86:0/4010460936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 -- 172.21.15.86:0/4010460936 >> 172.21.15.86:0/4010460936 conn(0x55821eeccd30 msgr2=0x55821eecb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 -- 172.21.15.86:0/4010460936 shutdown_connections 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.882+0000 7ff01b96f740 1 -- 172.21.15.86:0/4010460936 wait complete. 2024-04-25T19:17:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 -- start start 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 Processor -- start 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed1140 0x55821eee1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 0x55821eee15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141b1340 con 0x55821eedba00 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141b0c70 con 0x55821eed0820 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.883+0000 7ff01b96f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141adf20 con 0x55821eed1140 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff01b96f740 10 client.0 _ll_register_callbacks cb 0x7ff004009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff012800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 0x55821eee15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff009400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed1140 0x55821eee1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff012800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 0x55821eee15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39464/0 (socket says 172.21.15.86:39464) 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff011e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff012800640 1 -- 172.21.15.86:0/827245037 learned_addr learned my addr 172.21.15.86:0/827245037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff009400640 1 -- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 msgr2=0x55821eee3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff009400640 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff009400640 1 -- 172.21.15.86:0/827245037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 msgr2=0x55821eee15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff009400640 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 0x55821eee15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff009400640 1 -- 172.21.15.86:0/827245037 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55821eee1ca0 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.884+0000 7ff012800640 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 0x55821eee15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7ff009400640 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed1140 0x55821eee1070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55821eec7e70 tx=0x7feffc00efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d07bd5bda5158c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7feff7400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7feff7400640 1 -- 172.21.15.86:0/827245037 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc09c970 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feffc0d4590 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7feff7400640 1 -- 172.21.15.86:0/827245037 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7feffc09ac40 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7feff7400640 1 -- 172.21.15.86:0/827245037 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feffc09c6e0 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7ff01b96f740 10 client.26953 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55821ef52260 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.885+0000 7feff7400640 1 -- 172.21.15.86:0/827245037 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc0d4590 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7feff7400640 1 -- 172.21.15.86:0/827245037 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feffc099210 con 0x55821eed1140 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff01b96f740 10 client.26953 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff01b96f740 10 client.26953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff01b96f740 20 client.26953 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff01b96f740 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821ef52490 0x55821ef54880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff01b96f740 4 client.26953 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:17:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55821ee3e190 con 0x55821ef52490 2024-04-25T19:17:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.886+0000 7ff011e00640 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821ef52490 0x55821ef54880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.887+0000 7ff011e00640 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821ef52490 0x55821ef54880 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcad96ed5e6bf7e1 server_cookie=a4dad1a8a08c8a7f in_seq=0 out_seq=0 2024-04-25T19:17:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.887+0000 7feff7400640 10 client.26953 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7feff7400640 1 -- 172.21.15.86:0/827245037 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55821ee3e190 con 0x55821ef52490 2024-04-25T19:17:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7feff7400640 10 client.26953 handle_command_reply: tid=0 2024-04-25T19:17:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 client.26953 shutdown 2024-04-25T19:17:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed1140 msgr2=0x55821eee1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed1140 0x55821eee1070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55821eec7e70 tx=0x7feffc00efd0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 shutdown_connections 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55821ef52490 0x55821ef54880 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55821eedba00 0x55821eee15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55821eed1140 0x55821eee1070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7feff7400640 0 client.26953 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 --2- 172.21.15.86:0/827245037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55821eed0820 0x55821eee3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 >> 172.21.15.86:0/827245037 conn(0x55821eeccd30 msgr2=0x55821eecbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 shutdown_connections 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.902+0000 7ff01b96f740 1 -- 172.21.15.86:0/827245037 wait complete. 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff01b96f740 20 client.26953 trim_cache size 0 max 16384 2024-04-25T19:17:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:17:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:17:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:17:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:17:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feff80a3dc0 msgr2=0x7feff80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feff80a3dc0 0x7feff80a6280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0124a0 tx=0x7ff00c005f90 comp rx=0 tx=0).stop 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 msgr2=0x7ff0141a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 0x7ff0141a0b40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9c0 tx=0x7feffc00ee80 comp rx=0 tx=0).stop 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 shutdown_connections 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feff80a3dc0 0x7feff80a6280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.903+0000 7ff018a00640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01419a6c0 0x7ff01419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.904+0000 7ff018a00640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014109490 0x7ff0141a0b40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.904+0000 7ff018a00640 1 --2- 172.21.15.86:0/2291336564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0141043c0 0x7ff0141a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.904+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 >> 172.21.15.86:0/2291336564 conn(0x7ff0140ffc20 msgr2=0x7ff014101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.904+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 shutdown_connections 2024-04-25T19:17:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:17.904+0000 7ff018a00640 1 -- 172.21.15.86:0/2291336564 wait complete. 2024-04-25T19:17:19.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:18 smithi086 ceph-mon[27783]: pgmap v451: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:19.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:18 smithi183 ceph-mon[32344]: pgmap v451: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:19.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:18 smithi194 ceph-mon[32740]: pgmap v451: 97 pgs: 97 active+clean; 4.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T19:17:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:20 smithi086 ceph-mon[27783]: pgmap v452: 97 pgs: 97 active+clean; 4.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-25T19:17:21.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:20 smithi183 ceph-mon[32344]: pgmap v452: 97 pgs: 97 active+clean; 4.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-25T19:17:21.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:20 smithi194 ceph-mon[32740]: pgmap v452: 97 pgs: 97 active+clean; 4.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-25T19:17:23.034 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:22 smithi194 ceph-mon[32740]: pgmap v453: 97 pgs: 97 active+clean; 4.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:17:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:22 smithi086 ceph-mon[27783]: pgmap v453: 97 pgs: 97 active+clean; 4.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:17:23.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:22 smithi183 ceph-mon[32344]: pgmap v453: 97 pgs: 97 active+clean; 4.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:17:23.825 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-04-25T19:17:23.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:17:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:17:25.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:24 smithi086 ceph-mon[27783]: pgmap v454: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:17:25.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:24 smithi183 ceph-mon[32344]: pgmap v454: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:17:25.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:24 smithi194 ceph-mon[32740]: pgmap v454: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 -- 172.21.15.86:0/171125359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0101970 msgr2=0x7f92b0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 --2- 172.21.15.86:0/171125359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0101970 0x7f92b0101df0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f92980099c0 tx=0x7f9298009cd0 comp rx=0 tx=0).stop 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 -- 172.21.15.86:0/171125359 shutdown_connections 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 --2- 172.21.15.86:0/171125359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0102330 0x7f92b010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 --2- 172.21.15.86:0/171125359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92b01079c0 0x7f92b0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 --2- 172.21.15.86:0/171125359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0101970 0x7f92b0101df0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 -- 172.21.15.86:0/171125359 >> 172.21.15.86:0/171125359 conn(0x7f92b0078980 msgr2=0x7f92b0078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:26.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.722+0000 7f92b5a00700 1 -- 172.21.15.86:0/171125359 shutdown_connections 2024-04-25T19:17:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.723+0000 7f92b5a00700 1 -- 172.21.15.86:0/171125359 wait complete. 2024-04-25T19:17:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.723+0000 7f92b5a00700 1 Processor -- start 2024-04-25T19:17:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 -- start start 2024-04-25T19:17:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 0x7f92b0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0102330 0x7f92b0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92b01079c0 0x7f92b019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0111b40 con 0x7f92b0102330 2024-04-25T19:17:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01119c0 con 0x7f92b01079c0 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92b5a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0111cc0 con 0x7f92b0101970 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92aea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 0x7f92b0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92aea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 0x7f92b0199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47640/0 (socket says 172.21.15.86:47640) 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0102330 0x7f92b0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.724+0000 7f92aea00700 1 -- 172.21.15.86:0/1623754836 learned_addr learned my addr 172.21.15.86:0/1623754836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92af400700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92b01079c0 0x7f92b019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92aea00700 1 -- 172.21.15.86:0/1623754836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92b01079c0 msgr2=0x7f92b019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92aea00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92b01079c0 0x7f92b019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92aea00700 1 -- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0102330 msgr2=0x7f92b0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92aea00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0102330 0x7f92b0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92aea00700 1 -- 172.21.15.86:0/1623754836 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b019e1c0 con 0x7f92b0101970 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.725+0000 7f92aea00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 0x7f92b0199200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e9a0 tx=0x7f92a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.726+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a000ca10 con 0x7f92b0101970 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.726+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b019e4b0 con 0x7f92b0101970 2024-04-25T19:17:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.726+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a000cbb0 con 0x7f92b0101970 2024-04-25T19:17:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.726+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0018680 con 0x7f92b0101970 2024-04-25T19:17:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.726+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01a5d30 con 0x7f92b0101970 2024-04-25T19:17:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.727+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b0103090 con 0x7f92b0101970 2024-04-25T19:17:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.729+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92a0010430 con 0x7f92b0101970 2024-04-25T19:17:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.730+0000 7f92a7400700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f92940a3db0 0x7f92940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.731+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f92a00d0ab0 con 0x7f92b0101970 2024-04-25T19:17:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.731+0000 7f92a7e00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f92940a3db0 0x7f92940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.731+0000 7f92a7e00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f92940a3db0 0x7f92940a6270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92b00791a0 tx=0x7f9298005fb0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:26.733+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92a009d080 con 0x7f92b0101970 2024-04-25T19:17:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:26 smithi086 ceph-mon[27783]: pgmap v455: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:27.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.008+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b002df80 con 0x7f92b0101970 2024-04-25T19:17:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.009+0000 7f92a7400700 1 -- 172.21.15.86:0/1623754836 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f92a009c980 con 0x7f92b0101970 2024-04-25T19:17:27.011 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:17:27.011 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:17:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f92940a3db0 msgr2=0x7f92940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f92940a3db0 0x7f92940a6270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92b00791a0 tx=0x7f9298005fb0 comp rx=0 tx=0).stop 2024-04-25T19:17:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 msgr2=0x7f92b0199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 0x7f92b0199200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e9a0 tx=0x7f92a000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 shutdown_connections 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92b01079c0 0x7f92b019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f92940a3db0 0x7f92940a6270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92b0101970 0x7f92b0199200 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 --2- 172.21.15.86:0/1623754836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b0102330 0x7f92b0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 >> 172.21.15.86:0/1623754836 conn(0x7f92b0078980 msgr2=0x7f92b006b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 shutdown_connections 2024-04-25T19:17:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:27.012+0000 7f92b5a00700 1 -- 172.21.15.86:0/1623754836 wait complete. 2024-04-25T19:17:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:17:27.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:26 smithi183 ceph-mon[32344]: pgmap v455: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:27.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:26 smithi194 ceph-mon[32740]: pgmap v455: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:27.930 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:27 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/1623754836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:17:27.989 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 84.7 --await-for 42.0' 2024-04-25T19:17:27.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 84.7 --await-for 42.0 2024-04-25T19:17:28.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:27 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/1623754836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:17:28.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:27 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/1623754836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:17:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:17:28.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:28 smithi086 ceph-mon[27783]: pgmap v456: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:29.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:28 smithi183 ceph-mon[32344]: pgmap v456: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:29.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:28 smithi194 ceph-mon[32740]: pgmap v456: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:17:30.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 -- 172.21.15.86:0/1484354921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5341092f0 msgr2=0x7fc5341096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 --2- 172.21.15.86:0/1484354921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5341092f0 0x7fc5341096d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc52800b0d0 tx=0x7fc52800b3e0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 -- 172.21.15.86:0/1484354921 shutdown_connections 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 --2- 172.21.15.86:0/1484354921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53410e3d0 0x7fc534115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 --2- 172.21.15.86:0/1484354921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5341092f0 0x7fc5341096d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 --2- 172.21.15.86:0/1484354921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc534109ca0 0x7fc53410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.509+0000 7fc53c200700 1 -- 172.21.15.86:0/1484354921 >> 172.21.15.86:0/1484354921 conn(0x7fc534076250 msgr2=0x7fc534107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.510+0000 7fc53c200700 1 -- 172.21.15.86:0/1484354921 shutdown_connections 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.510+0000 7fc53c200700 1 -- 172.21.15.86:0/1484354921 wait complete. 2024-04-25T19:17:30.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.511+0000 7fc53c200700 1 Processor -- start 2024-04-25T19:17:30.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.511+0000 7fc53c200700 1 -- start start 2024-04-25T19:17:30.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53c200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 0x7fc53419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc534109ca0 0x7fc53419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53c200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53410e3d0 0x7fc5341a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc534119260 con 0x7fc534109ca0 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53c200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341190e0 con 0x7fc53410e3d0 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53c200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341193e0 con 0x7fc5341092f0 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc539800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 0x7fc53419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc538e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc534109ca0 0x7fc53419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc539800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 0x7fc53419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51582/0 (socket says 172.21.15.86:51582) 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc539800700 1 -- 172.21.15.86:0/2195435993 learned_addr learned my addr 172.21.15.86:0/2195435993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:30.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.512+0000 7fc53a200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53410e3d0 0x7fc5341a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc539800700 1 -- 172.21.15.86:0/2195435993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53410e3d0 msgr2=0x7fc5341a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc539800700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53410e3d0 0x7fc5341a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc539800700 1 -- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc534109ca0 msgr2=0x7fc53419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc539800700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc534109ca0 0x7fc53419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc539800700 1 -- 172.21.15.86:0/2195435993 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5341a46c0 con 0x7fc5341092f0 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc538e00700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc534109ca0 0x7fc53419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc539800700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 0x7fc53419f6b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc528007a30 tx=0x7fc528007a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc532000700 1 -- 172.21.15.86:0/2195435993 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52801f070 con 0x7fc5341092f0 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5341a4950 con 0x7fc5341092f0 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc532000700 1 -- 172.21.15.86:0/2195435993 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc528004c50 con 0x7fc5341092f0 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.513+0000 7fc532000700 1 -- 172.21.15.86:0/2195435993 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52801b690 con 0x7fc5341092f0 2024-04-25T19:17:30.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.514+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5341ac230 con 0x7fc5341092f0 2024-04-25T19:17:30.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.517+0000 7fc532000700 1 -- 172.21.15.86:0/2195435993 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc528022070 con 0x7fc5341092f0 2024-04-25T19:17:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.518+0000 7fc546a8eb80 1 Processor -- start 2024-04-25T19:17:30.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.518+0000 7fc546a8eb80 1 -- start start 2024-04-25T19:17:30.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31000d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc532000700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc51c0a3ed0 0x7fc51c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af3100bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af3100e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af30f54020 con 0x55af310012d0 2024-04-25T19:17:30.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af30efa8c0 con 0x55af310009b0 2024-04-25T19:17:30.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55af30f4ffe0 con 0x55af3100c180 2024-04-25T19:17:30.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc532000700 1 -- 172.21.15.86:0/2195435993 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7fc52800f020 con 0x7fc5341092f0 2024-04-25T19:17:30.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc539800700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc51c0a3ed0 0x7fc51c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc53a200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31000d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc53a200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31000d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40242/0 (socket says 172.21.15.86:40242) 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc53a200700 1 -- 172.21.15.86:0/1701937921 learned_addr learned my addr 172.21.15.86:0/1701937921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc538e00700 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af3100bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.519+0000 7fc53a200700 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af3100e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc53a200700 1 -- 172.21.15.86:0/1701937921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 msgr2=0x55af3100e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc53a200700 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af3100e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc53a200700 1 -- 172.21.15.86:0/1701937921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 msgr2=0x55af3100bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc53a200700 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af3100bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc53a200700 1 -- 172.21.15.86:0/1701937921 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af30ef0810 con 0x55af310009b0 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc539800700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc51c0a3ed0 0x7fc51c0a6390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5280ceee0 tx=0x7fc5280099e0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:30.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc53a200700 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31000d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0046e0 tx=0x7fc52c009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fab2c0a1f0637cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.520+0000 7fc526a00700 1 -- 172.21.15.86:0/1701937921 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c010420 con 0x55af310009b0 2024-04-25T19:17:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc526a00700 1 -- 172.21.15.86:0/1701937921 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c010a70 con 0x55af310009b0 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc526a00700 1 -- 172.21.15.86:0/1701937921 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00f600 con 0x55af310009b0 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 -- 172.21.15.86:0/1701937921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 msgr2=0x55af31000d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31000d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0046e0 tx=0x7fc52c009850 comp rx=0 tx=0).stop 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 -- 172.21.15.86:0/1701937921 shutdown_connections 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af3100e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31000d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/1701937921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af3100bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.521+0000 7fc546a8eb80 1 -- 172.21.15.86:0/1701937921 >> 172.21.15.86:0/1701937921 conn(0x55af30f2f730 msgr2=0x55af30ffce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.522+0000 7fc546a8eb80 1 -- 172.21.15.86:0/1701937921 shutdown_connections 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.522+0000 7fc546a8eb80 1 -- 172.21.15.86:0/1701937921 wait complete. 2024-04-25T19:17:30.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.522+0000 7fc546a8eb80 1 -- start start 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.522+0000 7fc546a8eb80 1 Processor -- start 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af3100b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af31008be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af31009120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af3100eca0 con 0x55af310012d0 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af30f0d9f0 con 0x55af310009b0 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55af3100ee20 con 0x55af3100c180 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc546a8eb80 10 client.0 _ll_register_callbacks cb 0x55af30eec8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc538e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af31008be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc538e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af31008be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52462/0 (socket says 172.21.15.86:52462) 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc538e00700 1 -- 172.21.15.86:0/3477888595 learned_addr learned my addr 172.21.15.86:0/3477888595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.523+0000 7fc53a200700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af3100b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc53a200700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af31009120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc538e00700 1 -- 172.21.15.86:0/3477888595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 msgr2=0x55af31009120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc538e00700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af31009120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc538e00700 1 -- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 msgr2=0x55af3100b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc538e00700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af3100b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc538e00700 1 -- 172.21.15.86:0/3477888595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af30ea1a60 con 0x55af310012d0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc53a200700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af3100b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc538e00700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af31008be0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc520012d20 tx=0x7fc52000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22b9ddf32a1679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc51b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc51b400700 1 -- 172.21.15.86:0/3477888595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000fc30 con 0x55af310012d0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc51b400700 1 -- 172.21.15.86:0/3477888595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7fc520012e00 con 0x55af310012d0 2024-04-25T19:17:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.524+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc52c010a70 con 0x55af310012d0 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.525+0000 7fc51b400700 1 -- 172.21.15.86:0/3477888595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc52000fdd0 con 0x55af310012d0 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.525+0000 7fc546a8eb80 10 client.16878 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.525+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af30eeac80 con 0x55af310012d0 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.525+0000 7fc51b400700 1 -- 172.21.15.86:0/3477888595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c010a70 con 0x55af310012d0 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.525+0000 7fc51b400700 1 -- 172.21.15.86:0/3477888595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc520003c50 con 0x55af310012d0 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc546a8eb80 10 client.16878 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc546a8eb80 10 client.16878 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc546a8eb80 20 client.16878 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af3101ba00 0x55af3101ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc546a8eb80 4 client.16878 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:17:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af30f61810 con 0x55af3101ba00 2024-04-25T19:17:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.526+0000 7fc53a200700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af3101ba00 0x55af3101ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.527+0000 7fc53a200700 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af3101ba00 0x55af3101ddf0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e673368d9276e8 server_cookie=d13bcde995ab691b in_seq=0 out_seq=0 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.527+0000 7fc51b400700 10 client.16878 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.527+0000 7fc51b400700 1 -- 172.21.15.86:0/3477888595 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af30f61810 con 0x55af3101ba00 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.527+0000 7fc51b400700 10 client.16878 handle_command_reply: tid=0 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.527+0000 7fc546a8eb80 1 client.16878 shutdown 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 msgr2=0x55af31008be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af31008be0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc520012d20 tx=0x7fc52000f6a0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 shutdown_connections 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af3101ba00 0x55af3101ddf0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af3100c180 0x55af31009120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af3100b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3477888595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af310012d0 0x55af31008be0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc51b400700 0 client.16878 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 >> 172.21.15.86:0/3477888595 conn(0x55af30f2f730 msgr2=0x55af30ffca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 shutdown_connections 2024-04-25T19:17:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3477888595 wait complete. 2024-04-25T19:17:30.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.528+0000 7fc546a8eb80 20 client.16878 trim_cache size 0 max 16384 2024-04-25T19:17:30.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.549+0000 7fc546a8eb80 1 Processor -- start 2024-04-25T19:17:30.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.549+0000 7fc546a8eb80 1 -- start start 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af30ffb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31017ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af3100ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341b3a50 con 0x55af3100c180 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341b5590 con 0x55af310009b0 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341b4120 con 0x55af310012d0 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc53a200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af30ffb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc538e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31017ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc538e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31017ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51608/0 (socket says 172.21.15.86:51608) 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc538e00700 1 -- 172.21.15.86:0/2253411710 learned_addr learned my addr 172.21.15.86:0/2253411710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:30.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.550+0000 7fc53a200700 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af3100ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc538e00700 1 -- 172.21.15.86:0/2253411710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 msgr2=0x55af30ffb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc538e00700 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af30ffb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc538e00700 1 -- 172.21.15.86:0/2253411710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 msgr2=0x55af3100ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc538e00700 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af3100ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc538e00700 1 -- 172.21.15.86:0/2253411710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3101e450 con 0x55af310012d0 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc53a200700 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af30ffb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc538e00700 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31017ba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc52000ffd0 tx=0x7fc520009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b713883dea7468d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc526000700 1 -- 172.21.15.86:0/2253411710 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520028070 con 0x55af310012d0 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc526000700 1 -- 172.21.15.86:0/2253411710 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5200106f0 con 0x55af310012d0 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.551+0000 7fc526000700 1 -- 172.21.15.86:0/2253411710 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520003680 con 0x55af310012d0 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 -- 172.21.15.86:0/2253411710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 msgr2=0x55af31017ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31017ba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc52000ffd0 tx=0x7fc520009c90 comp rx=0 tx=0).stop 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 -- 172.21.15.86:0/2253411710 shutdown_connections 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af3100ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af30ffb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/2253411710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31017ba0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 -- 172.21.15.86:0/2253411710 >> 172.21.15.86:0/2253411710 conn(0x55af30f2f730 msgr2=0x55af3100a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 -- 172.21.15.86:0/2253411710 shutdown_connections 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.552+0000 7fc546a8eb80 1 -- 172.21.15.86:0/2253411710 wait complete. 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 -- start start 2024-04-25T19:17:30.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 Processor -- start 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31014490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31011080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af310115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341b4ec0 con 0x55af3100c180 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341b47f0 con 0x55af310009b0 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341b1a10 con 0x55af310012d0 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.553+0000 7fc546a8eb80 10 client.0 _ll_register_callbacks cb 0x55af30eec8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31011080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc53a200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31014490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31011080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51614/0 (socket says 172.21.15.86:51614) 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 -- 172.21.15.86:0/3435399926 learned_addr learned my addr 172.21.15.86:0/3435399926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc53a200700 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af310115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 -- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 msgr2=0x55af31014490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31014490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 -- 172.21.15.86:0/3435399926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 msgr2=0x55af310115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af310115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 -- 172.21.15.86:0/3435399926 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af310016b0 con 0x55af310012d0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc538e00700 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31011080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc5200105f0 tx=0x7fc520005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4086f965bb1b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.554+0000 7fc51b400700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc51b400700 1 -- 172.21.15.86:0/3435399926 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520031060 con 0x55af310012d0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc51b400700 1 -- 172.21.15.86:0/3435399926 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7fc5200108a0 con 0x55af310012d0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5200106f0 con 0x55af310012d0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc51b400700 1 -- 172.21.15.86:0/3435399926 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc520028040 con 0x55af310012d0 2024-04-25T19:17:30.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc546a8eb80 10 client.26977 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af31082d90 con 0x55af310012d0 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.555+0000 7fc51b400700 1 -- 172.21.15.86:0/3435399926 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5200106f0 con 0x55af310012d0 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc51b400700 1 -- 172.21.15.86:0/3435399926 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc520024450 con 0x55af310012d0 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc546a8eb80 10 client.26977 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc546a8eb80 10 client.26977 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc546a8eb80 20 client.26977 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af310833c0 0x55af310857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc546a8eb80 4 client.26977 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.7, "await_for": 42.0}] 2024-04-25T19:17:30.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.7, "await_for": 42.0}) -- 0x55af30f5b8b0 con 0x55af310833c0 2024-04-25T19:17:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.556+0000 7fc53a200700 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af310833c0 0x55af310857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.557+0000 7fc53a200700 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af310833c0 0x55af310857e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2fc1bf3ff413fb server_cookie=cda465a3918b6f30 in_seq=0 out_seq=0 2024-04-25T19:17:30.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.557+0000 7fc51b400700 10 client.26977 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:30.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc51b400700 1 -- 172.21.15.86:0/3435399926 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55af30f5b8b0 con 0x55af310833c0 2024-04-25T19:17:30.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc51b400700 10 client.26977 handle_command_reply: tid=0 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 client.26977 shutdown 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 msgr2=0x55af31011080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31011080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc5200105f0 tx=0x7fc520005410 comp rx=0 tx=0).stop 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 shutdown_connections 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55af310833c0 0x55af310857e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af3100c180 0x55af310115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af310009b0 0x55af31014490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 --2- 172.21.15.86:0/3435399926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55af310012d0 0x55af31011080 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc51b400700 0 client.26977 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 >> 172.21.15.86:0/3435399926 conn(0x55af30f2f730 msgr2=0x55af31003140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 shutdown_connections 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.571+0000 7fc546a8eb80 1 -- 172.21.15.86:0/3435399926 wait complete. 2024-04-25T19:17:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc546a8eb80 20 client.26977 trim_cache size 0 max 16384 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 42, 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "4a5a5612": { 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 42, 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:17:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 84.7, 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:17:30.576 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc51c0a3ed0 msgr2=0x7fc51c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc51c0a3ed0 0x7fc51c0a6390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5280ceee0 tx=0x7fc5280099e0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 msgr2=0x7fc53419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 0x7fc53419f6b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc528007a30 tx=0x7fc528007a60 comp rx=0 tx=0).stop 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 shutdown_connections 2024-04-25T19:17:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53410e3d0 0x7fc5341a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc51c0a3ed0 0x7fc51c0a6390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5341092f0 0x7fc53419f6b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 --2- 172.21.15.86:0/2195435993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc534109ca0 0x7fc53419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.572+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 >> 172.21.15.86:0/2195435993 conn(0x7fc534076250 msgr2=0x7fc5340777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.573+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 shutdown_connections 2024-04-25T19:17:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:30.573+0000 7fc53c200700 1 -- 172.21.15.86:0/2195435993 wait complete. 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 -- 172.21.15.86:0/195597465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 msgr2=0x7f391c074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 --2- 172.21.15.86:0/195597465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c074630 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02a490 comp rx=0 tx=0).stop 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 -- 172.21.15.86:0/195597465 shutdown_connections 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 --2- 172.21.15.86:0/195597465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391c10e840 0x7f391c110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 --2- 172.21.15.86:0/195597465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f391c074b70 0x7f391c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 --2- 172.21.15.86:0/195597465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c074630 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 -- 172.21.15.86:0/195597465 >> 172.21.15.86:0/195597465 conn(0x7f391c0793e0 msgr2=0x7f391c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 -- 172.21.15.86:0/195597465 shutdown_connections 2024-04-25T19:17:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.047+0000 7f3921a00640 1 -- 172.21.15.86:0/195597465 wait complete. 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.048+0000 7f3921a00640 1 Processor -- start 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.048+0000 7f3921a00640 1 -- start start 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.048+0000 7f3921a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.048+0000 7f3921a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391c074b70 0x7f391c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.048+0000 7f3921a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f391c10e840 0x7f391c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f3921a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f391c114450 con 0x7f391c074250 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f3921a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1142d0 con 0x7f391c074b70 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f3921a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1145d0 con 0x7f391c10e840 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c1a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52514/0 (socket says 172.21.15.86:52514) 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 -- 172.21.15.86:0/1683262380 learned_addr learned my addr 172.21.15.86:0/1683262380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391b400640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f391c10e840 0x7f391c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f3913e00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391c074b70 0x7f391c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 -- 172.21.15.86:0/1683262380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f391c10e840 msgr2=0x7f391c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f391c10e840 0x7f391c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 -- 172.21.15.86:0/1683262380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391c074b70 msgr2=0x7f391c1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391c074b70 0x7f391c1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.049+0000 7f391aa00640 1 -- 172.21.15.86:0/1683262380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c19af70 con 0x7f391c074250 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.050+0000 7f391aa00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c1a05a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f390c005ab0 tx=0x7f390c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.050+0000 7f3913400640 1 -- 172.21.15.86:0/1683262380 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c03f070 con 0x7f391c074250 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.050+0000 7f3913400640 1 -- 172.21.15.86:0/1683262380 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c031b60 con 0x7f391c074250 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.050+0000 7f3913400640 1 -- 172.21.15.86:0/1683262380 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c035030 con 0x7f391c074250 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.050+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c19b200 con 0x7f391c074250 2024-04-25T19:17:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.050+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a7520 con 0x7f391c074250 2024-04-25T19:17:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.054+0000 7f3924807740 1 Processor -- start 2024-04-25T19:17:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.054+0000 7f3924807740 1 -- start start 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3913400640 1 -- 172.21.15.86:0/1683262380 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f390c042070 con 0x7f391c074250 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3924807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf79820 0x564b8cf79c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3924807740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf7a140 0x564b8cf844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3924807740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf84a00 0x564b8cf86df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3924807740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564b8cedf900 con 0x564b8cf7a140 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3924807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b8ce85ef0 con 0x564b8cf79820 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3924807740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b8cedb8c0 con 0x564b8cf84a00 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f391aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf7a140 0x564b8cf844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f391b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf79820 0x564b8cf79c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f3913e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf84a00 0x564b8cf86df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f391b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf79820 0x564b8cf79c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40290/0 (socket says 172.21.15.86:40290) 2024-04-25T19:17:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.055+0000 7f391b400640 1 -- 172.21.15.86:0/1015617774 learned_addr learned my addr 172.21.15.86:0/1015617774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913e00640 1 -- 172.21.15.86:0/1015617774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf79820 msgr2=0x564b8cf79c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913e00640 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf79820 0x564b8cf79c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913e00640 1 -- 172.21.15.86:0/1015617774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf7a140 msgr2=0x564b8cf844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913e00640 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf7a140 0x564b8cf844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913e00640 1 -- 172.21.15.86:0/1015617774 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8cbf2210 con 0x564b8cf84a00 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913e00640 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf84a00 0x564b8cf86df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3904009bd0 tx=0x7f3904011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba385518a7d7306f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f390be00640 1 -- 172.21.15.86:0/1015617774 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904004230 con 0x564b8cf84a00 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f390be00640 1 -- 172.21.15.86:0/1015617774 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3904009e40 con 0x564b8cf84a00 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f390be00640 1 -- 172.21.15.86:0/1015617774 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904004c60 con 0x564b8cf84a00 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.056+0000 7f3913400640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39000a3e40 0x7f39000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3913400640 1 -- 172.21.15.86:0/1683262380 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f390c0f0e70 con 0x7f391c074250 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f391b400640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39000a3e40 0x7f39000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 -- 172.21.15.86:0/1015617774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf84a00 msgr2=0x564b8cf86df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf84a00 0x564b8cf86df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3904009bd0 tx=0x7f3904011810 comp rx=0 tx=0).stop 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 -- 172.21.15.86:0/1015617774 shutdown_connections 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf84a00 0x564b8cf86df0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf7a140 0x564b8cf844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 --2- 172.21.15.86:0/1015617774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf79820 0x564b8cf79c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.057+0000 7f3924807740 1 -- 172.21.15.86:0/1015617774 >> 172.21.15.86:0/1015617774 conn(0x564b8cf75d30 msgr2=0x564b8cf76110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.058+0000 7f3924807740 1 -- 172.21.15.86:0/1015617774 shutdown_connections 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.058+0000 7f391b400640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39000a3e40 0x7f39000a6300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3914011630 tx=0x7f3914009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.058+0000 7f3924807740 1 -- 172.21.15.86:0/1015617774 wait complete. 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.058+0000 7f3924807740 1 -- start start 2024-04-25T19:17:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.058+0000 7f3924807740 1 Processor -- start 2024-04-25T19:17:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf79820 0x564b8cf818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf7a140 0x564b8cf81df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 0x564b8cf82330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564b8cf87630 con 0x564b8cf79820 2024-04-25T19:17:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b8ce99310 con 0x564b8cf84a00 2024-04-25T19:17:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b8cf877b0 con 0x564b8cf7a140 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3924807740 10 client.0 _ll_register_callbacks cb 0x564b8ce67cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf79820 0x564b8cf818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 0x564b8cf82330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f3913e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf7a140 0x564b8cf81df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 0x564b8cf82330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40296/0 (socket says 172.21.15.86:40296) 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 -- 172.21.15.86:0/3949355135 learned_addr learned my addr 172.21.15.86:0/3949355135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf79820 0x564b8cf818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52530/0 (socket says 172.21.15.86:52530) 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 -- 172.21.15.86:0/3949355135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf7a140 msgr2=0x564b8cf81df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf7a140 0x564b8cf81df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 -- 172.21.15.86:0/3949355135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf79820 msgr2=0x564b8cf818b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf79820 0x564b8cf818b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.059+0000 7f391b400640 1 -- 172.21.15.86:0/3949355135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8ccbe620 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f391b400640 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 0x564b8cf82330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3914003f10 tx=0x7f3914004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc813bc51f3027e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3908c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3908c00640 1 -- 172.21.15.86:0/3949355135 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400f830 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3908c00640 1 -- 172.21.15.86:0/3949355135 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f391400f9d0 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b8cef38b0 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3908c00640 1 -- 172.21.15.86:0/3949355135 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f391400fbd0 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3924807740 10 client.26990 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.060+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b8cf93880 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3908c00640 1 -- 172.21.15.86:0/3949355135 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564b8cef38b0 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3908c00640 1 -- 172.21.15.86:0/3949355135 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3914026070 con 0x564b8cf84a00 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3924807740 10 client.26990 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3924807740 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3924807740 20 client.26990 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3924807740 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8cf94d90 0x564b8cf971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3924807740 4 client.26990 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b8ceed320 con 0x564b8cf94d90 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.061+0000 7f391aa00640 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8cf94d90 0x564b8cf971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.062+0000 7f391aa00640 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8cf94d90 0x564b8cf971a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864dc3741d8130d6 server_cookie=96f6c8cf6c1aa0e0 in_seq=0 out_seq=0 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.062+0000 7f3908c00640 10 client.26990 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3908c00640 1 -- 172.21.15.86:0/3949355135 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b8ceed320 con 0x564b8cf94d90 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3908c00640 10 client.26990 handle_command_reply: tid=0 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 client.26990 shutdown 2024-04-25T19:17:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 msgr2=0x564b8cf82330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 0x564b8cf82330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3914003f10 tx=0x7f3914004a40 comp rx=0 tx=0).stop 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 shutdown_connections 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8cf94d90 0x564b8cf971a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf84a00 0x564b8cf82330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf7a140 0x564b8cf81df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 --2- 172.21.15.86:0/3949355135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf79820 0x564b8cf818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3908c00640 0 client.26990 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 >> 172.21.15.86:0/3949355135 conn(0x564b8cf75d30 msgr2=0x564b8cf7bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 shutdown_connections 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 1 -- 172.21.15.86:0/3949355135 wait complete. 2024-04-25T19:17:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.063+0000 7f3924807740 20 client.26990 trim_cache size 0 max 16384 2024-04-25T19:17:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:30 smithi086 ceph-mon[27783]: pgmap v457: 97 pgs: 97 active+clean; 4.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.079+0000 7f3924807740 1 Processor -- start 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 -- start start 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf90520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf7c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8cf9cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1afec0 con 0x564b8cf84a00 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b1a00 con 0x564b8cf7a140 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3924807740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b0590 con 0x564b8cf79820 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f391b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8cf9cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f391aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f3913e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf7c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f391aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf90520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51638/0 (socket says 172.21.15.86:51638) 2024-04-25T19:17:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.080+0000 7f391aa00640 1 -- 172.21.15.86:0/4246580155 learned_addr learned my addr 172.21.15.86:0/4246580155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f3913e00640 1 -- 172.21.15.86:0/4246580155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 msgr2=0x564b8cf90520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f3913e00640 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf90520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f3913e00640 1 -- 172.21.15.86:0/4246580155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 msgr2=0x564b8cf9cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f3913e00640 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8cf9cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f3913e00640 1 -- 172.21.15.86:0/4246580155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8cf76f60 con 0x564b8cf7a140 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f391aa00640 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf90520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f3913e00640 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf7c230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f39040096e0 tx=0x7f39040046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9511a43e1e27985f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f390b400640 1 -- 172.21.15.86:0/4246580155 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904027020 con 0x564b8cf7a140 2024-04-25T19:17:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f390b400640 1 -- 172.21.15.86:0/4246580155 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3904009800 con 0x564b8cf7a140 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.081+0000 7f390b400640 1 -- 172.21.15.86:0/4246580155 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390401e040 con 0x564b8cf7a140 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 -- 172.21.15.86:0/4246580155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 msgr2=0x564b8cf7c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf7c230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f39040096e0 tx=0x7f39040046e0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 -- 172.21.15.86:0/4246580155 shutdown_connections 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8cf9cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf7c230 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 --2- 172.21.15.86:0/4246580155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf90520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 -- 172.21.15.86:0/4246580155 >> 172.21.15.86:0/4246580155 conn(0x564b8cf75d30 msgr2=0x564b8cf80a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 -- 172.21.15.86:0/4246580155 shutdown_connections 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 -- 172.21.15.86:0/4246580155 wait complete. 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.082+0000 7f3924807740 1 -- start start 2024-04-25T19:17:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 Processor -- start 2024-04-25T19:17:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf88120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8d001b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b1330 con 0x564b8cf84a00 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b0c60 con 0x564b8cf7a140 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1adec0 con 0x564b8cf79820 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3924807740 10 client.0 _ll_register_callbacks cb 0x7f3914003dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f391b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8d001b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f391aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.083+0000 7f3913e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf88120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8d001b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52548/0 (socket says 172.21.15.86:52548) 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 -- 172.21.15.86:0/479721769 learned_addr learned my addr 172.21.15.86:0/479721769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 -- 172.21.15.86:0/479721769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 msgr2=0x564b8cf87be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf87be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 -- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 msgr2=0x564b8cf88120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf88120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 -- 172.21.15.86:0/479721769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8d002060 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f391b400640 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8d001b20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3914004020 tx=0x7f391400ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e42b520a20cc0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f3908c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.084+0000 7f3908c00640 1 -- 172.21.15.86:0/479721769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391401e6e0 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3908c00640 1 -- 172.21.15.86:0/479721769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f391401e880 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b8cee7190 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3908c00640 1 -- 172.21.15.86:0/479721769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3914040050 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 10 client.16902 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b8d003350 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3908c00640 1 -- 172.21.15.86:0/479721769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564b8cee7190 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3908c00640 1 -- 172.21.15.86:0/479721769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3914037070 con 0x564b8cf84a00 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 10 client.16902 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 20 client.16902 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8d003ad0 0x564b8d005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 4 client.16902 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:17:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.085+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b8ce634b0 con 0x564b8d003ad0 2024-04-25T19:17:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.087+0000 7f391aa00640 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8d003ad0 0x564b8d005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.091+0000 7f391aa00640 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8d003ad0 0x564b8d005ef0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89817d3d0286c339 server_cookie=e704b621b0938949 in_seq=0 out_seq=0 2024-04-25T19:17:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.091+0000 7f3908c00640 10 client.16902 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.111+0000 7f3908c00640 1 -- 172.21.15.86:0/479721769 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b8ce634b0 con 0x564b8d003ad0 2024-04-25T19:17:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.111+0000 7f3908c00640 10 client.16902 handle_command_reply: tid=0 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.111+0000 7f3924807740 1 client.16902 shutdown 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 msgr2=0x564b8d001b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8d001b20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3914004020 tx=0x7f391400ff80 comp rx=0 tx=0).stop 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 shutdown_connections 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x564b8d003ad0 0x564b8d005ef0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b8cf84a00 0x564b8d001b20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b8cf7a140 0x564b8cf88120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3908c00640 0 client.16902 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 --2- 172.21.15.86:0/479721769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b8cf79820 0x564b8cf87be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 >> 172.21.15.86:0/479721769 conn(0x564b8cf75d30 msgr2=0x564b8cf74600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 shutdown_connections 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 1 -- 172.21.15.86:0/479721769 wait complete. 2024-04-25T19:17:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.112+0000 7f3924807740 20 client.16902 trim_cache size 0 max 16384 2024-04-25T19:17:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:17:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:17:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:17:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:17:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39000a3e40 msgr2=0x7f39000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39000a3e40 0x7f39000a6300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3914011630 tx=0x7f3914009500 comp rx=0 tx=0).stop 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 msgr2=0x7f391c1a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c1a05a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f390c005ab0 tx=0x7f390c003c10 comp rx=0 tx=0).stop 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 shutdown_connections 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f39000a3e40 0x7f39000a6300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f391c10e840 0x7f391c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391c074b70 0x7f391c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 --2- 172.21.15.86:0/1683262380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391c074250 0x7f391c1a05a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 >> 172.21.15.86:0/1683262380 conn(0x7f391c0793e0 msgr2=0x7f391c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 shutdown_connections 2024-04-25T19:17:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:31.113+0000 7f3921a00640 1 -- 172.21.15.86:0/1683262380 wait complete. 2024-04-25T19:17:31.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:30 smithi183 ceph-mon[32344]: pgmap v457: 97 pgs: 97 active+clean; 4.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:17:31.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:30 smithi194 ceph-mon[32740]: pgmap v457: 97 pgs: 97 active+clean; 4.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:17:31.523 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4a5a5612', quiesce took 7.7 seconds. Will release after: 24.7 2024-04-25T19:17:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:32 smithi086 ceph-mon[27783]: pgmap v458: 97 pgs: 97 active+clean; 4.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:17:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:32 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:17:33.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:32 smithi183 ceph-mon[32344]: pgmap v458: 97 pgs: 97 active+clean; 4.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:17:33.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:32 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:17:33.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:32 smithi194 ceph-mon[32740]: pgmap v458: 97 pgs: 97 active+clean; 4.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:17:33.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:32 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:17:34.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:33 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:17:34.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:33 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:17:34.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:33 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:17:34.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:33 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:17:34.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:33 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:17:34.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:33 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:17:34.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:33 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:17:34.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:33 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:17:34.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:33 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:17:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:34 smithi086 ceph-mon[27783]: pgmap v459: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-25T19:17:35.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:34 smithi183 ceph-mon[32344]: pgmap v459: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-25T19:17:35.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:34 smithi194 ceph-mon[32740]: pgmap v459: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-25T19:17:37.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:36 smithi086 ceph-mon[27783]: pgmap v460: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-25T19:17:37.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:36 smithi183 ceph-mon[32344]: pgmap v460: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-25T19:17:37.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:36 smithi194 ceph-mon[32740]: pgmap v460: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-25T19:17:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:38 smithi086 ceph-mon[27783]: pgmap v461: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-25T19:17:39.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:38 smithi183 ceph-mon[32344]: pgmap v461: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-25T19:17:39.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:38 smithi194 ceph-mon[32740]: pgmap v461: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-25T19:17:41.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:40 smithi183 ceph-mon[32344]: pgmap v462: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T19:17:41.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:40 smithi194 ceph-mon[32740]: pgmap v462: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T19:17:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:40 smithi086 ceph-mon[27783]: pgmap v462: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T19:17:43.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:42 smithi183 ceph-mon[32344]: pgmap v463: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-25T19:17:43.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:42 smithi194 ceph-mon[32740]: pgmap v463: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-25T19:17:43.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:42 smithi086 ceph-mon[27783]: pgmap v463: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-25T19:17:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 -- 172.21.15.86:0/2930496733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4814104d60 msgr2=0x7f4814108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 --2- 172.21.15.86:0/2930496733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4814104d60 0x7f4814108da0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009900 tx=0x7f47fc02a410 comp rx=0 tx=0).stop 2024-04-25T19:17:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 -- 172.21.15.86:0/2930496733 shutdown_connections 2024-04-25T19:17:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 --2- 172.21.15.86:0/2930496733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814109480 0x7f4814111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 --2- 172.21.15.86:0/2930496733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4814104d60 0x7f4814108da0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 --2- 172.21.15.86:0/2930496733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48141043b0 0x7f4814104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 -- 172.21.15.86:0/2930496733 >> 172.21.15.86:0/2930496733 conn(0x7f48140ffc50 msgr2=0x7f4814102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.256+0000 7f4818e00640 1 -- 172.21.15.86:0/2930496733 shutdown_connections 2024-04-25T19:17:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.257+0000 7f4818e00640 1 -- 172.21.15.86:0/2930496733 wait complete. 2024-04-25T19:17:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.257+0000 7f4818e00640 1 Processor -- start 2024-04-25T19:17:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 -- start start 2024-04-25T19:17:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 0x7f48141a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814109480 0x7f48141a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f481419a720 0x7f481419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48141143c0 con 0x7f48141043b0 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4814114240 con 0x7f481419a720 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4818e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4814114540 con 0x7f4814109480 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4811e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 0x7f48141a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4811e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 0x7f48141a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40280/0 (socket says 172.21.15.86:40280) 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4811e00640 1 -- 172.21.15.86:0/769067371 learned_addr learned my addr 172.21.15.86:0/769067371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4809400640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814109480 0x7f48141a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.258+0000 7f4812800640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f481419a720 0x7f481419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f4811e00640 1 -- 172.21.15.86:0/769067371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814109480 msgr2=0x7f48141a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f4811e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814109480 0x7f48141a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f4811e00640 1 -- 172.21.15.86:0/769067371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f481419a720 msgr2=0x7f481419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f4811e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f481419a720 0x7f481419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f4811e00640 1 -- 172.21.15.86:0/769067371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481419b320 con 0x7f48141043b0 2024-04-25T19:17:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f4811e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 0x7f48141a0660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f480400e9f0 tx=0x7f480400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f480b400640 1 -- 172.21.15.86:0/769067371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400cd80 con 0x7f48141043b0 2024-04-25T19:17:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f480b400640 1 -- 172.21.15.86:0/769067371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48040108e0 con 0x7f48141043b0 2024-04-25T19:17:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.259+0000 7f480b400640 1 -- 172.21.15.86:0/769067371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804018980 con 0x7f48141043b0 2024-04-25T19:17:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.260+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48141a7400 con 0x7f48141043b0 2024-04-25T19:17:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.260+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141a77b0 con 0x7f48141043b0 2024-04-25T19:17:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 Processor -- start 2024-04-25T19:17:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f480b400640 1 -- 172.21.15.86:0/769067371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4804010430 con 0x7f48141043b0 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 -- start start 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d37820 0x5580f6d37c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d38140 0x5580f6d424c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d42a00 0x5580f6d44df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580f6c9d900 con 0x5580f6d38140 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580f6c43ef0 con 0x5580f6d42a00 2024-04-25T19:17:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f481bbf5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5580f6c998c0 con 0x5580f6d37820 2024-04-25T19:17:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f4811e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d38140 0x5580f6d424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.263+0000 7f4812800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d37820 0x5580f6d37c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4812800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d37820 0x5580f6d37c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:54566/0 (socket says 172.21.15.86:54566) 2024-04-25T19:17:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4812800640 1 -- 172.21.15.86:0/3017691773 learned_addr learned my addr 172.21.15.86:0/3017691773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d42a00 0x5580f6d44df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 -- 172.21.15.86:0/3017691773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d37820 msgr2=0x5580f6d37c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d37820 0x5580f6d37c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 -- 172.21.15.86:0/3017691773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d38140 msgr2=0x5580f6d424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d38140 0x5580f6d424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 -- 172.21.15.86:0/3017691773 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580f69b0210 con 0x5580f6d42a00 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f4809400640 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d42a00 0x5580f6d44df0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006010 tx=0x7f47fc005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45264116905ea188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.264+0000 7f480b400640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47f80a3e40 0x7f47f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f4803400640 1 -- 172.21.15.86:0/3017691773 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc040070 con 0x5580f6d42a00 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f480b400640 1 -- 172.21.15.86:0/769067371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f48040d0b60 con 0x7f48141043b0 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f4803400640 1 -- 172.21.15.86:0/3017691773 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc004030 con 0x5580f6d42a00 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f4803400640 1 -- 172.21.15.86:0/3017691773 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc035040 con 0x5580f6d42a00 2024-04-25T19:17:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f4812800640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47f80a3e40 0x7f47f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f481bbf5740 1 -- 172.21.15.86:0/3017691773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d42a00 msgr2=0x5580f6d44df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f481bbf5740 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d42a00 0x5580f6d44df0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006010 tx=0x7f47fc005e40 comp rx=0 tx=0).stop 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.265+0000 7f4812800640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47f80a3e40 0x7f47f80a6300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f480c011630 tx=0x7f480c009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 -- 172.21.15.86:0/3017691773 shutdown_connections 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d42a00 0x5580f6d44df0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d38140 0x5580f6d424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 --2- 172.21.15.86:0/3017691773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d37820 0x5580f6d37c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 -- 172.21.15.86:0/3017691773 >> 172.21.15.86:0/3017691773 conn(0x5580f6d33d30 msgr2=0x5580f6d34110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 -- 172.21.15.86:0/3017691773 shutdown_connections 2024-04-25T19:17:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 -- 172.21.15.86:0/3017691773 wait complete. 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 -- start start 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.266+0000 7f481bbf5740 1 Processor -- start 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 0x5580f6d3f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 0x5580f6d3ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d42a00 0x5580f6d40460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580f6d455d0 con 0x5580f6d37820 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580f6c57310 con 0x5580f6d38140 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5580f6d45750 con 0x5580f6d42a00 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f481bbf5740 10 client.0 _ll_register_callbacks cb 0x5580f6c25cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4811e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 0x5580f6d3f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 0x5580f6d3ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4811e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 0x5580f6d3f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40296/0 (socket says 172.21.15.86:40296) 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 0x5580f6d3ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37466/0 (socket says 172.21.15.86:37466) 2024-04-25T19:17:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4811e00640 1 -- 172.21.15.86:0/2338365397 learned_addr learned my addr 172.21.15.86:0/2338365397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4812800640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d42a00 0x5580f6d40460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 -- 172.21.15.86:0/2338365397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d42a00 msgr2=0x5580f6d40460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d42a00 0x5580f6d40460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 -- 172.21.15.86:0/2338365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 msgr2=0x5580f6d3f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 0x5580f6d3f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4809400640 1 -- 172.21.15.86:0/2338365397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580f6a7c620 con 0x5580f6d38140 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.267+0000 7f4811e00640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 0x5580f6d3f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.268+0000 7f4809400640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 0x5580f6d3ff20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5580f6d35150 tx=0x7f47fc032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91a625b41f8b022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.268+0000 7f47f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.268+0000 7f47f7e00640 1 -- 172.21.15.86:0/2338365397 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc040070 con 0x5580f6d38140 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.268+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc004030 con 0x5580f6d38140 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.268+0000 7f47f7e00640 1 -- 172.21.15.86:0/2338365397 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f47fc0316a0 con 0x5580f6d38140 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f47f7e00640 1 -- 172.21.15.86:0/2338365397 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47fc056050 con 0x5580f6d38140 2024-04-25T19:17:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 10 client.27008 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580f6d51880 con 0x5580f6d38140 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f47f7e00640 1 -- 172.21.15.86:0/2338365397 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc004030 con 0x5580f6d38140 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f47f7e00640 1 -- 172.21.15.86:0/2338365397 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f47fc004bd0 con 0x5580f6d38140 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 10 client.27008 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 10 client.27008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 20 client.27008 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6d52bd0 0x5580f6d54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 4 client.27008 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.269+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580f6cb18b0 con 0x5580f6d52bd0 2024-04-25T19:17:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.270+0000 7f4811e00640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6d52bd0 0x5580f6d54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.270+0000 7f47f7e00640 10 client.27008 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.270+0000 7f4811e00640 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6d52bd0 0x5580f6d54fc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3447241b86bc49 server_cookie=81d0a45bca6ca5a6 in_seq=0 out_seq=0 2024-04-25T19:17:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f47f7e00640 1 -- 172.21.15.86:0/2338365397 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580f6cb18b0 con 0x5580f6d52bd0 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f47f7e00640 10 client.27008 handle_command_reply: tid=0 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 client.27008 shutdown 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 msgr2=0x5580f6d3ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 0x5580f6d3ff20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5580f6d35150 tx=0x7f47fc032ed0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 shutdown_connections 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6d52bd0 0x5580f6d54fc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d42a00 0x5580f6d40460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d38140 0x5580f6d3ff20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f47f7e00640 0 client.27008 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 --2- 172.21.15.86:0/2338365397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d37820 0x5580f6d3f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 >> 172.21.15.86:0/2338365397 conn(0x5580f6d33d30 msgr2=0x5580f6d39ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.271+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 shutdown_connections 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.272+0000 7f481bbf5740 1 -- 172.21.15.86:0/2338365397 wait complete. 2024-04-25T19:17:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.272+0000 7f481bbf5740 20 client.27008 trim_cache size 0 max 16384 2024-04-25T19:17:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 Processor -- start 2024-04-25T19:17:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 -- start start 2024-04-25T19:17:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6d330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6d3a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d3df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48141aff70 con 0x5580f6d42a00 2024-04-25T19:17:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48141b1ab0 con 0x5580f6d37820 2024-04-25T19:17:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f481bbf5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f48141b0640 con 0x5580f6d38140 2024-04-25T19:17:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.288+0000 7f4811e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6d330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6d330d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37472/0 (socket says 172.21.15.86:37472) 2024-04-25T19:17:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 -- 172.21.15.86:0/1977112029 learned_addr learned my addr 172.21.15.86:0/1977112029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4812800640 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d3df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4809400640 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6d3a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 -- 172.21.15.86:0/1977112029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 msgr2=0x5580f6d3a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6d3a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 -- 172.21.15.86:0/1977112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 msgr2=0x5580f6d3df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d3df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 -- 172.21.15.86:0/1977112029 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580f6d45680 con 0x5580f6d37820 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4811e00640 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6d330d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f480400e9c0 tx=0x7f48040d2fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce846afef50acf09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.289+0000 7f4802a00640 1 -- 172.21.15.86:0/1977112029 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480409c440 con 0x5580f6d37820 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f4802a00640 1 -- 172.21.15.86:0/1977112029 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4804010720 con 0x5580f6d37820 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f4802a00640 1 -- 172.21.15.86:0/1977112029 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040bd0b0 con 0x5580f6d37820 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 -- 172.21.15.86:0/1977112029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 msgr2=0x5580f6d330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6d330d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f480400e9c0 tx=0x7f48040d2fb0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 -- 172.21.15.86:0/1977112029 shutdown_connections 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d3df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6d3a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 --2- 172.21.15.86:0/1977112029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6d330d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 -- 172.21.15.86:0/1977112029 >> 172.21.15.86:0/1977112029 conn(0x5580f6d33d30 msgr2=0x5580f6d4e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 -- 172.21.15.86:0/1977112029 shutdown_connections 2024-04-25T19:17:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.290+0000 7f481bbf5740 1 -- 172.21.15.86:0/1977112029 wait complete. 2024-04-25T19:17:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.291+0000 7f481bbf5740 1 -- start start 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.291+0000 7f481bbf5740 1 Processor -- start 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.291+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6db10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.291+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6db1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.291+0000 7f481bbf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.291+0000 7f481bbf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48141b13e0 con 0x5580f6d42a00 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f481bbf5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48141b0d10 con 0x5580f6d37820 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f481bbf5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f48141adf70 con 0x5580f6d38140 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f481bbf5740 10 client.0 _ll_register_callbacks cb 0x7f47fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4809400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6db1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d57230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40312/0 (socket says 172.21.15.86:40312) 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 -- 172.21.15.86:0/283827890 learned_addr learned my addr 172.21.15.86:0/283827890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4809400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6db1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:54588/0 (socket says 172.21.15.86:54588) 2024-04-25T19:17:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4811e00640 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6db10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 -- 172.21.15.86:0/283827890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 msgr2=0x5580f6db1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6db1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 -- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 msgr2=0x5580f6db10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6db10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.292+0000 7f4812800640 1 -- 172.21.15.86:0/283827890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580f6d57920 con 0x5580f6d42a00 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f4812800640 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d57230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f480c003fb0 tx=0x7f480c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa2e011cbbebaa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f47f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f47f7e00640 1 -- 172.21.15.86:0/283827890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c02e060 con 0x5580f6d42a00 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f47f7e00640 1 -- 172.21.15.86:0/283827890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f480c00f9d0 con 0x5580f6d42a00 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580f6cab320 con 0x5580f6d42a00 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f47f7e00640 1 -- 172.21.15.86:0/283827890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f480c00fbd0 con 0x5580f6d42a00 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f481bbf5740 10 client.16920 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580f6db88f0 con 0x5580f6d42a00 2024-04-25T19:17:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.293+0000 7f47f7e00640 1 -- 172.21.15.86:0/283827890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580f6cab320 con 0x5580f6d42a00 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f47f7e00640 1 -- 172.21.15.86:0/283827890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f480c031420 con 0x5580f6d42a00 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f481bbf5740 10 client.16920 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f481bbf5740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f481bbf5740 20 client.16920 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f481bbf5740 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6db9070 0x5580f6dbb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f481bbf5740 4 client.16920 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580f6ca5190 con 0x5580f6db9070 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.294+0000 7f4811e00640 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6db9070 0x5580f6dbb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.295+0000 7f4811e00640 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6db9070 0x5580f6dbb490 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edebd2def316d7a5 server_cookie=ba4aeb60b01d7952 in_seq=0 out_seq=0 2024-04-25T19:17:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.295+0000 7f47f7e00640 10 client.16920 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f47f7e00640 1 -- 172.21.15.86:0/283827890 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580f6ca5190 con 0x5580f6db9070 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f47f7e00640 10 client.16920 handle_command_reply: tid=0 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 client.16920 shutdown 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 msgr2=0x5580f6d57230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d57230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f480c003fb0 tx=0x7f480c004a40 comp rx=0 tx=0).stop 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 shutdown_connections 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5580f6db9070 0x5580f6dbb490 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580f6d42a00 0x5580f6d57230 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580f6d38140 0x5580f6db1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 --2- 172.21.15.86:0/283827890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580f6d37820 0x5580f6db10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f47f7e00640 0 client.16920 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 >> 172.21.15.86:0/283827890 conn(0x5580f6d33d30 msgr2=0x5580f6d4d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 shutdown_connections 2024-04-25T19:17:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 1 -- 172.21.15.86:0/283827890 wait complete. 2024-04-25T19:17:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.308+0000 7f481bbf5740 20 client.16920 trim_cache size 0 max 16384 2024-04-25T19:17:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:17:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:17:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:17:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47f80a3e40 msgr2=0x7f47f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47f80a3e40 0x7f47f80a6300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f480c011630 tx=0x7f480c009500 comp rx=0 tx=0).stop 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 msgr2=0x7f48141a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 0x7f48141a0660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f480400e9f0 tx=0x7f480400eeb0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 shutdown_connections 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47f80a3e40 0x7f47f80a6300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f481419a720 0x7f481419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814109480 0x7f48141a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 --2- 172.21.15.86:0/769067371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48141043b0 0x7f48141a0660 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.309+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 >> 172.21.15.86:0/769067371 conn(0x7f48140ffc50 msgr2=0x7f4814101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.310+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 shutdown_connections 2024-04-25T19:17:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:44.310+0000 7f4818e00640 1 -- 172.21.15.86:0/769067371 wait complete. 2024-04-25T19:17:45.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:44 smithi183 ceph-mon[32344]: pgmap v464: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T19:17:45.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:44 smithi194 ceph-mon[32740]: pgmap v464: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T19:17:45.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:44 smithi086 ceph-mon[27783]: pgmap v464: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T19:17:47.123 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:46 smithi086 ceph-mon[27783]: pgmap v465: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:17:47.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:46 smithi183 ceph-mon[32344]: pgmap v465: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:17:47.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:46 smithi194 ceph-mon[32740]: pgmap v465: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:17:49.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:48 smithi183 ceph-mon[32344]: pgmap v466: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:17:49.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:48 smithi194 ceph-mon[32740]: pgmap v466: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:17:49.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:48 smithi086 ceph-mon[27783]: pgmap v466: 97 pgs: 97 active+clean; 4.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T19:17:51.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:50 smithi183 ceph-mon[32344]: pgmap v467: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-25T19:17:51.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:50 smithi194 ceph-mon[32740]: pgmap v467: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-25T19:17:51.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:50 smithi086 ceph-mon[27783]: pgmap v467: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-25T19:17:53.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:52 smithi183 ceph-mon[32344]: pgmap v468: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T19:17:53.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:52 smithi194 ceph-mon[32740]: pgmap v468: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T19:17:53.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:52 smithi086 ceph-mon[27783]: pgmap v468: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T19:17:55.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:54 smithi183 ceph-mon[32344]: pgmap v469: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-04-25T19:17:55.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:54 smithi194 ceph-mon[32740]: pgmap v469: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-04-25T19:17:55.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:54 smithi086 ceph-mon[27783]: pgmap v469: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-04-25T19:17:56.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:17:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:17:57.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:56 smithi086 ceph-mon[27783]: pgmap v470: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T19:17:57.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:56 smithi183 ceph-mon[32344]: pgmap v470: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T19:17:57.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:56 smithi194 ceph-mon[32740]: pgmap v470: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T19:17:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 -- 172.21.15.86:0/3676378737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108e10 msgr2=0x7f5a74115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 --2- 172.21.15.86:0/3676378737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108e10 0x7f5a74115250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009c10 tx=0x7f5a6c02a6f0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 -- 172.21.15.86:0/3676378737 shutdown_connections 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 --2- 172.21.15.86:0/3676378737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108e10 0x7f5a74115250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 --2- 172.21.15.86:0/3676378737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108490 0x7f5a741088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 --2- 172.21.15.86:0/3676378737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a7410e420 0x7f5a7410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 -- 172.21.15.86:0/3676378737 >> 172.21.15.86:0/3676378737 conn(0x7f5a74077d30 msgr2=0x7f5a74106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 -- 172.21.15.86:0/3676378737 shutdown_connections 2024-04-25T19:17:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.453+0000 7f5a79c00640 1 -- 172.21.15.86:0/3676378737 wait complete. 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 Processor -- start 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 -- start start 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108490 0x7f5a741a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 0x7f5a741a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a7410e420 0x7f5a7419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741185e0 con 0x7f5a74108e10 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74118460 con 0x7f5a7410e420 2024-04-25T19:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.454+0000 7f5a79c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74118760 con 0x7f5a74108490 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 0x7f5a741a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 0x7f5a741a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54238/0 (socket says 172.21.15.86:54238) 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 -- 172.21.15.86:0/4293083921 learned_addr learned my addr 172.21.15.86:0/4293083921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a73e00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108490 0x7f5a741a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a79200640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a7410e420 0x7f5a7419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 -- 172.21.15.86:0/4293083921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108490 msgr2=0x7f5a741a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108490 0x7f5a741a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 -- 172.21.15.86:0/4293083921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a7410e420 msgr2=0x7f5a7419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a7410e420 0x7f5a7419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 -- 172.21.15.86:0/4293083921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7419f0c0 con 0x7f5a74108e10 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.455+0000 7f5a6be00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 0x7f5a741a4cd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9c0 tx=0x7f5a5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.456+0000 7f5a71600640 1 -- 172.21.15.86:0/4293083921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00cde0 con 0x7f5a74108e10 2024-04-25T19:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.456+0000 7f5a71600640 1 -- 172.21.15.86:0/4293083921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a5c0035f0 con 0x7f5a74108e10 2024-04-25T19:17:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.456+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7419f3b0 con 0x7f5a74108e10 2024-04-25T19:17:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.456+0000 7f5a71600640 1 -- 172.21.15.86:0/4293083921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c010690 con 0x7f5a74108e10 2024-04-25T19:17:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.456+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741ab650 con 0x7f5a74108e10 2024-04-25T19:17:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.459+0000 7f5a7e018740 1 Processor -- start 2024-04-25T19:17:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.459+0000 7f5a71600640 1 -- 172.21.15.86:0/4293083921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a5c004000 con 0x7f5a74108e10 2024-04-25T19:17:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.459+0000 7f5a7e018740 1 -- start start 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48b9b0 0x5639bb48bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 0x5639bb496650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb496b90 0x5639bb498f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a7e018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639bb3f1900 con 0x5639bb496b90 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a7e018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639bb397ef0 con 0x5639bb48c2d0 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a7e018740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5639bb3ed8c0 con 0x5639bb48b9b0 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb496b90 0x5639bb498f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 0x5639bb496650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a79200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48b9b0 0x5639bb48bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 0x5639bb496650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40500/0 (socket says 172.21.15.86:40500) 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a73e00640 1 -- 172.21.15.86:0/3320404820 learned_addr learned my addr 172.21.15.86:0/3320404820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a73e00640 1 -- 172.21.15.86:0/3320404820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48b9b0 msgr2=0x5639bb48bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.460+0000 7f5a73e00640 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48b9b0 0x5639bb48bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a73e00640 1 -- 172.21.15.86:0/3320404820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb496b90 msgr2=0x5639bb498f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a73e00640 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb496b90 0x5639bb498f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a73e00640 1 -- 172.21.15.86:0/3320404820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639bb104210 con 0x5639bb48c2d0 2024-04-25T19:17:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a73e00640 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 0x5639bb496650 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400bc30 tx=0x7f5a640107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74177c2b938d4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a69600640 1 -- 172.21.15.86:0/3320404820 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64004030 con 0x5639bb48c2d0 2024-04-25T19:17:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a69600640 1 -- 172.21.15.86:0/3320404820 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a640041d0 con 0x5639bb48c2d0 2024-04-25T19:17:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a71600640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a580a3d10 0x7f5a580a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a69600640 1 -- 172.21.15.86:0/3320404820 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64011680 con 0x5639bb48c2d0 2024-04-25T19:17:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.461+0000 7f5a71600640 1 -- 172.21.15.86:0/4293083921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f5a5c0d0bd0 con 0x7f5a74108e10 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a79200640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a580a3d10 0x7f5a580a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 -- 172.21.15.86:0/3320404820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 msgr2=0x5639bb496650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 0x5639bb496650 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400bc30 tx=0x7f5a640107a0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 -- 172.21.15.86:0/3320404820 shutdown_connections 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb496b90 0x5639bb498f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48c2d0 0x5639bb496650 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 --2- 172.21.15.86:0/3320404820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48b9b0 0x5639bb48bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 -- 172.21.15.86:0/3320404820 >> 172.21.15.86:0/3320404820 conn(0x5639bb487ec0 msgr2=0x5639bb4882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 -- 172.21.15.86:0/3320404820 shutdown_connections 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a7e018740 1 -- 172.21.15.86:0/3320404820 wait complete. 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.462+0000 7f5a79200640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a580a3d10 0x7f5a580a61d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c02ac10 tx=0x7f5a6c005fd0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.463+0000 7f5a7e018740 1 -- start start 2024-04-25T19:17:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.463+0000 7f5a7e018740 1 Processor -- start 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.463+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 0x5639bb493b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.463+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48c2d0 0x5639bb4940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.463+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb496b90 0x5639bb4945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.463+0000 7f5a7e018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639bb4997c0 con 0x5639bb48b9b0 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a7e018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639bb3ab310 con 0x5639bb496b90 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a7e018740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5639bb499940 con 0x5639bb48c2d0 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a7e018740 10 client.0 _ll_register_callbacks cb 0x5639bb379cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 0x5639bb493b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 0x5639bb493b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54266/0 (socket says 172.21.15.86:54266) 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 -- 172.21.15.86:0/1392093738 learned_addr learned my addr 172.21.15.86:0/1392093738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a79200640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb496b90 0x5639bb4945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a6be00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48c2d0 0x5639bb4940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 -- 172.21.15.86:0/1392093738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48c2d0 msgr2=0x5639bb4940b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48c2d0 0x5639bb4940b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 -- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb496b90 msgr2=0x5639bb4945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb496b90 0x5639bb4945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 -- 172.21.15.86:0/1392093738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639bb1d0620 con 0x5639bb48b9b0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a6be00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48c2d0 0x5639bb4940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.464+0000 7f5a73e00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 0x5639bb493b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5a64000c00 tx=0x7f5a640045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83408d8ff91e3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a62000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a62000640 1 -- 172.21.15.86:0/1392093738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64003fd0 con 0x5639bb48b9b0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a62000640 1 -- 172.21.15.86:0/1392093738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f5a640035f0 con 0x5639bb48b9b0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a640041d0 con 0x5639bb48b9b0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a62000640 1 -- 172.21.15.86:0/1392093738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a64003810 con 0x5639bb48b9b0 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a7e018740 10 client.16932 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639bb4a5a70 con 0x5639bb48b9b0 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a62000640 1 -- 172.21.15.86:0/1392093738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a640041d0 con 0x5639bb48b9b0 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.465+0000 7f5a62000640 1 -- 172.21.15.86:0/1392093738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a640046c0 con 0x5639bb48b9b0 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a7e018740 10 client.16932 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a7e018740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a7e018740 20 client.16932 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a7e018740 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb4a6e70 0x5639bb4a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a7e018740 4 client.16932 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639bb4058b0 con 0x5639bb4a6e70 2024-04-25T19:17:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.466+0000 7f5a6be00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb4a6e70 0x5639bb4a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.467+0000 7f5a6be00640 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb4a6e70 0x5639bb4a9260 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102faf56948244c2 server_cookie=319fe7d5abdbfa6a in_seq=0 out_seq=0 2024-04-25T19:17:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.467+0000 7f5a62000640 10 client.16932 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.467+0000 7f5a62000640 1 -- 172.21.15.86:0/1392093738 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639bb4058b0 con 0x5639bb4a6e70 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.467+0000 7f5a62000640 10 client.16932 handle_command_reply: tid=0 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 client.16932 shutdown 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 msgr2=0x5639bb493b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 0x5639bb493b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5a64000c00 tx=0x7f5a640045b0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 shutdown_connections 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb4a6e70 0x5639bb4a9260 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb496b90 0x5639bb4945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb48c2d0 0x5639bb4940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 --2- 172.21.15.86:0/1392093738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48b9b0 0x5639bb493b70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 >> 172.21.15.86:0/1392093738 conn(0x5639bb487ec0 msgr2=0x5639bb48dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a62000640 0 client.16932 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 shutdown_connections 2024-04-25T19:17:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 1 -- 172.21.15.86:0/1392093738 wait complete. 2024-04-25T19:17:57.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.468+0000 7f5a7e018740 20 client.16932 trim_cache size 0 max 16384 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 Processor -- start 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 -- start start 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb4a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb4ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb4aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741b4290 con 0x5639bb48c2d0 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741b5dd0 con 0x5639bb48b9b0 2024-04-25T19:17:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.484+0000 7f5a7e018740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741b4960 con 0x5639bb496b90 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb4ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb4ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54272/0 (socket says 172.21.15.86:54272) 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a6be00640 1 -- 172.21.15.86:0/361535840 learned_addr learned my addr 172.21.15.86:0/361535840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a73e00640 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb4a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a79200640 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb4aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a73e00640 1 -- 172.21.15.86:0/361535840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 msgr2=0x5639bb4aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a73e00640 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb4aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a73e00640 1 -- 172.21.15.86:0/361535840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 msgr2=0x5639bb4ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a73e00640 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb4ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a73e00640 1 -- 172.21.15.86:0/361535840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639bb4920c0 con 0x5639bb48b9b0 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.485+0000 7f5a6be00640 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb4ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a73e00640 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb4a2680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b620 tx=0x7f5a64003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7914f34db89102fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:57.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a68c00640 1 -- 172.21.15.86:0/361535840 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64005850 con 0x5639bb48b9b0 2024-04-25T19:17:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a68c00640 1 -- 172.21.15.86:0/361535840 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a640035f0 con 0x5639bb48b9b0 2024-04-25T19:17:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a68c00640 1 -- 172.21.15.86:0/361535840 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a640039b0 con 0x5639bb48b9b0 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a7e018740 1 -- 172.21.15.86:0/361535840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 msgr2=0x5639bb4a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a7e018740 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb4a2680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b620 tx=0x7f5a64003dd0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a7e018740 1 -- 172.21.15.86:0/361535840 shutdown_connections 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a7e018740 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb4aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a7e018740 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb4ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.486+0000 7f5a7e018740 1 --2- 172.21.15.86:0/361535840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb4a2680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.487+0000 7f5a7e018740 1 -- 172.21.15.86:0/361535840 >> 172.21.15.86:0/361535840 conn(0x5639bb487ec0 msgr2=0x5639bb4882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.487+0000 7f5a7e018740 1 -- 172.21.15.86:0/361535840 shutdown_connections 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.487+0000 7f5a7e018740 1 -- 172.21.15.86:0/361535840 wait complete. 2024-04-25T19:17:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.487+0000 7f5a7e018740 1 -- start start 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.487+0000 7f5a7e018740 1 Processor -- start 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb49eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb49f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb49bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741b5700 con 0x5639bb48c2d0 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741b5030 con 0x5639bb48b9b0 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741b2290 con 0x5639bb496b90 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a7e018740 10 client.0 _ll_register_callbacks cb 0x7f5a6400ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb49eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb49f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb49eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40520/0 (socket says 172.21.15.86:40520) 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 -- 172.21.15.86:0/2244942468 learned_addr learned my addr 172.21.15.86:0/2244942468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a79200640 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb49bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 -- 172.21.15.86:0/2244942468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 msgr2=0x5639bb49bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb49bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 -- 172.21.15.86:0/2244942468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 msgr2=0x5639bb49f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb49f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.488+0000 7f5a73e00640 1 -- 172.21.15.86:0/2244942468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639bb49c2a0 con 0x5639bb48b9b0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a73e00640 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb49eb90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b6d0 tx=0x7f5a6400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38c7ac1a9703bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a62000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a62000640 1 -- 172.21.15.86:0/2244942468 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64003e40 con 0x5639bb48b9b0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a640035f0 con 0x5639bb48b9b0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a62000640 1 -- 172.21.15.86:0/2244942468 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7f5a64003fe0 con 0x5639bb48b9b0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a62000640 1 -- 172.21.15.86:0/2244942468 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a640041e0 con 0x5639bb48b9b0 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a7e018740 10 client.27032 fetch_fsmap learned FSMap version 29 2024-04-25T19:17:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639bb50bc60 con 0x5639bb48b9b0 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.489+0000 7f5a62000640 1 -- 172.21.15.86:0/2244942468 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a640035f0 con 0x5639bb48b9b0 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a62000640 1 -- 172.21.15.86:0/2244942468 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a64035070 con 0x5639bb48b9b0 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a7e018740 10 client.27032 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a7e018740 10 client.27032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a7e018740 20 client.27032 populate_metadata read hostname 'smithi086' 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a7e018740 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb50c210 0x5639bb50e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a7e018740 4 client.27032 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639bb3ff320 con 0x5639bb50c210 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.490+0000 7f5a6be00640 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb50c210 0x5639bb50e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.491+0000 7f5a6be00640 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb50c210 0x5639bb50e600 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23fa70f18e5bb5d server_cookie=d024464dba7960e4 in_seq=0 out_seq=0 2024-04-25T19:17:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.491+0000 7f5a62000640 10 client.27032 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.501+0000 7f5a62000640 1 -- 172.21.15.86:0/2244942468 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639bb3ff320 con 0x5639bb50c210 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.501+0000 7f5a62000640 10 client.27032 handle_command_reply: tid=0 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.501+0000 7f5a7e018740 1 client.27032 shutdown 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 msgr2=0x5639bb49eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb49eb90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b6d0 tx=0x7f5a6400b880 comp rx=0 tx=0).stop 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 shutdown_connections 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5639bb50c210 0x5639bb50e600 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639bb496b90 0x5639bb49bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639bb48c2d0 0x5639bb49f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 --2- 172.21.15.86:0/2244942468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639bb48b9b0 0x5639bb49eb90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 >> 172.21.15.86:0/2244942468 conn(0x5639bb487ec0 msgr2=0x5639bb4a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a62000640 0 client.27032 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 shutdown_connections 2024-04-25T19:17:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 1 -- 172.21.15.86:0/2244942468 wait complete. 2024-04-25T19:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.502+0000 7f5a7e018740 20 client.27032 trim_cache size 0 max 16384 2024-04-25T19:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:17:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a580a3d10 msgr2=0x7f5a580a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a580a3d10 0x7f5a580a61d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c02ac10 tx=0x7f5a6c005fd0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 msgr2=0x7f5a741a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 0x7f5a741a4cd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9c0 tx=0x7f5a5c00ee80 comp rx=0 tx=0).stop 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 shutdown_connections 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a580a3d10 0x7f5a580a61d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a7410e420 0x7f5a7419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74108e10 0x7f5a741a4cd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 --2- 172.21.15.86:0/4293083921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a74108490 0x7f5a741a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.503+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 >> 172.21.15.86:0/4293083921 conn(0x7f5a74077d30 msgr2=0x7f5a74078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.504+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 shutdown_connections 2024-04-25T19:17:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:17:57.504+0000 7f5a79c00640 1 -- 172.21.15.86:0/4293083921 wait complete. 2024-04-25T19:17:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:58 smithi086 ceph-mon[27783]: pgmap v471: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T19:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 -- 172.21.15.86:0/3374826444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 msgr2=0x7ff5301096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 --2- 172.21.15.86:0/3374826444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301096b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff52000b0d0 tx=0x7ff52000b3e0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 -- 172.21.15.86:0/3374826444 shutdown_connections 2024-04-25T19:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 --2- 172.21.15.86:0/3374826444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff53010e3b0 0x7ff530115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 --2- 172.21.15.86:0/3374826444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301096b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 --2- 172.21.15.86:0/3374826444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff530109c80 0x7ff53010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 -- 172.21.15.86:0/3374826444 >> 172.21.15.86:0/3374826444 conn(0x7ff530076210 msgr2=0x7ff530107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 -- 172.21.15.86:0/3374826444 shutdown_connections 2024-04-25T19:17:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.134+0000 7ff537600700 1 -- 172.21.15.86:0/3374826444 wait complete. 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.135+0000 7ff537600700 1 Processor -- start 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 -- start start 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff530109c80 0x7ff5301a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff53010e3b0 0x7ff53019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff530119290 con 0x7ff5301092d0 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff530119110 con 0x7ff530109c80 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff537600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff530119410 con 0x7ff53010e3b0 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff534c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff534c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54292/0 (socket says 172.21.15.86:54292) 2024-04-25T19:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff534c00700 1 -- 172.21.15.86:0/128264089 learned_addr learned my addr 172.21.15.86:0/128264089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.136+0000 7ff527e00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff530109c80 0x7ff5301a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff535600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff53010e3b0 0x7ff53019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff534c00700 1 -- 172.21.15.86:0/128264089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff53010e3b0 msgr2=0x7ff53019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff534c00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff53010e3b0 0x7ff53019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff534c00700 1 -- 172.21.15.86:0/128264089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff530109c80 msgr2=0x7ff5301a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff534c00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff530109c80 0x7ff5301a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff534c00700 1 -- 172.21.15.86:0/128264089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53019fd10 con 0x7ff5301092d0 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff527e00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff530109c80 0x7ff5301a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff535600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff53010e3b0 0x7ff53019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.137+0000 7ff534c00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301a53f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff520007a30 tx=0x7ff520007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.138+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52001f070 con 0x7ff5301092d0 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.138+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff520004c50 con 0x7ff5301092d0 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.138+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52001b690 con 0x7ff5301092d0 2024-04-25T19:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.138+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53019ffa0 con 0x7ff5301092d0 2024-04-25T19:17:59.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.138+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5301ac2c0 con 0x7ff5301092d0 2024-04-25T19:17:59.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.139+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff530069730 con 0x7ff5301092d0 2024-04-25T19:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.142+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff520022070 con 0x7ff5301092d0 2024-04-25T19:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.143+0000 7ff52e000700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff5140a3e30 0x7ff5140a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.143+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7ff52000f020 con 0x7ff5301092d0 2024-04-25T19:17:59.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.143+0000 7ff527e00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff5140a3e30 0x7ff5140a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:59.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.144+0000 7ff527e00700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff5140a3e30 0x7ff5140a62f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009680 tx=0x7ff51c009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:59.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.146+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff520029020 con 0x7ff5301092d0 2024-04-25T19:17:59.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:58 smithi183 ceph-mon[32344]: pgmap v471: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T19:17:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:58 smithi194 ceph-mon[32740]: pgmap v471: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T19:17:59.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.423+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff53002df80 con 0x7ff5301092d0 2024-04-25T19:17:59.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.424+0000 7ff52e000700 1 -- 172.21.15.86:0/128264089 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7ff52009e2a0 con 0x7ff5301092d0 2024-04-25T19:17:59.426 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:17:59.426 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:17:59.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff5140a3e30 msgr2=0x7ff5140a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff5140a3e30 0x7ff5140a62f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009680 tx=0x7ff51c009260 comp rx=0 tx=0).stop 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 msgr2=0x7ff5301a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301a53f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff520007a30 tx=0x7ff520007a60 comp rx=0 tx=0).stop 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 shutdown_connections 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff5140a3e30 0x7ff5140a62f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.427+0000 7ff537600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff53010e3b0 0x7ff53019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.428+0000 7ff537600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5301092d0 0x7ff5301a53f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.428+0000 7ff537600700 1 --2- 172.21.15.86:0/128264089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff530109c80 0x7ff5301a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:59.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.428+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 >> 172.21.15.86:0/128264089 conn(0x7ff530076210 msgr2=0x7ff530077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.428+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 shutdown_connections 2024-04-25T19:17:59.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:17:59.428+0000 7ff537600700 1 -- 172.21.15.86:0/128264089 wait complete. 2024-04-25T19:17:59.431 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:18:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:59 smithi183 ceph-mon[32344]: pgmap v472: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-04-25T19:18:00.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:17:59 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/128264089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:18:00.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:59 smithi194 ceph-mon[32740]: pgmap v472: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-04-25T19:18:00.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:17:59 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/128264089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:18:00.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:59 smithi086 ceph-mon[27783]: pgmap v472: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 KiB/s wr, 16 op/s 2024-04-25T19:18:00.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:17:59 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/128264089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:18:00.388 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 4a5a5612 --release --await' 2024-04-25T19:18:00.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 4a5a5612 --release --await 2024-04-25T19:18:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:18:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:02 smithi086 ceph-mon[27783]: pgmap v473: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:18:02.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:02 smithi183 ceph-mon[32344]: pgmap v473: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:18:02.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:02 smithi194 ceph-mon[32740]: pgmap v473: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-04-25T19:18:03.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 -- 172.21.15.86:0/1070197009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18108aa0 msgr2=0x7fbb18108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1070197009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18108aa0 0x7fbb18108e80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00b0a0 tx=0x7fbb0c00b3b0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 -- 172.21.15.86:0/1070197009 shutdown_connections 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1070197009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb181034b0 0x7fbb1810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1070197009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18108aa0 0x7fbb18108e80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1070197009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb18102af0 0x7fbb18102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 -- 172.21.15.86:0/1070197009 >> 172.21.15.86:0/1070197009 conn(0x7fbb18079b90 msgr2=0x7fbb18079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 -- 172.21.15.86:0/1070197009 shutdown_connections 2024-04-25T19:18:03.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.198+0000 7fbb1f600700 1 -- 172.21.15.86:0/1070197009 wait complete. 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.199+0000 7fbb1f600700 1 Processor -- start 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 -- start start 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18102af0 0x7fbb1819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 0x7fbb1819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb18108aa0 0x7fbb1819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18112cd0 con 0x7fbb181034b0 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18112b50 con 0x7fbb18108aa0 2024-04-25T19:18:03.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb1f600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18112e50 con 0x7fbb18102af0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 0x7fbb1819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 0x7fbb1819b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40056/0 (socket says 172.21.15.86:40056) 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.200+0000 7fbb17e00700 1 -- 172.21.15.86:0/1945967931 learned_addr learned my addr 172.21.15.86:0/1945967931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18102af0 0x7fbb1819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb17e00700 1 -- 172.21.15.86:0/1945967931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18102af0 msgr2=0x7fbb1819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb17e00700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18102af0 0x7fbb1819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb17e00700 1 -- 172.21.15.86:0/1945967931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb18108aa0 msgr2=0x7fbb1819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb17e00700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb18108aa0 0x7fbb1819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb17e00700 1 -- 172.21.15.86:0/1945967931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb181a0300 con 0x7fbb181034b0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.201+0000 7fbb17e00700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 0x7fbb1819b830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400d950 tx=0x7fbb0400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.202+0000 7fbb15600700 1 -- 172.21.15.86:0/1945967931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04016070 con 0x7fbb181034b0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.202+0000 7fbb15600700 1 -- 172.21.15.86:0/1945967931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04010460 con 0x7fbb181034b0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.202+0000 7fbb15600700 1 -- 172.21.15.86:0/1945967931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f620 con 0x7fbb181034b0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.202+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb181a05f0 con 0x7fbb181034b0 2024-04-25T19:18:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.202+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb181a7e70 con 0x7fbb181034b0 2024-04-25T19:18:03.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.205+0000 7fbb15600700 1 -- 172.21.15.86:0/1945967931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb04019070 con 0x7fbb181034b0 2024-04-25T19:18:03.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.206+0000 7fbb15600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb000a3e30 0x7fbb000a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.206+0000 7fbb15600700 1 -- 172.21.15.86:0/1945967931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7fbb040d04b0 con 0x7fbb181034b0 2024-04-25T19:18:03.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 Processor -- start 2024-04-25T19:18:03.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb000a3e30 0x7fbb000a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 -- start start 2024-04-25T19:18:03.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc0720120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc0720660 0x557cc072af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc072d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557cc0672f90 con 0x557cc071fd40 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557cc0619830 con 0x557cc072b4d0 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.207+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557cc066ef50 con 0x557cc0720660 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb000a3e30 0x7fbb000a62f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009e60 tx=0x7fbb0c00b7f0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc0720120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc0720120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40062/0 (socket says 172.21.15.86:40062) 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 -- 172.21.15.86:0/2199471739 learned_addr learned my addr 172.21.15.86:0/2199471739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb17e00700 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc0720660 0x557cc072af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1d600700 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc072d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 -- 172.21.15.86:0/2199471739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc0720660 msgr2=0x557cc072af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc0720660 0x557cc072af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 -- 172.21.15.86:0/2199471739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 msgr2=0x557cc072d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc072d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.208+0000 7fbb1cc00700 1 -- 172.21.15.86:0/2199471739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc060f780 con 0x557cc071fd40 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.209+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc0720120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c015000 tx=0x7fbb0c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15c2c0a4f308b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.209+0000 7fbb0a000700 1 -- 172.21.15.86:0/2199471739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c007b70 con 0x557cc071fd40 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.209+0000 7fbb0a000700 1 -- 172.21.15.86:0/2199471739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c02e070 con 0x557cc071fd40 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.209+0000 7fbb0a000700 1 -- 172.21.15.86:0/2199471739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c01c6a0 con 0x557cc071fd40 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 -- 172.21.15.86:0/2199471739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 msgr2=0x557cc0720120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc0720120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c015000 tx=0x7fbb0c01c000 comp rx=0 tx=0).stop 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 -- 172.21.15.86:0/2199471739 shutdown_connections 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc072d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc0720120 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/2199471739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc0720660 0x557cc072af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 -- 172.21.15.86:0/2199471739 >> 172.21.15.86:0/2199471739 conn(0x557cc064f050 msgr2=0x557cc071c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 -- 172.21.15.86:0/2199471739 shutdown_connections 2024-04-25T19:18:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.210+0000 7fbb29f83b80 1 -- 172.21.15.86:0/2199471739 wait complete. 2024-04-25T19:18:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 -- start start 2024-04-25T19:18:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 Processor -- start 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc072ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc0727e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc072b4d0 0x557cc07283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557cc072df60 con 0x557cc071fd40 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557cc062c960 con 0x557cc0720660 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557cc072e0e0 con 0x557cc072b4d0 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.211+0000 7fbb29f83b80 10 client.0 _ll_register_callbacks cb 0x557cc060b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc072ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1d600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc072b4d0 0x557cc07283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc072ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40072/0 (socket says 172.21.15.86:40072) 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1cc00700 1 -- 172.21.15.86:0/3590346121 learned_addr learned my addr 172.21.15.86:0/3590346121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb17e00700 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc0727e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1d600700 1 -- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 msgr2=0x557cc0727e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1d600700 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc0727e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1d600700 1 -- 172.21.15.86:0/3590346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 msgr2=0x557cc072ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1d600700 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc072ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.212+0000 7fbb1d600700 1 -- 172.21.15.86:0/3590346121 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc05c09d0 con 0x557cc072b4d0 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbb1d600700 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc072b4d0 0x557cc07283b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbb100059d0 tx=0x7fbb1000f9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad0c483b4e9ce52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbafea00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbafea00700 1 -- 172.21.15.86:0/3590346121 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1001d060 con 0x557cc072b4d0 2024-04-25T19:18:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbafea00700 1 -- 172.21.15.86:0/3590346121 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7fbb10010840 con 0x557cc072b4d0 2024-04-25T19:18:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c02e070 con 0x557cc072b4d0 2024-04-25T19:18:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbafea00700 1 -- 172.21.15.86:0/3590346121 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb10010a40 con 0x557cc072b4d0 2024-04-25T19:18:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbb29f83b80 10 client.27046 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.213+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cc0609bf0 con 0x557cc072b4d0 2024-04-25T19:18:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbafea00700 1 -- 172.21.15.86:0/3590346121 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c02e070 con 0x557cc072b4d0 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbafea00700 1 -- 172.21.15.86:0/3590346121 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbb10016340 con 0x557cc072b4d0 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb29f83b80 10 client.27046 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb29f83b80 10 client.27046 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb29f83b80 20 client.27046 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc073ac90 0x557cc073d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb29f83b80 4 client.27046 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cc0680780 con 0x557cc073ac90 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.214+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc073ac90 0x557cc073d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.215+0000 7fbafea00700 10 client.27046 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.215+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc073ac90 0x557cc073d0a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412f24b07767f455 server_cookie=af5f4d0675f95875 in_seq=0 out_seq=0 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbafea00700 1 -- 172.21.15.86:0/3590346121 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557cc0680780 con 0x557cc073ac90 2024-04-25T19:18:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbafea00700 10 client.27046 handle_command_reply: tid=0 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 client.27046 shutdown 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc072b4d0 msgr2=0x557cc07283b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc072b4d0 0x557cc07283b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbb100059d0 tx=0x7fbb1000f9a0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 shutdown_connections 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc073ac90 0x557cc073d0a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc072b4d0 0x557cc07283b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc071fd40 0x557cc072ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/3590346121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc0727e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 >> 172.21.15.86:0/3590346121 conn(0x557cc064f050 msgr2=0x557cc071be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbafea00700 0 client.27046 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 shutdown_connections 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.216+0000 7fbb29f83b80 1 -- 172.21.15.86:0/3590346121 wait complete. 2024-04-25T19:18:03.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.217+0000 7fbb29f83b80 20 client.27046 trim_cache size 0 max 16384 2024-04-25T19:18:03.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 Processor -- start 2024-04-25T19:18:03.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 -- start start 2024-04-25T19:18:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc0736e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc0720660 0x557cc0742970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc0742eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181af5f0 con 0x557cc0720660 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b1130 con 0x557cc072b4d0 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.238+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181afcc0 con 0x557cc071fd40 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc0736e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc0736e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59014/0 (socket says 172.21.15.86:59014) 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 -- 172.21.15.86:0/371801083 learned_addr learned my addr 172.21.15.86:0/371801083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb17e00700 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc0720660 0x557cc0742970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1d600700 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc0742eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 -- 172.21.15.86:0/371801083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 msgr2=0x557cc0742eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc0742eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 -- 172.21.15.86:0/371801083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc0720660 msgr2=0x557cc0742970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc0720660 0x557cc0742970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.239+0000 7fbb1cc00700 1 -- 172.21.15.86:0/371801083 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc072dcf0 con 0x557cc071fd40 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.240+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc0736e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009fd0 tx=0x7fbb0c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208afab10675aed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.240+0000 7fbb09600700 1 -- 172.21.15.86:0/371801083 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c03cdc0 con 0x557cc071fd40 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.240+0000 7fbb09600700 1 -- 172.21.15.86:0/371801083 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c02e050 con 0x557cc071fd40 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.240+0000 7fbb09600700 1 -- 172.21.15.86:0/371801083 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0053e0 con 0x557cc071fd40 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 -- 172.21.15.86:0/371801083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 msgr2=0x557cc0736e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc0736e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009fd0 tx=0x7fbb0c00b040 comp rx=0 tx=0).stop 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 -- 172.21.15.86:0/371801083 shutdown_connections 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc072b4d0 0x557cc0742eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc0736e90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/371801083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc0720660 0x557cc0742970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 -- 172.21.15.86:0/371801083 >> 172.21.15.86:0/371801083 conn(0x557cc064f050 msgr2=0x557cc0729f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 -- 172.21.15.86:0/371801083 shutdown_connections 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 -- 172.21.15.86:0/371801083 wait complete. 2024-04-25T19:18:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 -- start start 2024-04-25T19:18:03.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.241+0000 7fbb29f83b80 1 Processor -- start 2024-04-25T19:18:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc072e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc072eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc072b4d0 0x557cc079b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b0a60 con 0x557cc072b4d0 2024-04-25T19:18:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b0390 con 0x557cc0720660 2024-04-25T19:18:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181ad5b0 con 0x557cc071fd40 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb29f83b80 10 client.0 _ll_register_callbacks cb 0x557cc060b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb1d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc072b4d0 0x557cc079b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc072e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb17e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc072eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb1cc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc072e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:59022/0 (socket says 172.21.15.86:59022) 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.242+0000 7fbb1cc00700 1 -- 172.21.15.86:0/854744268 learned_addr learned my addr 172.21.15.86:0/854744268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1d600700 1 -- 172.21.15.86:0/854744268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 msgr2=0x557cc072e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1d600700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc072e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1d600700 1 -- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 msgr2=0x557cc072eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1d600700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc072eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1d600700 1 -- 172.21.15.86:0/854744268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc079b7a0 con 0x557cc072b4d0 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb17e00700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc072eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc072e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb1d600700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc072b4d0 0x557cc079b1b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb10001eb0 tx=0x7fbb100108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0793cdb6d3360b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbafea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbafea00700 1 -- 172.21.15.86:0/854744268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10016de0 con 0x557cc072b4d0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbafea00700 1 -- 172.21.15.86:0/854744268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8637+0+0 (secure 0 0 0) 0x7fbb10004830 con 0x557cc072b4d0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.243+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c02e050 con 0x557cc072b4d0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbafea00700 1 -- 172.21.15.86:0/854744268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb1000fcf0 con 0x557cc072b4d0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 10 client.16962 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cc07a2be0 con 0x557cc072b4d0 2024-04-25T19:18:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbafea00700 1 -- 172.21.15.86:0/854744268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c02e050 con 0x557cc072b4d0 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbafea00700 1 -- 172.21.15.86:0/854744268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbb10005560 con 0x557cc072b4d0 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 10 client.16962 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 10 client.16962 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 20 client.16962 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc07a3290 0x557cc07a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 4 client.16962 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4a5a5612", "release": true, "await": true}] 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.244+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4a5a5612", "release": true, "await": true}) -- 0x557cc067a820 con 0x557cc07a3290 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.245+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc07a3290 0x557cc07a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:03.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.245+0000 7fbb1cc00700 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc07a3290 0x557cc07a5680 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90069a3602d494e3 server_cookie=768ffcbd73f5c923 in_seq=0 out_seq=0 2024-04-25T19:18:03.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.245+0000 7fbafea00700 10 client.16962 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.248+0000 7fbafea00700 1 -- 172.21.15.86:0/854744268 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557cc067a820 con 0x557cc07a3290 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.248+0000 7fbafea00700 10 client.16962 handle_command_reply: tid=0 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.248+0000 7fbb29f83b80 1 client.16962 shutdown 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc072b4d0 msgr2=0x557cc079b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc072b4d0 0x557cc079b1b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb10001eb0 tx=0x7fbb100108f0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 shutdown_connections 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557cc07a3290 0x557cc07a5680 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557cc072b4d0 0x557cc079b1b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cc071fd40 0x557cc072e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 --2- 172.21.15.86:0/854744268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557cc0720660 0x557cc072eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbafea00700 0 client.16962 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 >> 172.21.15.86:0/854744268 conn(0x557cc064f050 msgr2=0x557cc07296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:03.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 shutdown_connections 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 1 -- 172.21.15.86:0/854744268 wait complete. 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.249+0000 7fbb29f83b80 20 client.16962 trim_cache size 0 max 16384 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 44, 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:18:03.252 INFO:teuthology.orchestra.run.smithi086.stdout: "4a5a5612": { 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 44, 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 84.7, 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:18:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:03.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:03.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:03.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:03.254 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:03.254 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:18:03.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb000a3e30 msgr2=0x7fbb000a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb000a3e30 0x7fbb000a62f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009e60 tx=0x7fbb0c00b7f0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 msgr2=0x7fbb1819b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 0x7fbb1819b830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400d950 tx=0x7fbb0400dd10 comp rx=0 tx=0).stop 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 shutdown_connections 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb18108aa0 0x7fbb1819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbb000a3e30 0x7fbb000a62f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18102af0 0x7fbb1819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 --2- 172.21.15.86:0/1945967931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb181034b0 0x7fbb1819b830 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.250+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 >> 172.21.15.86:0/1945967931 conn(0x7fbb18079b90 msgr2=0x7fbb18100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.251+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 shutdown_connections 2024-04-25T19:18:03.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:03.251+0000 7fbb1f600700 1 -- 172.21.15.86:0/1945967931 wait complete. 2024-04-25T19:18:04.267 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4a5a5612', seconds elapsed: 40.4 2024-04-25T19:18:04.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:04 smithi086 ceph-mon[27783]: pgmap v474: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 31 op/s 2024-04-25T19:18:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:04 smithi183 ceph-mon[32344]: pgmap v474: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 31 op/s 2024-04-25T19:18:04.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:04 smithi194 ceph-mon[32740]: pgmap v474: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 31 op/s 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:437/600: open fd = 3 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:438/600: open fd = 3 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:439/600: open fd = 3 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:440/600: open fd = 3 2024-04-25T19:18:06.295 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:441/600: open fd = 3 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:442/600: open fd = 3 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:443/600: open fd = 3 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:444/600: open fd = 3 2024-04-25T19:18:06.296 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:445/600: open fd = 3 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:446/600: open fd = 3 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:447/600: open fd = 3 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:448/600: open fd = 3 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.297 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:449/600: open fd = 3 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:450/600: open fd = 3 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:451/600: open fd = 3 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:452/600: open fd = 3 2024-04-25T19:18:06.298 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:453/600: open fd = 3 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:454/600: open fd = 3 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:455/600: open fd = 3 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:456/600: open fd = 3 2024-04-25T19:18:06.299 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:457/600: open fd = 3 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:458/600: open fd = 3 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:459/600: open fd = 3 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.300 INFO:tasks.workunit.client.0.smithi086.stdout:460/600: open fd = 3 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:461/600: open fd = 3 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:462/600: open fd = 3 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:463/600: open fd = 3 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.301 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:464/600: open fd = 3 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:465/600: open fd = 3 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:466/600: open fd = 3 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:467/600: open fd = 3 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.302 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:468/600: open fd = 3 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:469/600: open fd = 3 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:470/600: open fd = 3 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:471/600: open fd = 3 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.303 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:472/600: open fd = 3 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:473/600: open fd = 3 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:474/600: open fd = 3 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:475/600: open fd = 3 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.304 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:476/600: open fd = 3 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:477/600: open fd = 3 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:478/600: open fd = 3 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:479/600: open fd = 3 2024-04-25T19:18:06.305 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:480/600: open fd = 3 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:481/600: open fd = 3 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:482/600: open fd = 3 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.306 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:483/600: open fd = 3 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:484/600: open fd = 3 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:485/600: open fd = 3 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:486/600: open fd = 3 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.307 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:487/600: open fd = 3 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:488/600: open fd = 3 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:489/600: open fd = 3 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:490/600: open fd = 3 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.308 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:491/600: open fd = 3 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:492/600: open fd = 3 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:493/600: open fd = 3 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.309 INFO:tasks.workunit.client.0.smithi086.stdout:494/600: open fd = 3 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:495/600: open fd = 3 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:496/600: open fd = 3 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:497/600: open fd = 3 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.310 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.311 INFO:tasks.workunit.client.0.smithi086.stdout:498/600: open fd = 3 2024-04-25T19:18:06.311 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.311 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:18:06.311 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:18:06.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:06 smithi086 ceph-mon[27783]: pgmap v475: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-25T19:18:06.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:06 smithi183 ceph-mon[32344]: pgmap v475: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-25T19:18:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:06 smithi194 ceph-mon[32740]: pgmap v475: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-25T19:18:07.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:07 smithi086 ceph-mon[27783]: osdmap e113: 12 total, 12 up, 12 in 2024-04-25T19:18:07.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:07 smithi183 ceph-mon[32344]: osdmap e113: 12 total, 12 up, 12 in 2024-04-25T19:18:07.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:07 smithi194 ceph-mon[32740]: osdmap e113: 12 total, 12 up, 12 in 2024-04-25T19:18:08.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:08 smithi086 ceph-mon[27783]: pgmap v477: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:18:08.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:08 smithi183 ceph-mon[32344]: pgmap v477: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:18:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:08 smithi194 ceph-mon[32740]: pgmap v477: 97 pgs: 97 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 -- 172.21.15.86:0/3724335630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d810e840 msgr2=0x7f62d8110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 --2- 172.21.15.86:0/3724335630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d810e840 0x7f62d8110c30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009900 tx=0x7f62c802a480 comp rx=0 tx=0).stop 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 -- 172.21.15.86:0/3724335630 shutdown_connections 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 --2- 172.21.15.86:0/3724335630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d810e840 0x7f62d8110c30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 --2- 172.21.15.86:0/3724335630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074b40 0x7f62d810e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 --2- 172.21.15.86:0/3724335630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62d8074220 0x7f62d8074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 -- 172.21.15.86:0/3724335630 >> 172.21.15.86:0/3724335630 conn(0x7f62d80793e0 msgr2=0x7f62d80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 -- 172.21.15.86:0/3724335630 shutdown_connections 2024-04-25T19:18:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.645+0000 7f62dd600640 1 -- 172.21.15.86:0/3724335630 wait complete. 2024-04-25T19:18:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.646+0000 7f62dd600640 1 Processor -- start 2024-04-25T19:18:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.646+0000 7f62dd600640 1 -- start start 2024-04-25T19:18:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62dd600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074220 0x7f62d81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62dd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 0x7f62d81a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62dd600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62d810e840 0x7f62d819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62dd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81143d0 con 0x7f62d8074b40 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62dd600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8114250 con 0x7f62d810e840 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62dd600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8114550 con 0x7f62d8074220 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074220 0x7f62d81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 0x7f62d81a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 0x7f62d81a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47178/0 (socket says 172.21.15.86:47178) 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074220 0x7f62d81a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33592/0 (socket says 172.21.15.86:33592) 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 -- 172.21.15.86:0/193525971 learned_addr learned my addr 172.21.15.86:0/193525971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62d810e840 0x7f62d819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 -- 172.21.15.86:0/193525971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074220 msgr2=0x7f62d81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074220 0x7f62d81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 -- 172.21.15.86:0/193525971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62d810e840 msgr2=0x7f62d819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62d810e840 0x7f62d819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.647+0000 7f62d6000640 1 -- 172.21.15.86:0/193525971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d819af70 con 0x7f62d8074b40 2024-04-25T19:18:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.648+0000 7f62d6000640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 0x7f62d81a0b30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f62c400e9f0 tx=0x7f62c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.648+0000 7f62cf400640 1 -- 172.21.15.86:0/193525971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c400cd80 con 0x7f62d8074b40 2024-04-25T19:18:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.648+0000 7f62cf400640 1 -- 172.21.15.86:0/193525971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c40054d0 con 0x7f62d8074b40 2024-04-25T19:18:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.648+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d819b260 con 0x7f62d8074b40 2024-04-25T19:18:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.648+0000 7f62cf400640 1 -- 172.21.15.86:0/193525971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4010690 con 0x7f62d8074b40 2024-04-25T19:18:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.648+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d81a7570 con 0x7f62d8074b40 2024-04-25T19:18:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.651+0000 7f62e0436740 1 Processor -- start 2024-04-25T19:18:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.651+0000 7f62e0436740 1 -- start start 2024-04-25T19:18:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62e0436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 0x5640f1db0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62e0436740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dbb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62e0436740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1dbddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62e0436740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640f1d16900 con 0x5640f1db1140 2024-04-25T19:18:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62e0436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5640f1cbcef0 con 0x5640f1db0820 2024-04-25T19:18:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62e0436740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5640f1d128c0 con 0x5640f1dbba00 2024-04-25T19:18:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62cf400640 1 -- 172.21.15.86:0/193525971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62c4004e90 con 0x7f62d8074b40 2024-04-25T19:18:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dbb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62d7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 0x5640f1db0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62d7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 0x5640f1db0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58614/0 (socket says 172.21.15.86:58614) 2024-04-25T19:18:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62d7400640 1 -- 172.21.15.86:0/2857975491 learned_addr learned my addr 172.21.15.86:0/2857975491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.652+0000 7f62d6000640 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1dbddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d7400640 1 -- 172.21.15.86:0/2857975491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 msgr2=0x5640f1dbddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d7400640 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1dbddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d7400640 1 -- 172.21.15.86:0/2857975491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 msgr2=0x5640f1dbb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d7400640 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dbb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d7400640 1 -- 172.21.15.86:0/2857975491 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f1a29210 con 0x5640f1db0820 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d6000640 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1dbddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62d7400640 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 0x5640f1db0c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62c8031440 tx=0x7f62c8032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66a1d9cdfd3f1111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62c3400640 1 -- 172.21.15.86:0/2857975491 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8040070 con 0x5640f1db0820 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62c3400640 1 -- 172.21.15.86:0/2857975491 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c8004030 con 0x5640f1db0820 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62c3400640 1 -- 172.21.15.86:0/2857975491 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8035040 con 0x5640f1db0820 2024-04-25T19:18:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.653+0000 7f62cf400640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f62bc0a3e40 0x7f62bc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62cf400640 1 -- 172.21.15.86:0/193525971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8681+0+0 (secure 0 0 0) 0x7f62c40d0bc0 con 0x7f62d8074b40 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62d6a00640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f62bc0a3e40 0x7f62bc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 -- 172.21.15.86:0/2857975491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 msgr2=0x5640f1db0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 0x5640f1db0c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f62c8031440 tx=0x7f62c8032b20 comp rx=0 tx=0).stop 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 -- 172.21.15.86:0/2857975491 shutdown_connections 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1dbddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dbb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 --2- 172.21.15.86:0/2857975491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db0820 0x5640f1db0c00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 -- 172.21.15.86:0/2857975491 >> 172.21.15.86:0/2857975491 conn(0x5640f1dacd30 msgr2=0x5640f1dad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 -- 172.21.15.86:0/2857975491 shutdown_connections 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62e0436740 1 -- 172.21.15.86:0/2857975491 wait complete. 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.654+0000 7f62d6a00640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f62bc0a3e40 0x7f62bc0a6300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5640f1db1dc0 tx=0x7f62d0009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.655+0000 7f62e0436740 1 -- start start 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.655+0000 7f62e0436740 1 Processor -- start 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.655+0000 7f62e0436740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 0x5640f1db89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.655+0000 7f62e0436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1db8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62e0436740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1db9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62e0436740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640f1dbe5d0 con 0x5640f1db0820 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62e0436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5640f1cd0310 con 0x5640f1db1140 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62e0436740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5640f1dbe750 con 0x5640f1dbba00 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62e0436740 10 client.0 _ll_register_callbacks cb 0x5640f1c9ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 0x5640f1db89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1db8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 0x5640f1db89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47196/0 (socket says 172.21.15.86:47196) 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6a00640 1 -- 172.21.15.86:0/266478624 learned_addr learned my addr 172.21.15.86:0/266478624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d7400640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1db9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 -- 172.21.15.86:0/266478624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 msgr2=0x5640f1db9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1db9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 -- 172.21.15.86:0/266478624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 msgr2=0x5640f1db89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 0x5640f1db89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 -- 172.21.15.86:0/266478624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f1af5620 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6a00640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 0x5640f1db89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.656+0000 7f62d6000640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1db8ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f62c40ce270 tx=0x7f62c409c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89def6567283d178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62b7e00640 1 -- 172.21.15.86:0/266478624 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c409d1c0 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c8004030 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62b7e00640 1 -- 172.21.15.86:0/266478624 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8681+0+0 (secure 0 0 0) 0x7f62c40ce4f0 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62b7e00640 1 -- 172.21.15.86:0/266478624 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62c40ce6f0 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62e0436740 10 client.27065 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640f1dca900 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.657+0000 7f62b7e00640 1 -- 172.21.15.86:0/266478624 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c8004030 con 0x5640f1db1140 2024-04-25T19:18:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62b7e00640 1 -- 172.21.15.86:0/266478624 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f62c40a52a0 con 0x5640f1db1140 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62e0436740 10 client.27065 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62e0436740 10 client.27065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62e0436740 20 client.27065 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62e0436740 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1dcbd70 0x5640f1dce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62e0436740 4 client.27065 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640f1d2a8b0 con 0x5640f1dcbd70 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.658+0000 7f62d6a00640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1dcbd70 0x5640f1dce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.659+0000 7f62d6a00640 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1dcbd70 0x5640f1dce160 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a61899029b14a9 server_cookie=44b45e44286158ba in_seq=0 out_seq=0 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.659+0000 7f62b7e00640 10 client.27065 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62b7e00640 1 -- 172.21.15.86:0/266478624 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640f1d2a8b0 con 0x5640f1dcbd70 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62b7e00640 10 client.27065 handle_command_reply: tid=0 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 client.27065 shutdown 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 msgr2=0x5640f1db8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1db8ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f62c40ce270 tx=0x7f62c409c6c0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 shutdown_connections 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1dcbd70 0x5640f1dce160 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1dbba00 0x5640f1db9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62b7e00640 0 client.27065 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1db8ef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 --2- 172.21.15.86:0/266478624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db0820 0x5640f1db89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 >> 172.21.15.86:0/266478624 conn(0x5640f1dacd30 msgr2=0x5640f1db2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 shutdown_connections 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 1 -- 172.21.15.86:0/266478624 wait complete. 2024-04-25T19:18:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.660+0000 7f62e0436740 20 client.27065 trim_cache size 0 max 16384 2024-04-25T19:18:10.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:10 smithi183 ceph-mon[32344]: pgmap v478: 97 pgs: 97 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-04-25T19:18:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.676+0000 7f62e0436740 1 Processor -- start 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.676+0000 7f62e0436740 1 -- start start 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62e0436740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1db3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62e0436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1dd3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62e0436740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1dbba00 0x5640f1dd3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62e0436740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81affc0 con 0x5640f1dbba00 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62e0436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81b1b00 con 0x5640f1db1140 2024-04-25T19:18:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62e0436740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81b0690 con 0x5640f1db0820 2024-04-25T19:18:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1db3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1db3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33632/0 (socket says 172.21.15.86:33632) 2024-04-25T19:18:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 -- 172.21.15.86:0/2125055106 learned_addr learned my addr 172.21.15.86:0/2125055106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d7400640 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1dbba00 0x5640f1dd3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6000640 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1dd3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 -- 172.21.15.86:0/2125055106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 msgr2=0x5640f1dd3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1dd3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 -- 172.21.15.86:0/2125055106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1dbba00 msgr2=0x5640f1dd3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.677+0000 7f62d6a00640 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1dbba00 0x5640f1dd3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.678+0000 7f62d6a00640 1 -- 172.21.15.86:0/2125055106 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f1dce7c0 con 0x5640f1db0820 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.678+0000 7f62d6a00640 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1db3230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005fd0 tx=0x7f62d0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df64242cd1822d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.678+0000 7f62c2a00640 1 -- 172.21.15.86:0/2125055106 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0004b20 con 0x5640f1db0820 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.678+0000 7f62c2a00640 1 -- 172.21.15.86:0/2125055106 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d0004cc0 con 0x5640f1db0820 2024-04-25T19:18:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.678+0000 7f62c2a00640 1 -- 172.21.15.86:0/2125055106 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00054c0 con 0x5640f1db0820 2024-04-25T19:18:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 -- 172.21.15.86:0/2125055106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 msgr2=0x5640f1db3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1db3230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005fd0 tx=0x7f62d0018000 comp rx=0 tx=0).stop 2024-04-25T19:18:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 -- 172.21.15.86:0/2125055106 shutdown_connections 2024-04-25T19:18:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1dbba00 0x5640f1dd3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1db1140 0x5640f1dd3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 --2- 172.21.15.86:0/2125055106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1db3230 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 -- 172.21.15.86:0/2125055106 >> 172.21.15.86:0/2125055106 conn(0x5640f1dacd30 msgr2=0x5640f1dab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 -- 172.21.15.86:0/2125055106 shutdown_connections 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 -- 172.21.15.86:0/2125055106 wait complete. 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.679+0000 7f62e0436740 1 -- start start 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 Processor -- start 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1dc3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dc1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1dbba00 0x5640f1dc1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81b1430 con 0x5640f1db1140 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81b0d60 con 0x5640f1dbba00 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81adfc0 con 0x5640f1db0820 2024-04-25T19:18:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62e0436740 10 client.0 _ll_register_callbacks cb 0x7f62c4098e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62d6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dc1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62d6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dc1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47220/0 (socket says 172.21.15.86:47220) 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62d6000640 1 -- 172.21.15.86:0/1813704691 learned_addr learned my addr 172.21.15.86:0/1813704691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62d6a00640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1dc3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.680+0000 7f62d7400640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1dbba00 0x5640f1dc1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62d6000640 1 -- 172.21.15.86:0/1813704691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 msgr2=0x5640f1dc3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62d6000640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1dc3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62d6000640 1 -- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1dbba00 msgr2=0x5640f1dc1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62d6000640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1dbba00 0x5640f1dc1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62d6000640 1 -- 172.21.15.86:0/1813704691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f1dbddf0 con 0x5640f1db1140 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62d6000640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dc1010 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f62c4099440 tx=0x7f62c40048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fc1872836518af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62b7e00640 1 -- 172.21.15.86:0/1813704691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4005180 con 0x5640f1db1140 2024-04-25T19:18:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62b7e00640 1 -- 172.21.15.86:0/1813704691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8681+0+0 (secure 0 0 0) 0x7f62c409bf10 con 0x5640f1db1140 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.681+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d0004cc0 con 0x5640f1db1140 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62b7e00640 1 -- 172.21.15.86:0/1813704691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62c40ce6f0 con 0x5640f1db1140 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 10 client.16980 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640f1e32040 con 0x5640f1db1140 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62b7e00640 1 -- 172.21.15.86:0/1813704691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d0004cc0 con 0x5640f1db1140 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62b7e00640 1 -- 172.21.15.86:0/1813704691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f62c40a2000 con 0x5640f1db1140 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 10 client.16980 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 20 client.16980 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1e32790 0x5640f1e34bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 4 client.16980 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:18:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.682+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640f1d24320 con 0x5640f1e32790 2024-04-25T19:18:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.683+0000 7f62d6a00640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1e32790 0x5640f1e34bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.683+0000 7f62d6a00640 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1e32790 0x5640f1e34bb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d626b3fddcfe server_cookie=42eceb0294c14d1 in_seq=0 out_seq=0 2024-04-25T19:18:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.683+0000 7f62b7e00640 10 client.16980 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.694+0000 7f62b7e00640 1 -- 172.21.15.86:0/1813704691 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640f1d24320 con 0x5640f1e32790 2024-04-25T19:18:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.694+0000 7f62b7e00640 10 client.16980 handle_command_reply: tid=0 2024-04-25T19:18:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 client.16980 shutdown 2024-04-25T19:18:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 msgr2=0x5640f1dc1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dc1010 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f62c4099440 tx=0x7f62c40048d0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 shutdown_connections 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5640f1e32790 0x5640f1e34bb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640f1dbba00 0x5640f1dc1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f1db1140 0x5640f1dc1010 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 --2- 172.21.15.86:0/1813704691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640f1db0820 0x5640f1dc3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 >> 172.21.15.86:0/1813704691 conn(0x5640f1dacd30 msgr2=0x5640f1dab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62b7e00640 0 client.16980 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 shutdown_connections 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 1 -- 172.21.15.86:0/1813704691 wait complete. 2024-04-25T19:18:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.695+0000 7f62e0436740 20 client.16980 trim_cache size 0 max 16384 2024-04-25T19:18:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:18:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:18:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:18:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:18:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f62bc0a3e40 msgr2=0x7f62bc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f62bc0a3e40 0x7f62bc0a6300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5640f1db1dc0 tx=0x7f62d0009500 comp rx=0 tx=0).stop 2024-04-25T19:18:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 msgr2=0x7f62d81a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 0x7f62d81a0b30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f62c400e9f0 tx=0x7f62c400edf0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 shutdown_connections 2024-04-25T19:18:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f62bc0a3e40 0x7f62bc0a6300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62d810e840 0x7f62d819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d8074b40 0x7f62d81a0b30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 --2- 172.21.15.86:0/193525971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d8074220 0x7f62d81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 >> 172.21.15.86:0/193525971 conn(0x7f62d80793e0 msgr2=0x7f62d80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 shutdown_connections 2024-04-25T19:18:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:10.696+0000 7f62dd600640 1 -- 172.21.15.86:0/193525971 wait complete. 2024-04-25T19:18:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:10 smithi194 ceph-mon[32740]: pgmap v478: 97 pgs: 97 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-04-25T19:18:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:10 smithi086 ceph-mon[27783]: pgmap v478: 97 pgs: 97 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-04-25T19:18:12.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:12 smithi183 ceph-mon[32344]: pgmap v479: 97 pgs: 97 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-04-25T19:18:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:12 smithi194 ceph-mon[32740]: pgmap v479: 97 pgs: 97 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-04-25T19:18:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:12 smithi086 ceph-mon[27783]: pgmap v479: 97 pgs: 97 active+clean; 5.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-04-25T19:18:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:14 smithi183 ceph-mon[32344]: pgmap v480: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T19:18:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:14 smithi183 ceph-mon[32344]: osdmap e114: 12 total, 12 up, 12 in 2024-04-25T19:18:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:14 smithi194 ceph-mon[32740]: pgmap v480: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T19:18:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:14 smithi194 ceph-mon[32740]: osdmap e114: 12 total, 12 up, 12 in 2024-04-25T19:18:15.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:14 smithi086 ceph-mon[27783]: pgmap v480: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T19:18:15.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:14 smithi086 ceph-mon[27783]: osdmap e114: 12 total, 12 up, 12 in 2024-04-25T19:18:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:16 smithi183 ceph-mon[32344]: pgmap v482: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T19:18:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:16 smithi194 ceph-mon[32740]: pgmap v482: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T19:18:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:16 smithi086 ceph-mon[27783]: pgmap v482: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T19:18:18.874 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:18 smithi086 ceph-mon[27783]: pgmap v483: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T19:18:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:18 smithi183 ceph-mon[32344]: pgmap v483: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T19:18:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:18 smithi194 ceph-mon[32740]: pgmap v483: 97 pgs: 97 active+clean; 1.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-25T19:18:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:20 smithi183 ceph-mon[32344]: pgmap v484: 97 pgs: 97 active+clean; 1.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T19:18:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:20 smithi194 ceph-mon[32740]: pgmap v484: 97 pgs: 97 active+clean; 1.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T19:18:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:20 smithi086 ceph-mon[27783]: pgmap v484: 97 pgs: 97 active+clean; 1.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T19:18:22.918 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:22 smithi183 ceph-mon[32344]: pgmap v485: 97 pgs: 97 active+clean; 1.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T19:18:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:22 smithi194 ceph-mon[32740]: pgmap v485: 97 pgs: 97 active+clean; 1.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T19:18:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:22 smithi086 ceph-mon[27783]: pgmap v485: 97 pgs: 97 active+clean; 1.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-25T19:18:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 -- 172.21.15.86:0/2304363443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 msgr2=0x7fe1f810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 --2- 172.21.15.86:0/2304363443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f810ea00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009900 tx=0x7fe1e802a400 comp rx=0 tx=0).stop 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 -- 172.21.15.86:0/2304363443 shutdown_connections 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 --2- 172.21.15.86:0/2304363443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 0x7fe1f81153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 --2- 172.21.15.86:0/2304363443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 0x7fe1f81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 --2- 172.21.15.86:0/2304363443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f810ea00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 -- 172.21.15.86:0/2304363443 >> 172.21.15.86:0/2304363443 conn(0x7fe1f80764e0 msgr2=0x7fe1f8078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 -- 172.21.15.86:0/2304363443 shutdown_connections 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.836+0000 7fe1ff200640 1 -- 172.21.15.86:0/2304363443 wait complete. 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.837+0000 7fe1ff200640 1 Processor -- start 2024-04-25T19:18:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.837+0000 7fe1ff200640 1 -- start start 2024-04-25T19:18:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1ff200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 0x7fe1f81a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1ff200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 0x7fe1f81a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1ff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1ff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8118800 con 0x7fe1f810e620 2024-04-25T19:18:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1ff200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8118680 con 0x7fe1f8108f20 2024-04-25T19:18:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1ff200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8118980 con 0x7fe1f8108560 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 0x7fe1f81a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 0x7fe1f81a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34448/0 (socket says 172.21.15.86:34448) 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 0x7fe1f81a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1f7e00640 1 -- 172.21.15.86:0/529178337 learned_addr learned my addr 172.21.15.86:0/529178337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34636/0 (socket says 172.21.15.86:34636) 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 -- 172.21.15.86:0/529178337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 msgr2=0x7fe1f81a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 0x7fe1f81a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 -- 172.21.15.86:0/529178337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 msgr2=0x7fe1f81a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 0x7fe1f81a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1fd200640 1 -- 172.21.15.86:0/529178337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f819f200 con 0x7fe1f810e620 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.838+0000 7fe1f7400640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 0x7fe1f81a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.839+0000 7fe1fd200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f819ea80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000ec50 tx=0x7fe1f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.839+0000 7fe1f4c00640 1 -- 172.21.15.86:0/529178337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0003b70 con 0x7fe1f810e620 2024-04-25T19:18:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.839+0000 7fe1f4c00640 1 -- 172.21.15.86:0/529178337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f0003d10 con 0x7fe1f810e620 2024-04-25T19:18:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.839+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f819f4f0 con 0x7fe1f810e620 2024-04-25T19:18:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.839+0000 7fe1f4c00640 1 -- 172.21.15.86:0/529178337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0010830 con 0x7fe1f810e620 2024-04-25T19:18:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.839+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81ab790 con 0x7fe1f810e620 2024-04-25T19:18:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.843+0000 7fe202051740 1 Processor -- start 2024-04-25T19:18:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe1f4c00640 1 -- 172.21.15.86:0/529178337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1f0003ec0 con 0x7fe1f810e620 2024-04-25T19:18:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 -- start start 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabd3820 0x55ffdabd3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabde4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 0x55ffdabe0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffdab39900 con 0x55ffdabdea00 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ffdaadfef0 con 0x55ffdabd3820 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.844+0000 7fe202051740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ffdab358c0 con 0x55ffdabd4140 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 0x55ffdabe0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 0x55ffdabe0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34644/0 (socket says 172.21.15.86:34644) 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 -- 172.21.15.86:0/1420226884 learned_addr learned my addr 172.21.15.86:0/1420226884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabde4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabd3820 0x55ffdabd3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 -- 172.21.15.86:0/1420226884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 msgr2=0x55ffdabde4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabde4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 -- 172.21.15.86:0/1420226884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabd3820 msgr2=0x55ffdabd3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabd3820 0x55ffdabd3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7400640 1 -- 172.21.15.86:0/1420226884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffda84c210 con 0x55ffdabdea00 2024-04-25T19:18:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.845+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabde4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1f7400640 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 0x55ffdabe0df0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4013bd0 tx=0x7fe1e4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396596af937df43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1f4c00640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1f4c00640 1 -- 172.21.15.86:0/529178337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7fe1f00d0e10 con 0x7fe1f810e620 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1ed600640 1 -- 172.21.15.86:0/1420226884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4013cc0 con 0x55ffdabdea00 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1ed600640 1 -- 172.21.15.86:0/1420226884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e4011c70 con 0x55ffdabdea00 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.846+0000 7fe1ed600640 1 -- 172.21.15.86:0/1420226884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4003cc0 con 0x55ffdabdea00 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 -- 172.21.15.86:0/1420226884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 msgr2=0x55ffdabe0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ffdabd4dc0 tx=0x7fe1e80046c0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 0x55ffdabe0df0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4013bd0 tx=0x7fe1e4009b80 comp rx=0 tx=0).stop 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 -- 172.21.15.86:0/1420226884 shutdown_connections 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabdea00 0x55ffdabe0df0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabde4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 --2- 172.21.15.86:0/1420226884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabd3820 0x55ffdabd3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 -- 172.21.15.86:0/1420226884 >> 172.21.15.86:0/1420226884 conn(0x55ffdabcfd30 msgr2=0x55ffdabd0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 -- 172.21.15.86:0/1420226884 shutdown_connections 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 -- 172.21.15.86:0/1420226884 wait complete. 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.847+0000 7fe202051740 1 -- start start 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 Processor -- start 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabdbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabdc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabdc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffdabe1630 con 0x55ffdabd4140 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ffdaaf3310 con 0x55ffdabdea00 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ffdabe17b0 con 0x55ffdabd3820 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe202051740 10 client.0 _ll_register_callbacks cb 0x55ffdaac1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabdc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabdc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34658/0 (socket says 172.21.15.86:34658) 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.848+0000 7fe1f7400640 1 -- 172.21.15.86:0/4215960379 learned_addr learned my addr 172.21.15.86:0/4215960379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1fd200640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabdc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabdbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7400640 1 -- 172.21.15.86:0/4215960379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 msgr2=0x55ffdabdbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7400640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabdbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7400640 1 -- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 msgr2=0x55ffdabdc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7400640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabdc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7400640 1 -- 172.21.15.86:0/4215960379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffda918620 con 0x55ffdabd4140 2024-04-25T19:18:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1f7400640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabdc010 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4013ba0 tx=0x7fe1e4009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ea6f7942d332fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1e2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1e2000640 1 -- 172.21.15.86:0/4215960379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4005450 con 0x55ffdabd4140 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.849+0000 7fe1e2000640 1 -- 172.21.15.86:0/4215960379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7fe1e40055f0 con 0x55ffdabd4140 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e4011c70 con 0x55ffdabd4140 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe1e2000640 1 -- 172.21.15.86:0/4215960379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1e40046c0 con 0x55ffdabd4140 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 10 client.16992 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffdabeda10 con 0x55ffdabd4140 2024-04-25T19:18:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe1e2000640 1 -- 172.21.15.86:0/4215960379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e4011c70 con 0x55ffdabd4140 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe1e2000640 1 -- 172.21.15.86:0/4215960379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe1e4020910 con 0x55ffdabd4140 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 10 client.16992 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 20 client.16992 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdabeedc0 0x55ffdabf11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 4 client.16992 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.850+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffdab4d8b0 con 0x55ffdabeedc0 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.851+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdabeedc0 0x55ffdabf11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.851+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdabeedc0 0x55ffdabf11b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c97a8419f0a8dc server_cookie=b1e0652cbea609a8 in_seq=0 out_seq=0 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.851+0000 7fe1e2000640 10 client.16992 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.852+0000 7fe1e2000640 1 -- 172.21.15.86:0/4215960379 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffdab4d8b0 con 0x55ffdabeedc0 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.852+0000 7fe1e2000640 10 client.16992 handle_command_reply: tid=0 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.852+0000 7fe202051740 1 client.16992 shutdown 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 msgr2=0x55ffdabdc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabdc010 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4013ba0 tx=0x7fe1e4009f20 comp rx=0 tx=0).stop 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 shutdown_connections 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdabeedc0 0x55ffdabf11b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabdc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe1e2000640 0 client.16992 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabdc010 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 --2- 172.21.15.86:0/4215960379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabdbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 >> 172.21.15.86:0/4215960379 conn(0x55ffdabcfd30 msgr2=0x55ffdabd5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 shutdown_connections 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 1 -- 172.21.15.86:0/4215960379 wait complete. 2024-04-25T19:18:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.853+0000 7fe202051740 20 client.16992 trim_cache size 0 max 16384 2024-04-25T19:18:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 Processor -- start 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 -- start start 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 0x55ffdabea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabd61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabf68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b4350 con 0x55ffdabd3820 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b5e90 con 0x55ffdabdea00 2024-04-25T19:18:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.869+0000 7fe202051740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b4a20 con 0x55ffdabd4140 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 0x55ffdabea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabd61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabd61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:34480/0 (socket says 172.21.15.86:34480) 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7400640 1 -- 172.21.15.86:0/1102449629 learned_addr learned my addr 172.21.15.86:0/1102449629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 0x55ffdabea4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34670/0 (socket says 172.21.15.86:34670) 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1fd200640 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabf68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 -- 172.21.15.86:0/1102449629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 msgr2=0x55ffdabd61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabd61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 -- 172.21.15.86:0/1102449629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 msgr2=0x55ffdabf68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabf68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 -- 172.21.15.86:0/1102449629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffdabd3f70 con 0x55ffdabd3820 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7400640 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabd61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.870+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 0x55ffdabea4f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8000c00 tx=0x7fe1e803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efbe0d058dc1c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe1ecc00640 1 -- 172.21.15.86:0/1102449629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8057070 con 0x55ffdabd3820 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe1ecc00640 1 -- 172.21.15.86:0/1102449629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8003e40 con 0x55ffdabd3820 2024-04-25T19:18:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe1ecc00640 1 -- 172.21.15.86:0/1102449629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8035040 con 0x55ffdabd3820 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 -- 172.21.15.86:0/1102449629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 msgr2=0x55ffdabea4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 0x55ffdabea4f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8000c00 tx=0x7fe1e803f000 comp rx=0 tx=0).stop 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 -- 172.21.15.86:0/1102449629 shutdown_connections 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabf68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd4140 0x55ffdabd61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 --2- 172.21.15.86:0/1102449629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd3820 0x55ffdabea4f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.871+0000 7fe202051740 1 -- 172.21.15.86:0/1102449629 >> 172.21.15.86:0/1102449629 conn(0x55ffdabcfd30 msgr2=0x55ffdabd0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.872+0000 7fe202051740 1 -- 172.21.15.86:0/1102449629 shutdown_connections 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.872+0000 7fe202051740 1 -- 172.21.15.86:0/1102449629 wait complete. 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.872+0000 7fe202051740 1 -- start start 2024-04-25T19:18:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.872+0000 7fe202051740 1 Processor -- start 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabd8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabd8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabd93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b57c0 con 0x55ffdabd4140 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b50f0 con 0x55ffdabdea00 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b2350 con 0x55ffdabd3820 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe202051740 10 client.0 _ll_register_callbacks cb 0x7fe1e4013690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabd8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabd8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34682/0 (socket says 172.21.15.86:34682) 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 -- 172.21.15.86:0/647403318 learned_addr learned my addr 172.21.15.86:0/647403318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabd8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1fd200640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabd93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 -- 172.21.15.86:0/647403318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 msgr2=0x55ffdabd8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabd8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 -- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 msgr2=0x55ffdabd93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabd93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7400640 1 -- 172.21.15.86:0/647403318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffdac54210 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.873+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabd8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1f7400640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabd8e70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4015010 tx=0x7fe1e4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf51a538155458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1e2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1e2000640 1 -- 172.21.15.86:0/647403318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4035070 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1e2000640 1 -- 172.21.15.86:0/647403318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7fe1e4005250 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e8003e40 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1e2000640 1 -- 172.21.15.86:0/647403318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1e4005450 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe202051740 10 client.16998 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffdac554c0 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1e2000640 1 -- 172.21.15.86:0/647403318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8003e40 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.874+0000 7fe1e2000640 1 -- 172.21.15.86:0/647403318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe1e4004b00 con 0x55ffdabd4140 2024-04-25T19:18:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe202051740 10 client.16998 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe202051740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe202051740 20 client.16998 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe202051740 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdac55c00 0x55ffdac58020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe202051740 4 client.16998 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffdab47320 con 0x55ffdac55c00 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdac55c00 0x55ffdac58020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.876+0000 7fe1e2000640 10 client.16998 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.875+0000 7fe1f7e00640 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdac55c00 0x55ffdac58020 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3d0f1e1c636b66 server_cookie=7a13d72370d626e in_seq=0 out_seq=0 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.889+0000 7fe1e2000640 1 -- 172.21.15.86:0/647403318 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffdab47320 con 0x55ffdac55c00 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.889+0000 7fe1e2000640 10 client.16998 handle_command_reply: tid=0 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.889+0000 7fe202051740 1 client.16998 shutdown 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.889+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 msgr2=0x55ffdabd8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.889+0000 7fe202051740 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabd8e70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4015010 tx=0x7fe1e4009b30 comp rx=0 tx=0).stop 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 shutdown_connections 2024-04-25T19:18:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ffdac55c00 0x55ffdac58020 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ffdabdea00 0x55ffdabd93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffdabd4140 0x55ffdabd8e70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 --2- 172.21.15.86:0/647403318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffdabd3820 0x55ffdabd8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 >> 172.21.15.86:0/647403318 conn(0x55ffdabcfd30 msgr2=0x55ffdabce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe1e2000640 0 client.16998 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 shutdown_connections 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 1 -- 172.21.15.86:0/647403318 wait complete. 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.890+0000 7fe202051740 20 client.16998 trim_cache size 0 max 16384 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:18:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe1dc0a3dc0 msgr2=0x7fe1dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ffdabd4dc0 tx=0x7fe1e80046c0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 msgr2=0x7fe1f819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f819ea80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000ec50 tx=0x7fe1f000c6a0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 shutdown_connections 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1f810e620 0x7fe1f819ea80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1f8108f20 0x7fe1f81a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 --2- 172.21.15.86:0/529178337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1f8108560 0x7fe1f81a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 >> 172.21.15.86:0/529178337 conn(0x7fe1f80764e0 msgr2=0x7fe1f8077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 shutdown_connections 2024-04-25T19:18:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:23.891+0000 7fe1ff200640 1 -- 172.21.15.86:0/529178337 wait complete. 2024-04-25T19:18:24.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:24 smithi183 ceph-mon[32344]: pgmap v486: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:24 smithi194 ceph-mon[32740]: pgmap v486: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:24 smithi086 ceph-mon[27783]: pgmap v486: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:26.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:26 smithi183 ceph-mon[32344]: pgmap v487: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-25T19:18:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:26 smithi194 ceph-mon[32740]: pgmap v487: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-25T19:18:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:26 smithi086 ceph-mon[27783]: pgmap v487: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-25T19:18:28.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:28 smithi183 ceph-mon[32344]: pgmap v488: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:18:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:28 smithi194 ceph-mon[32740]: pgmap v488: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:18:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:28 smithi086 ceph-mon[27783]: pgmap v488: 97 pgs: 97 active+clean; 1.8 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:18:30.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:30 smithi183 ceph-mon[32344]: pgmap v489: 97 pgs: 97 active+clean; 2.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:18:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:30 smithi194 ceph-mon[32740]: pgmap v489: 97 pgs: 97 active+clean; 2.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:18:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:30 smithi086 ceph-mon[27783]: pgmap v489: 97 pgs: 97 active+clean; 2.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:18:32.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:32 smithi183 ceph-mon[32344]: pgmap v490: 97 pgs: 97 active+clean; 2.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:32 smithi194 ceph-mon[32740]: pgmap v490: 97 pgs: 97 active+clean; 2.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:32 smithi086 ceph-mon[27783]: pgmap v490: 97 pgs: 97 active+clean; 2.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:33.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:33 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:18:34.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:33 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:18:34.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:33 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:18:34.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:34 smithi194 ceph-mon[32740]: pgmap v491: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:18:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:34 smithi086 ceph-mon[27783]: pgmap v491: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:18:35.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:34 smithi183 ceph-mon[32344]: pgmap v491: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:18:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:35 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:18:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:35 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:18:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:35 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:18:36.068 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:35 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:18:36.068 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:35 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:18:36.068 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:35 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:18:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:35 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:18:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:35 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:18:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:35 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:18:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 -- 172.21.15.86:0/1344759926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8102f10 msgr2=0x7f4cf81032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/1344759926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8102f10 0x7f4cf81032f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce802a480 comp rx=0 tx=0).stop 2024-04-25T19:18:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 -- 172.21.15.86:0/1344759926 shutdown_connections 2024-04-25T19:18:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/1344759926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cf810c330 0x7f4cf810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/1344759926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8103830 0x7f4cf810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/1344759926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8102f10 0x7f4cf81032f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 -- 172.21.15.86:0/1344759926 >> 172.21.15.86:0/1344759926 conn(0x7f4cf80fc770 msgr2=0x7f4cf80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 -- 172.21.15.86:0/1344759926 shutdown_connections 2024-04-25T19:18:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.033+0000 7f4cf7e00640 1 -- 172.21.15.86:0/1344759926 wait complete. 2024-04-25T19:18:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.034+0000 7f4cf7e00640 1 Processor -- start 2024-04-25T19:18:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.034+0000 7f4cf7e00640 1 -- start start 2024-04-25T19:18:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 0x7f4cf8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 0x7f4cf8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cf810c330 0x7f4cf819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8111fa0 con 0x7f4cf8103830 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8111e20 con 0x7f4cf8102f10 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8112120 con 0x7f4cf810c330 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 0x7f4cf8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 0x7f4cf8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 0x7f4cf8198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38528/0 (socket says 172.21.15.86:38528) 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 0x7f4cf8198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55250/0 (socket says 172.21.15.86:55250) 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 -- 172.21.15.86:0/4066009195 learned_addr learned my addr 172.21.15.86:0/4066009195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf7400640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cf810c330 0x7f4cf819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 -- 172.21.15.86:0/4066009195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cf810c330 msgr2=0x7f4cf819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cf810c330 0x7f4cf819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 -- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 msgr2=0x7f4cf8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 0x7f4cf8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6a00640 1 -- 172.21.15.86:0/4066009195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf819d570 con 0x7f4cf8102f10 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.035+0000 7f4cf6000640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 0x7f4cf8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.036+0000 7f4cf6a00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 0x7f4cf8198580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005940 tx=0x7f4ce80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.036+0000 7f4cef400640 1 -- 172.21.15.86:0/4066009195 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce803f030 con 0x7f4cf8102f10 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.036+0000 7f4cef400640 1 -- 172.21.15.86:0/4066009195 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce80044c0 con 0x7f4cf8102f10 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.036+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf819d800 con 0x7f4cf8102f10 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.036+0000 7f4cef400640 1 -- 172.21.15.86:0/4066009195 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8035070 con 0x7f4cf8102f10 2024-04-25T19:18:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.036+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf8100a90 con 0x7f4cf8102f10 2024-04-25T19:18:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.039+0000 7f4cef400640 1 -- 172.21.15.86:0/4066009195 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ce8042030 con 0x7f4cf8102f10 2024-04-25T19:18:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 Processor -- start 2024-04-25T19:18:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 -- start start 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d911cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911d2d0 0x55b1d9127650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d9129f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1d9082900 con 0x55b1d911d2d0 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1d9028ef0 con 0x55b1d911c9b0 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4d007a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b1d907e8c0 con 0x55b1d9127b90 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.040+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d911cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911d2d0 0x55b1d9127650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d911cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38540/0 (socket says 172.21.15.86:38540) 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 -- 172.21.15.86:0/299330654 learned_addr learned my addr 172.21.15.86:0/299330654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf6000640 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d9129f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 -- 172.21.15.86:0/299330654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 msgr2=0x55b1d9129f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d9129f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 -- 172.21.15.86:0/299330654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911d2d0 msgr2=0x55b1d9127650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911d2d0 0x55b1d9127650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 -- 172.21.15.86:0/299330654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d8d95210 con 0x55b1d911c9b0 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cef400640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4cdc0a3d90 0x7f4cdc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.041+0000 7f4cf7400640 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d911cd90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0010dd0 tx=0x7f4cf0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e87c5034d3eca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4cef400640 1 -- 172.21.15.86:0/4066009195 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f4ce80f0b50 con 0x7f4cf8102f10 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4cf6a00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4cdc0a3d90 0x7f4cdc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4ce7e00640 1 -- 172.21.15.86:0/299330654 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf001a6c0 con 0x55b1d911c9b0 2024-04-25T19:18:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4ce7e00640 1 -- 172.21.15.86:0/299330654 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cf001ad10 con 0x55b1d911c9b0 2024-04-25T19:18:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4ce7e00640 1 -- 172.21.15.86:0/299330654 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0019c30 con 0x55b1d911c9b0 2024-04-25T19:18:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4d007a4740 1 -- 172.21.15.86:0/299330654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 msgr2=0x55b1d911cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4cf6a00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4cdc0a3d90 0x7f4cdc0a6250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55b1d911df50 tx=0x7f4ce80f4600 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.042+0000 7f4d007a4740 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d911cd90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0010dd0 tx=0x7f4cf0005f90 comp rx=0 tx=0).stop 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 -- 172.21.15.86:0/299330654 shutdown_connections 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d9129f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911d2d0 0x55b1d9127650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 --2- 172.21.15.86:0/299330654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d911cd90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 -- 172.21.15.86:0/299330654 >> 172.21.15.86:0/299330654 conn(0x55b1d9118ec0 msgr2=0x55b1d91192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 -- 172.21.15.86:0/299330654 shutdown_connections 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 -- 172.21.15.86:0/299330654 wait complete. 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 -- start start 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.043+0000 7f4d007a4740 1 Processor -- start 2024-04-25T19:18:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911c9b0 0x55b1d9124b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9125060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d91255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1d912a7c0 con 0x55b1d9127b90 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1d903c310 con 0x55b1d911d2d0 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b1d912a940 con 0x55b1d911c9b0 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4d007a4740 10 client.0 _ll_register_callbacks cb 0x55b1d900acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d91255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9125060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9125060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38542/0 (socket says 172.21.15.86:38542) 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4cf6000640 1 -- 172.21.15.86:0/3083036671 learned_addr learned my addr 172.21.15.86:0/3083036671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.044+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911c9b0 0x55b1d9124b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4cf6000640 1 -- 172.21.15.86:0/3083036671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911c9b0 msgr2=0x55b1d9124b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911c9b0 0x55b1d9124b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4cf6000640 1 -- 172.21.15.86:0/3083036671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 msgr2=0x55b1d91255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d91255a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4cf6000640 1 -- 172.21.15.86:0/3083036671 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d8e61620 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9125060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000fe00 tx=0x7f4ce0012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc1ec7b2a1566ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3083036671 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003970 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3083036671 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f4ce0003b10 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.045+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf001ad10 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3083036671 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ce0003d10 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 10 client.27101 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1d9136a80 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3083036671 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cf001ad10 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3083036671 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ce00106d0 con 0x55b1d911d2d0 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 10 client.27101 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 10 client.27101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 20 client.27101 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d9137f90 0x55b1d913a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 4 client.27101 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:18:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.046+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1d90968b0 con 0x55b1d9137f90 2024-04-25T19:18:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.047+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d9137f90 0x55b1d913a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.047+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d9137f90 0x55b1d913a380 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d5b05110ff2909 server_cookie=e9596ea5377c2c30 in_seq=0 out_seq=0 2024-04-25T19:18:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.047+0000 7f4ce4c00640 10 client.27101 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3083036671 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1d90968b0 con 0x55b1d9137f90 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4ce4c00640 10 client.27101 handle_command_reply: tid=0 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 client.27101 shutdown 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 msgr2=0x55b1d9125060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9125060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000fe00 tx=0x7f4ce0012820 comp rx=0 tx=0).stop 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 shutdown_connections 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d9137f90 0x55b1d913a380 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d91255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4ce4c00640 0 client.27101 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9125060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3083036671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911c9b0 0x55b1d9124b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.048+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 >> 172.21.15.86:0/3083036671 conn(0x55b1d9118ec0 msgr2=0x55b1d911ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.049+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 shutdown_connections 2024-04-25T19:18:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.049+0000 7f4d007a4740 1 -- 172.21.15.86:0/3083036671 wait complete. 2024-04-25T19:18:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.049+0000 7f4d007a4740 20 client.27101 trim_cache size 0 max 16384 2024-04-25T19:18:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 Processor -- start 2024-04-25T19:18:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 -- start start 2024-04-25T19:18:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911c9b0 0x55b1d9133680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9126a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d913fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81aba00 con 0x55b1d911c9b0 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ad540 con 0x55b1d911d2d0 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.065+0000 7f4d007a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ac0d0 con 0x55b1d9127b90 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9126a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9126a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38544/0 (socket says 172.21.15.86:38544) 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 -- 172.21.15.86:0/3428298132 learned_addr learned my addr 172.21.15.86:0/3428298132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911c9b0 0x55b1d9133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d913fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 -- 172.21.15.86:0/3428298132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 msgr2=0x55b1d913fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d913fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 -- 172.21.15.86:0/3428298132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911c9b0 msgr2=0x55b1d9133680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911c9b0 0x55b1d9133680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.066+0000 7f4cf6000640 1 -- 172.21.15.86:0/3428298132 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d913aa00 con 0x55b1d911d2d0 2024-04-25T19:18:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.067+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9126a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0010c60 tx=0x7f4ce0011f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ecbcbed305a6d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.067+0000 7f4ce7400640 1 -- 172.21.15.86:0/3428298132 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003970 con 0x55b1d911d2d0 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.067+0000 7f4ce7400640 1 -- 172.21.15.86:0/3428298132 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0003b10 con 0x55b1d911d2d0 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.067+0000 7f4ce7400640 1 -- 172.21.15.86:0/3428298132 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00056b0 con 0x55b1d911d2d0 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.067+0000 7f4d007a4740 1 -- 172.21.15.86:0/3428298132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 msgr2=0x55b1d9126a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.067+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9126a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0010c60 tx=0x7f4ce0011f70 comp rx=0 tx=0).stop 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 -- 172.21.15.86:0/3428298132 shutdown_connections 2024-04-25T19:18:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d9127b90 0x55b1d913fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911d2d0 0x55b1d9126a60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3428298132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d911c9b0 0x55b1d9133680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 -- 172.21.15.86:0/3428298132 >> 172.21.15.86:0/3428298132 conn(0x55b1d9118ec0 msgr2=0x55b1d9126350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 -- 172.21.15.86:0/3428298132 shutdown_connections 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 -- 172.21.15.86:0/3428298132 wait complete. 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 -- start start 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.068+0000 7f4d007a4740 1 Processor -- start 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d912d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 0x55b1d912dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d912e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ace70 con 0x55b1d9127b90 2024-04-25T19:18:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81ac7a0 con 0x55b1d911c9b0 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a9a00 con 0x55b1d911d2d0 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4d007a4740 10 client.0 _ll_register_callbacks cb 0x7f4ce000fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d912d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 0x55b1d912dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4cf6000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 0x55b1d912dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33862/0 (socket says 172.21.15.86:33862) 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4cf6000640 1 -- 172.21.15.86:0/3410352781 learned_addr learned my addr 172.21.15.86:0/3410352781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.069+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d912e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf7400640 1 -- 172.21.15.86:0/3410352781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 msgr2=0x55b1d912dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 0x55b1d912dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf7400640 1 -- 172.21.15.86:0/3410352781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 msgr2=0x55b1d912e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d912e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf7400640 1 -- 172.21.15.86:0/3410352781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d91a5fc0 con 0x55b1d911c9b0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 0x55b1d912dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4cf7400640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d912d850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0012010 tx=0x7f4cf0028040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f309146266121767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3410352781 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0019900 con 0x55b1d911c9b0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3410352781 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f4cf0019aa0 con 0x55b1d911c9b0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.070+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce0003b10 con 0x55b1d911c9b0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3410352781 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cf0019ca0 con 0x55b1d911c9b0 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 10 client.27107 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1d91a7310 con 0x55b1d911c9b0 2024-04-25T19:18:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3410352781 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0003b10 con 0x55b1d911c9b0 2024-04-25T19:18:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3410352781 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4cf00324e0 con 0x55b1d911c9b0 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 10 client.27107 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 10 client.27107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 20 client.27107 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d91a78a0 0x55b1d91a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 4 client.27107 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.071+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1d9090320 con 0x55b1d91a78a0 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.072+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d91a78a0 0x55b1d91a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.072+0000 7f4cf6000640 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d91a78a0 0x55b1d91a9c90 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b425fd2d038bfbfc server_cookie=227ab9a6c47a844b in_seq=0 out_seq=0 2024-04-25T19:18:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.072+0000 7f4ce4c00640 10 client.27107 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4ce4c00640 1 -- 172.21.15.86:0/3410352781 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1d9090320 con 0x55b1d91a78a0 2024-04-25T19:18:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4ce4c00640 10 client.27107 handle_command_reply: tid=0 2024-04-25T19:18:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4d007a4740 1 client.27107 shutdown 2024-04-25T19:18:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 msgr2=0x55b1d912d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d912d850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0012010 tx=0x7f4cf0028040 comp rx=0 tx=0).stop 2024-04-25T19:18:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 shutdown_connections 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.086+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55b1d91a78a0 0x55b1d91a9c90 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1d9127b90 0x55b1d912e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b1d911d2d0 0x55b1d912dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4ce4c00640 0 client.27107 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 1 --2- 172.21.15.86:0/3410352781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1d911c9b0 0x55b1d912d850 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 >> 172.21.15.86:0/3410352781 conn(0x55b1d9118ec0 msgr2=0x55b1d9126be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 shutdown_connections 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 1 -- 172.21.15.86:0/3410352781 wait complete. 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4d007a4740 20 client.27107 trim_cache size 0 max 16384 2024-04-25T19:18:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:18:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:18:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:18:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:18:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4cdc0a3d90 msgr2=0x7f4cdc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.087+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4cdc0a3d90 0x7f4cdc0a6250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55b1d911df50 tx=0x7f4ce80f4600 comp rx=0 tx=0).stop 2024-04-25T19:18:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 msgr2=0x7f4cf8198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 0x7f4cf8198580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8005940 tx=0x7f4ce80040c0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 shutdown_connections 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f4cdc0a3d90 0x7f4cdc0a6250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cf810c330 0x7f4cf819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8103830 0x7f4cf8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 --2- 172.21.15.86:0/4066009195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cf8102f10 0x7f4cf8198580 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 >> 172.21.15.86:0/4066009195 conn(0x7f4cf80fc770 msgr2=0x7f4cf80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 shutdown_connections 2024-04-25T19:18:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:37.088+0000 7f4cf7e00640 1 -- 172.21.15.86:0/4066009195 wait complete. 2024-04-25T19:18:37.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:36 smithi183 ceph-mon[32344]: pgmap v492: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:37.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:36 smithi194 ceph-mon[32740]: pgmap v492: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:37.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:36 smithi086 ceph-mon[27783]: pgmap v492: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:38.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:37 smithi183 ceph-mon[32344]: pgmap v493: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:38.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:37 smithi194 ceph-mon[32740]: pgmap v493: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:38.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:37 smithi086 ceph-mon[27783]: pgmap v493: 97 pgs: 97 active+clean; 2.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:40 smithi086 ceph-mon[27783]: pgmap v494: 97 pgs: 97 active+clean; 2.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T19:18:40.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:40 smithi183 ceph-mon[32344]: pgmap v494: 97 pgs: 97 active+clean; 2.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T19:18:40.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:40 smithi194 ceph-mon[32740]: pgmap v494: 97 pgs: 97 active+clean; 2.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T19:18:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:42 smithi086 ceph-mon[27783]: pgmap v495: 97 pgs: 97 active+clean; 2.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:42 smithi183 ceph-mon[32344]: pgmap v495: 97 pgs: 97 active+clean; 2.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:42.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:42 smithi194 ceph-mon[32740]: pgmap v495: 97 pgs: 97 active+clean; 2.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T19:18:44.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:44 smithi086 ceph-mon[27783]: pgmap v496: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:18:44.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:44 smithi183 ceph-mon[32344]: pgmap v496: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:18:44.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:44 smithi194 ceph-mon[32740]: pgmap v496: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:18:46.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:46 smithi086 ceph-mon[27783]: pgmap v497: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:46.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:46 smithi183 ceph-mon[32344]: pgmap v497: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:46.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:46 smithi194 ceph-mon[32740]: pgmap v497: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:48 smithi183 ceph-mon[32344]: pgmap v498: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:48.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:48 smithi194 ceph-mon[32740]: pgmap v498: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:48.756 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:48 smithi086 ceph-mon[27783]: pgmap v498: 97 pgs: 97 active+clean; 2.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:18:50.140 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-25T19:18:50.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:18:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.230+0000 7f10eb200640 1 -- 172.21.15.86:0/1156769963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec102f70 msgr2=0x7f10ec103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.230+0000 7f10eb200640 1 --2- 172.21.15.86:0/1156769963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec102f70 0x7f10ec103350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009ca0 tx=0x7f10dc02a950 comp rx=0 tx=0).stop 2024-04-25T19:18:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 -- 172.21.15.86:0/1156769963 shutdown_connections 2024-04-25T19:18:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 --2- 172.21.15.86:0/1156769963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec10c340 0x7f10ec10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 --2- 172.21.15.86:0/1156769963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec103890 0x7f10ec10bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 --2- 172.21.15.86:0/1156769963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec102f70 0x7f10ec103350 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 -- 172.21.15.86:0/1156769963 >> 172.21.15.86:0/1156769963 conn(0x7f10ec0fc770 msgr2=0x7f10ec0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 -- 172.21.15.86:0/1156769963 shutdown_connections 2024-04-25T19:18:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.231+0000 7f10eb200640 1 -- 172.21.15.86:0/1156769963 wait complete. 2024-04-25T19:18:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 Processor -- start 2024-04-25T19:18:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 -- start start 2024-04-25T19:18:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec102f70 0x7f10ec1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 0x7f10ec1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec10c340 0x7f10ec19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec111fc0 con 0x7f10ec103890 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec111e40 con 0x7f10ec10c340 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10eb200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec112140 con 0x7f10ec102f70 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 0x7f10ec1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 0x7f10ec1a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60966/0 (socket says 172.21.15.86:60966) 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10e9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec102f70 0x7f10ec1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.232+0000 7f10e1400640 1 -- 172.21.15.86:0/2714573596 learned_addr learned my addr 172.21.15.86:0/2714573596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10ea800640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec10c340 0x7f10ec19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e1400640 1 -- 172.21.15.86:0/2714573596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec102f70 msgr2=0x7f10ec1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e1400640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec102f70 0x7f10ec1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e1400640 1 -- 172.21.15.86:0/2714573596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec10c340 msgr2=0x7f10ec19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e1400640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec10c340 0x7f10ec19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e1400640 1 -- 172.21.15.86:0/2714573596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec19ad20 con 0x7f10ec103890 2024-04-25T19:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e9e00640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec102f70 0x7f10ec1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.233+0000 7f10e1400640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 0x7f10ec1a0930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f10d400e9f0 tx=0x7f10d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.234+0000 7f10e3400640 1 -- 172.21.15.86:0/2714573596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d400cd80 con 0x7f10ec103890 2024-04-25T19:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.234+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec19b010 con 0x7f10ec103890 2024-04-25T19:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.234+0000 7f10e3400640 1 -- 172.21.15.86:0/2714573596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d40108e0 con 0x7f10ec103890 2024-04-25T19:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.234+0000 7f10e3400640 1 -- 172.21.15.86:0/2714573596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4018980 con 0x7f10ec103890 2024-04-25T19:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.234+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1a72b0 con 0x7f10ec103890 2024-04-25T19:18:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 Processor -- start 2024-04-25T19:18:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 -- start start 2024-04-25T19:18:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b5259ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10e3400640 1 -- 172.21.15.86:0/2714573596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10d4018b20 con 0x7f10ec103890 2024-04-25T19:18:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b52504900 con 0x558b5259e9b0 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b524aaef0 con 0x558b525a9b90 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.238+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558b525008c0 con 0x558b5259f2d0 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b5259ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b5259ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60968/0 (socket says 172.21.15.86:60968) 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10ea800640 1 -- 172.21.15.86:0/3177780124 learned_addr learned my addr 172.21.15.86:0/3177780124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e1400640 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e9e00640 1 -- 172.21.15.86:0/3177780124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 msgr2=0x558b525abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e9e00640 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e9e00640 1 -- 172.21.15.86:0/3177780124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 msgr2=0x558b5259ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e9e00640 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b5259ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.239+0000 7f10e9e00640 1 -- 172.21.15.86:0/3177780124 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b52217210 con 0x558b5259f2d0 2024-04-25T19:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.240+0000 7f10e9e00640 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a9650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10dc02ae70 tx=0x7f10dc005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4eea3244a901d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.240+0000 7f10db400640 1 -- 172.21.15.86:0/3177780124 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc048070 con 0x558b5259f2d0 2024-04-25T19:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.240+0000 7f10db400640 1 -- 172.21.15.86:0/3177780124 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10dc003e20 con 0x558b5259f2d0 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.240+0000 7f10db400640 1 -- 172.21.15.86:0/3177780124 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc034040 con 0x558b5259f2d0 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 -- 172.21.15.86:0/3177780124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 msgr2=0x558b525a9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a9650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10dc02ae70 tx=0x7f10dc005ef0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 -- 172.21.15.86:0/3177780124 shutdown_connections 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a9650 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/3177780124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b5259ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 -- 172.21.15.86:0/3177780124 >> 172.21.15.86:0/3177780124 conn(0x558b5259aec0 msgr2=0x558b5259b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 -- 172.21.15.86:0/3177780124 shutdown_connections 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 -- 172.21.15.86:0/3177780124 wait complete. 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10e3400640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f10d00a3ca0 0x7f10d00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10e3400640 1 -- 172.21.15.86:0/2714573596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f10d40d0b30 con 0x7f10ec103890 2024-04-25T19:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10e9e00640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f10d00a3ca0 0x7f10d00a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.241+0000 7f10f3f9a740 1 -- start start 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 Processor -- start 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10e9e00640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f10d00a3ca0 0x7f10d00a6160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005e30 tx=0x7f10dc004ab0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b525ac760 con 0x558b5259f2d0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b524be310 con 0x558b525a9b90 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558b525ac8e0 con 0x558b5259e9b0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.242+0000 7f10f3f9a740 10 client.0 _ll_register_callbacks cb 0x558b5248ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10e9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:54584/0 (socket says 172.21.15.86:54584) 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 -- 172.21.15.86:0/4088678783 learned_addr learned my addr 172.21.15.86:0/4088678783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10e9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525a6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:49396/0 (socket says 172.21.15.86:49396) 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 -- 172.21.15.86:0/4088678783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 msgr2=0x558b525a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 -- 172.21.15.86:0/4088678783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 msgr2=0x558b525a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 -- 172.21.15.86:0/4088678783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b522e3620 con 0x558b525a9b90 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10e1400640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10e9e00640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10ea800640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a75d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f10e4012700 tx=0x7f10e4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b39a33e3d385eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.243+0000 7f10cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10cfe00640 1 -- 172.21.15.86:0/4088678783 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4004050 con 0x558b525a9b90 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10dc003e20 con 0x558b525a9b90 2024-04-25T19:18:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10cfe00640 1 -- 172.21.15.86:0/4088678783 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f10e40041f0 con 0x558b525a9b90 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10cfe00640 1 -- 172.21.15.86:0/4088678783 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10e40043f0 con 0x558b525a9b90 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10f3f9a740 10 client.27119 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b525b8aa0 con 0x558b525a9b90 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10cfe00640 1 -- 172.21.15.86:0/4088678783 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10dc003e20 con 0x558b525a9b90 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10cfe00640 1 -- 172.21.15.86:0/4088678783 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f10e4003b50 con 0x558b525a9b90 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10f3f9a740 10 client.27119 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.244+0000 7f10f3f9a740 10 client.27119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.245+0000 7f10f3f9a740 20 client.27119 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.245+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b525b9f60 0x558b525bc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.245+0000 7f10f3f9a740 4 client.27119 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.245+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b525188b0 con 0x558b525b9f60 2024-04-25T19:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.245+0000 7f10e9e00640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b525b9f60 0x558b525bc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.246+0000 7f10e9e00640 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b525b9f60 0x558b525bc370 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f479d97605840b04 server_cookie=cf8dee379ff0a7ee in_seq=0 out_seq=0 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.246+0000 7f10cfe00640 10 client.27119 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.246+0000 7f10cfe00640 1 -- 172.21.15.86:0/4088678783 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b525188b0 con 0x558b525b9f60 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.246+0000 7f10cfe00640 10 client.27119 handle_command_reply: tid=0 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 client.27119 shutdown 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 msgr2=0x558b525a75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a75d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f10e4012700 tx=0x7f10e4012bc0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 shutdown_connections 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b525b9f60 0x558b525bc370 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a75d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10cfe00640 0 client.27119 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/4088678783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 >> 172.21.15.86:0/4088678783 conn(0x558b5259aec0 msgr2=0x558b525a0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 shutdown_connections 2024-04-25T19:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 1 -- 172.21.15.86:0/4088678783 wait complete. 2024-04-25T19:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.247+0000 7f10f3f9a740 20 client.27119 trim_cache size 0 max 16384 2024-04-25T19:18:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.263+0000 7f10f3f9a740 1 Processor -- start 2024-04-25T19:18:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.263+0000 7f10f3f9a740 1 -- start start 2024-04-25T19:18:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b525b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1afe10 con 0x558b5259e9b0 2024-04-25T19:18:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1b1950 con 0x558b525a9b90 2024-04-25T19:18:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1b04e0 con 0x558b5259f2d0 2024-04-25T19:18:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b525b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b525b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60976/0 (socket says 172.21.15.86:60976) 2024-04-25T19:18:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10ea800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 -- 172.21.15.86:0/2342960907 learned_addr learned my addr 172.21.15.86:0/2342960907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e1400640 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 -- 172.21.15.86:0/2342960907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 msgr2=0x558b525a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 -- 172.21.15.86:0/2342960907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 msgr2=0x558b525a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10e9e00640 1 -- 172.21.15.86:0/2342960907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b525abf80 con 0x558b5259e9b0 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.264+0000 7f10ea800640 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.265+0000 7f10e9e00640 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b525b5620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009980 tx=0x7f10dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d1d29c4f705f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.265+0000 7f10daa00640 1 -- 172.21.15.86:0/2342960907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc048070 con 0x558b5259e9b0 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.265+0000 7f10daa00640 1 -- 172.21.15.86:0/2342960907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10dc003dc0 con 0x558b5259e9b0 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.265+0000 7f10daa00640 1 -- 172.21.15.86:0/2342960907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc034070 con 0x558b5259e9b0 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 -- 172.21.15.86:0/2342960907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 msgr2=0x558b525b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b525b5620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009980 tx=0x7f10dc004a40 comp rx=0 tx=0).stop 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 -- 172.21.15.86:0/2342960907 shutdown_connections 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259f2d0 0x558b525a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/2342960907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259e9b0 0x558b525b5620 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 -- 172.21.15.86:0/2342960907 >> 172.21.15.86:0/2342960907 conn(0x558b5259aec0 msgr2=0x558b525a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 -- 172.21.15.86:0/2342960907 shutdown_connections 2024-04-25T19:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 -- 172.21.15.86:0/2342960907 wait complete. 2024-04-25T19:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.266+0000 7f10f3f9a740 1 -- start start 2024-04-25T19:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 Processor -- start 2024-04-25T19:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1b1280 con 0x558b5259f2d0 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1b0bb0 con 0x558b525a9b90 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec1ade10 con 0x558b5259e9b0 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.267+0000 7f10f3f9a740 10 client.0 _ll_register_callbacks cb 0x7f10e4009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60990/0 (socket says 172.21.15.86:60990) 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 -- 172.21.15.86:0/1127348242 learned_addr learned my addr 172.21.15.86:0/1127348242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e9e00640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10ea800640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 -- 172.21.15.86:0/1127348242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 msgr2=0x558b525b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 -- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 msgr2=0x558b525af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 -- 172.21.15.86:0/1127348242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b525aff60 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10e1400640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525af220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f10d409c7b0 tx=0x7f10d400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8da5aa88b86f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.268+0000 7f10cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10cfe00640 1 -- 172.21.15.86:0/1127348242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d40bbdc0 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10cfe00640 1 -- 172.21.15.86:0/1127348242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f10d4010430 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10dc003dc0 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10cfe00640 1 -- 172.21.15.86:0/1127348242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10d40a0db0 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 10 client.17034 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b52620460 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10cfe00640 1 -- 172.21.15.86:0/1127348242 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d40b7080 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10cfe00640 1 -- 172.21.15.86:0/1127348242 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f10d409b970 con 0x558b5259f2d0 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 10 client.17034 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 20 client.17034 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b52620be0 0x558b52623000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 4 client.17034 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.269+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b52512320 con 0x558b52620be0 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.270+0000 7f10e9e00640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b52620be0 0x558b52623000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.270+0000 7f10e9e00640 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b52620be0 0x558b52623000 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76a164ace16a29a server_cookie=65735f255d0b2000 in_seq=0 out_seq=0 2024-04-25T19:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.270+0000 7f10cfe00640 10 client.17034 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10cfe00640 1 -- 172.21.15.86:0/1127348242 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b52512320 con 0x558b52620be0 2024-04-25T19:18:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10cfe00640 10 client.17034 handle_command_reply: tid=0 2024-04-25T19:18:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 client.17034 shutdown 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 msgr2=0x558b525af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525af220 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f10d409c7b0 tx=0x7f10d400cf20 comp rx=0 tx=0).stop 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 shutdown_connections 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x558b52620be0 0x558b52623000 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b525a9b90 0x558b525af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b5259f2d0 0x558b525af220 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 --2- 172.21.15.86:0/1127348242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b5259e9b0 0x558b525b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10cfe00640 0 client.17034 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 >> 172.21.15.86:0/1127348242 conn(0x558b5259aec0 msgr2=0x558b5259a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 shutdown_connections 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 1 -- 172.21.15.86:0/1127348242 wait complete. 2024-04-25T19:18:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.286+0000 7f10f3f9a740 20 client.17034 trim_cache size 0 max 16384 2024-04-25T19:18:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:18:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:18:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:18:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f10d00a3ca0 msgr2=0x7f10d00a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f10d00a3ca0 0x7f10d00a6160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005e30 tx=0x7f10dc004ab0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 msgr2=0x7f10ec1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 0x7f10ec1a0930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f10d400e9f0 tx=0x7f10d400eeb0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 shutdown_connections 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f10d00a3ca0 0x7f10d00a6160 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10ec10c340 0x7f10ec19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10ec103890 0x7f10ec1a0930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 --2- 172.21.15.86:0/2714573596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10ec102f70 0x7f10ec1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 >> 172.21.15.86:0/2714573596 conn(0x7f10ec0fc770 msgr2=0x7f10ec101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 shutdown_connections 2024-04-25T19:18:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:18:50.287+0000 7f10eb200640 1 -- 172.21.15.86:0/2714573596 wait complete. 2024-04-25T19:18:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:18:50.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:50 smithi183 ceph-mon[32344]: pgmap v499: 97 pgs: 97 active+clean; 2.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:18:50.705 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:50 smithi086 ceph-mon[27783]: pgmap v499: 97 pgs: 97 active+clean; 2.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:18:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:50 smithi194 ceph-mon[32740]: pgmap v499: 97 pgs: 97 active+clean; 2.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T19:18:52.598 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:52 smithi086 ceph-mon[27783]: pgmap v500: 97 pgs: 97 active+clean; 2.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:18:52.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:52 smithi183 ceph-mon[32344]: pgmap v500: 97 pgs: 97 active+clean; 2.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:18:52.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:52 smithi194 ceph-mon[32740]: pgmap v500: 97 pgs: 97 active+clean; 2.8 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T19:18:52.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.931+0000 7f9a28800700 1 -- 172.21.15.86:0/1927510903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a20109cc0 msgr2=0x7f9a2010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.931+0000 7f9a28800700 1 --2- 172.21.15.86:0/1927510903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a20109cc0 0x7f9a2010dcc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9a100099c0 tx=0x7f9a10009cd0 comp rx=0 tx=0).stop 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 -- 172.21.15.86:0/1927510903 shutdown_connections 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 --2- 172.21.15.86:0/1927510903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2010e3a0 0x7f9a20115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 --2- 172.21.15.86:0/1927510903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a201096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 --2- 172.21.15.86:0/1927510903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a20109cc0 0x7f9a2010dcc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 -- 172.21.15.86:0/1927510903 >> 172.21.15.86:0/1927510903 conn(0x7f9a20076210 msgr2=0x7f9a20078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 -- 172.21.15.86:0/1927510903 shutdown_connections 2024-04-25T19:18:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.932+0000 7f9a28800700 1 -- 172.21.15.86:0/1927510903 wait complete. 2024-04-25T19:18:52.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.933+0000 7f9a28800700 1 Processor -- start 2024-04-25T19:18:52.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.933+0000 7f9a28800700 1 -- start start 2024-04-25T19:18:52.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a28800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a2019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a28800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a20109cc0 0x7f9a2019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a28800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a2010e3a0 0x7f9a201a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a28800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20119210 con 0x7f9a20109310 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a28800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20119090 con 0x7f9a20109cc0 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a28800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20119390 con 0x7f9a2010e3a0 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a2019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a2019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60998/0 (socket says 172.21.15.86:60998) 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 -- 172.21.15.86:0/729502811 learned_addr learned my addr 172.21.15.86:0/729502811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a26800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a2010e3a0 0x7f9a201a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:52.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25400700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a20109cc0 0x7f9a2019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 -- 172.21.15.86:0/729502811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a2010e3a0 msgr2=0x7f9a201a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a2010e3a0 0x7f9a201a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 -- 172.21.15.86:0/729502811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a20109cc0 msgr2=0x7f9a2019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a20109cc0 0x7f9a2019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.934+0000 7f9a25e00700 1 -- 172.21.15.86:0/729502811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201a4760 con 0x7f9a20109310 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.935+0000 7f9a25e00700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a2019f7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400eb30 tx=0x7f9a1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:52.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.935+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400cbd0 con 0x7f9a20109310 2024-04-25T19:18:52.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.935+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201a4a50 con 0x7f9a20109310 2024-04-25T19:18:52.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.935+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a140044d0 con 0x7f9a20109310 2024-04-25T19:18:52.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.935+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14003df0 con 0x7f9a20109310 2024-04-25T19:18:52.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.935+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201ac2d0 con 0x7f9a20109310 2024-04-25T19:18:52.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.936+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a04005330 con 0x7f9a20109310 2024-04-25T19:18:52.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.939+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a14004000 con 0x7f9a20109310 2024-04-25T19:18:52.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.940+0000 7f9a1aa00700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9a080a3e30 0x7f9a080a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:52.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.940+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f9a140d0ae0 con 0x7f9a20109310 2024-04-25T19:18:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.941+0000 7f9a25400700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9a080a3e30 0x7f9a080a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.941+0000 7f9a25400700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9a080a3e30 0x7f9a080a62f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a10005fb0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:52.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:52.943+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a140042f0 con 0x7f9a20109310 2024-04-25T19:18:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.222+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a04005620 con 0x7f9a20109310 2024-04-25T19:18:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.224+0000 7f9a1aa00700 1 -- 172.21.15.86:0/729502811 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f9a1409d140 con 0x7f9a20109310 2024-04-25T19:18:53.227 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:18:53.227 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:18:53.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9a080a3e30 msgr2=0x7f9a080a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9a080a3e30 0x7f9a080a62f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a10005fb0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 msgr2=0x7f9a2019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a2019f7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400eb30 tx=0x7f9a1400ee40 comp rx=0 tx=0).stop 2024-04-25T19:18:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 shutdown_connections 2024-04-25T19:18:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9a080a3e30 0x7f9a080a62f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a2010e3a0 0x7f9a201a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a20109310 0x7f9a2019f7a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 --2- 172.21.15.86:0/729502811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a20109cc0 0x7f9a2019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 >> 172.21.15.86:0/729502811 conn(0x7f9a20076210 msgr2=0x7f9a200775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 shutdown_connections 2024-04-25T19:18:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:53.228+0000 7f9a28800700 1 -- 172.21.15.86:0/729502811 wait complete. 2024-04-25T19:18:53.232 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:18:53.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:53 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/729502811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:18:53.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:53 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/729502811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:18:53.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:53 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/729502811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:18:54.108 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-25T19:18:54.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-25T19:18:54.547 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:18:54.632 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:54 smithi194 ceph-mon[32740]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:18:54.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:54 smithi183 ceph-mon[32344]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:18:54.698 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:54 smithi086 ceph-mon[27783]: pgmap v501: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:18:56.466 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:56 smithi086 ceph-mon[27783]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:18:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:56 smithi183 ceph-mon[32344]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:18:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:56 smithi194 ceph-mon[32740]: pgmap v502: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:18:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 -- 172.21.15.86:0/3345928244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9410f4a0 msgr2=0x7f6d94111890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 --2- 172.21.15.86:0/3345928244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9410f4a0 0x7f6d94111890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b0a0 tx=0x7f6d8c00b3b0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 -- 172.21.15.86:0/3345928244 shutdown_connections 2024-04-25T19:18:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 --2- 172.21.15.86:0/3345928244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9410f4a0 0x7f6d94111890 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 --2- 172.21.15.86:0/3345928244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d94074c70 0x7f6d94075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 --2- 172.21.15.86:0/3345928244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94075590 0x7f6d9410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 -- 172.21.15.86:0/3345928244 >> 172.21.15.86:0/3345928244 conn(0x7f6d94079b10 msgr2=0x7f6d94079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 -- 172.21.15.86:0/3345928244 shutdown_connections 2024-04-25T19:18:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.768+0000 7f6d99400700 1 -- 172.21.15.86:0/3345928244 wait complete. 2024-04-25T19:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.769+0000 7f6d99400700 1 Processor -- start 2024-04-25T19:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.769+0000 7f6d99400700 1 -- start start 2024-04-25T19:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d99400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 0x7f6d9419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d99400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d94075590 0x7f6d9419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d99400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d9410f4a0 0x7f6d9419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d99400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94114e10 con 0x7f6d94075590 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d99400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94114c90 con 0x7f6d9410f4a0 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d99400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d94114f90 con 0x7f6d94074c70 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d94075590 0x7f6d9419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d92800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 0x7f6d9419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d92800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 0x7f6d9419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:49442/0 (socket says 172.21.15.86:49442) 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d92800700 1 -- 172.21.15.86:0/2824906976 learned_addr learned my addr 172.21.15.86:0/2824906976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.770+0000 7f6d93200700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d9410f4a0 0x7f6d9419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d92800700 1 -- 172.21.15.86:0/2824906976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d9410f4a0 msgr2=0x7f6d9419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d92800700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d9410f4a0 0x7f6d9419fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d92800700 1 -- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d94075590 msgr2=0x7f6d9419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d92800700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d94075590 0x7f6d9419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d92800700 1 -- 172.21.15.86:0/2824906976 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d941a01f0 con 0x7f6d94074c70 2024-04-25T19:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d89e00700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d94075590 0x7f6d9419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d93200700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d9410f4a0 0x7f6d9419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d92800700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 0x7f6d9419b1e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400eb30 tx=0x7f6d8400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d8be00700 1 -- 172.21.15.86:0/2824906976 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400cbd0 con 0x7f6d94074c70 2024-04-25T19:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d8be00700 1 -- 172.21.15.86:0/2824906976 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d840044d0 con 0x7f6d94074c70 2024-04-25T19:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d941a04e0 con 0x7f6d94074c70 2024-04-25T19:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.771+0000 7f6d8be00700 1 -- 172.21.15.86:0/2824906976 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84003e30 con 0x7f6d94074c70 2024-04-25T19:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.772+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941a7d60 con 0x7f6d94074c70 2024-04-25T19:18:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.775+0000 7f6d8be00700 1 -- 172.21.15.86:0/2824906976 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d84004000 con 0x7f6d94074c70 2024-04-25T19:18:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.776+0000 7f6da3d8ab80 1 Processor -- start 2024-04-25T19:18:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.776+0000 7f6d8be00700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d780a3e50 0x7f6d780a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.776+0000 7f6da3d8ab80 1 -- start start 2024-04-25T19:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.776+0000 7f6d8be00700 1 -- 172.21.15.86:0/2824906976 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f6d840d0c30 con 0x7f6d94074c70 2024-04-25T19:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 0x55d78f85e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d93200700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d780a3e50 0x7f6d780a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f869020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d78f7b1020 con 0x55d78f869560 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d78f7578c0 con 0x55d78f85e6f0 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d78f7acfe0 con 0x55d78f85ddd0 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 0x55d78f85e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 0x55d78f85e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:49450/0 (socket says 172.21.15.86:49450) 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 -- 172.21.15.86:0/1333625476 learned_addr learned my addr 172.21.15.86:0/1333625476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d89e00700 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f869020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d93200700 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 -- 172.21.15.86:0/1333625476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 msgr2=0x55d78f869020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f869020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 -- 172.21.15.86:0/1333625476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 msgr2=0x55d78f86b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.777+0000 7f6d92800700 1 -- 172.21.15.86:0/1333625476 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78f74d810 con 0x55d78f85ddd0 2024-04-25T19:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.778+0000 7f6d92800700 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 0x55d78f85e1b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6d840a5240 tx=0x7f6d84098da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972fe08033bb0d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.778+0000 7f6d93200700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d780a3e50 0x7f6d780a6310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55d78f85f7a0 tx=0x7f6d8c00bf60 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.778+0000 7f6d83e00700 1 -- 172.21.15.86:0/1333625476 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8409c460 con 0x55d78f85ddd0 2024-04-25T19:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.778+0000 7f6d83e00700 1 -- 172.21.15.86:0/1333625476 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d840042f0 con 0x55d78f85ddd0 2024-04-25T19:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.778+0000 7f6d83e00700 1 -- 172.21.15.86:0/1333625476 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840bd070 con 0x55d78f85ddd0 2024-04-25T19:18:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1333625476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 msgr2=0x55d78f85e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 0x55d78f85e1b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6d840a5240 tx=0x7f6d84098da0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1333625476 shutdown_connections 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85ddd0 0x55d78f85e1b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1333625476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f869020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1333625476 >> 172.21.15.86:0/1333625476 conn(0x55d78f78d0e0 msgr2=0x55d78f85a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1333625476 shutdown_connections 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.779+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1333625476 wait complete. 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 -- start start 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 Processor -- start 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f868cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85e6f0 0x55d78f865fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f866510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d78f86c050 con 0x55d78f85e6f0 2024-04-25T19:18:56.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d78f76a9f0 con 0x55d78f85ddd0 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.780+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d78f86c1d0 con 0x55d78f869560 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6da3d8ab80 10 client.0 _ll_register_callbacks cb 0x55d78f7498b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85e6f0 0x55d78f865fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d93200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f866510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85e6f0 0x55d78f865fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32814/0 (socket says 172.21.15.86:32814) 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d89e00700 1 -- 172.21.15.86:0/2623871156 learned_addr learned my addr 172.21.15.86:0/2623871156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d92800700 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f868cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d93200700 1 -- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 msgr2=0x55d78f868cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d93200700 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f868cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d93200700 1 -- 172.21.15.86:0/2623871156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85e6f0 msgr2=0x55d78f865fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d93200700 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85e6f0 0x55d78f865fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.781+0000 7f6d93200700 1 -- 172.21.15.86:0/2623871156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78f6fea60 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6d93200700 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f866510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00d010 tx=0x7f6d8c00b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e1392a115cb7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6d80c00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6d80c00700 1 -- 172.21.15.86:0/2623871156 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c02f030 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d840042f0 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6d80c00700 1 -- 172.21.15.86:0/2623871156 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f6d8c007db0 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6d80c00700 1 -- 172.21.15.86:0/2623871156 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d8c039050 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6da3d8ab80 10 client.27142 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.782+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d78f747c80 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6d80c00700 1 -- 172.21.15.86:0/2623871156 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d840042f0 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6d80c00700 1 -- 172.21.15.86:0/2623871156 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6d8c028070 con 0x55d78f869560 2024-04-25T19:18:56.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6da3d8ab80 10 client.27142 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6da3d8ab80 10 client.27142 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6da3d8ab80 20 client.27142 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f878d60 0x55d78f87b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6da3d8ab80 4 client.27142 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.783+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d78f7be810 con 0x55d78f878d60 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.784+0000 7f6d92800700 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f878d60 0x55d78f87b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.784+0000 7f6d92800700 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f878d60 0x55d78f87b170 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ca60dfe6156970 server_cookie=a5ab827da75ca7e7 in_seq=0 out_seq=0 2024-04-25T19:18:56.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.784+0000 7f6d80c00700 10 client.27142 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:56.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6d80c00700 1 -- 172.21.15.86:0/2623871156 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d78f7be810 con 0x55d78f878d60 2024-04-25T19:18:56.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6d80c00700 10 client.27142 handle_command_reply: tid=0 2024-04-25T19:18:56.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 client.27142 shutdown 2024-04-25T19:18:56.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 msgr2=0x55d78f866510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f866510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00d010 tx=0x7f6d8c00b720 comp rx=0 tx=0).stop 2024-04-25T19:18:56.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 shutdown_connections 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f878d60 0x55d78f87b170 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f866510 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f868cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2623871156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85e6f0 0x55d78f865fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 >> 172.21.15.86:0/2623871156 conn(0x55d78f78d0e0 msgr2=0x55d78f859ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.785+0000 7f6d80c00700 0 client.27142 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.786+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 shutdown_connections 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.786+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2623871156 wait complete. 2024-04-25T19:18:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.786+0000 7f6da3d8ab80 20 client.27142 trim_cache size 0 max 16384 2024-04-25T19:18:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 Processor -- start 2024-04-25T19:18:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 -- start start 2024-04-25T19:18:56.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 0x55d78f864f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f861650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f874f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941af660 con 0x55d78f85ddd0 2024-04-25T19:18:56.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941b11a0 con 0x55d78f85e6f0 2024-04-25T19:18:56.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.807+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941afd30 con 0x55d78f869560 2024-04-25T19:18:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 0x55d78f864f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d93200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f874f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 0x55d78f864f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32818/0 (socket says 172.21.15.86:32818) 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d89e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f861650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 -- 172.21.15.86:0/1967042944 learned_addr learned my addr 172.21.15.86:0/1967042944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 -- 172.21.15.86:0/1967042944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 msgr2=0x55d78f874f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f874f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 -- 172.21.15.86:0/1967042944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 msgr2=0x55d78f861650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f861650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d92800700 1 -- 172.21.15.86:0/1967042944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78f85fc40 con 0x55d78f85ddd0 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d93200700 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f874f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.808+0000 7f6d89e00700 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f861650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.809+0000 7f6d92800700 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 0x55d78f864f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6d8409ca90 tx=0x7f6d840d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3f33a6adb42252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.809+0000 7f6d83400700 1 -- 172.21.15.86:0/1967042944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840a0eb0 con 0x55d78f85ddd0 2024-04-25T19:18:56.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.809+0000 7f6d83400700 1 -- 172.21.15.86:0/1967042944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d8409d200 con 0x55d78f85ddd0 2024-04-25T19:18:56.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.809+0000 7f6d83400700 1 -- 172.21.15.86:0/1967042944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8409cd60 con 0x55d78f85ddd0 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1967042944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 msgr2=0x55d78f864f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 0x55d78f864f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6d8409ca90 tx=0x7f6d840d0040 comp rx=0 tx=0).stop 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1967042944 shutdown_connections 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f869560 0x55d78f874f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f85ddd0 0x55d78f864f30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/1967042944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85e6f0 0x55d78f861650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1967042944 >> 172.21.15.86:0/1967042944 conn(0x55d78f78d0e0 msgr2=0x55d78f860f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1967042944 shutdown_connections 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/1967042944 wait complete. 2024-04-25T19:18:56.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.810+0000 7f6da3d8ab80 1 -- start start 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 Processor -- start 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f86c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85e6f0 0x55d78f86c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941b0ad0 con 0x55d78f869560 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941b0400 con 0x55d78f85ddd0 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d941ad620 con 0x55d78f85e6f0 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.811+0000 7f6da3d8ab80 10 client.0 _ll_register_callbacks cb 0x55d78f7498b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d93200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d93200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32830/0 (socket says 172.21.15.86:32830) 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d93200700 1 -- 172.21.15.86:0/2038771290 learned_addr learned my addr 172.21.15.86:0/2038771290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d89e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85e6f0 0x55d78f86c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f86c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 -- 172.21.15.86:0/2038771290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85e6f0 msgr2=0x55d78f86c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85e6f0 0x55d78f86c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 -- 172.21.15.86:0/2038771290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 msgr2=0x55d78f86cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 -- 172.21.15.86:0/2038771290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78f8e7d50 con 0x55d78f85ddd0 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d93200700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d92800700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f86c420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c5b0 tx=0x7f6d84098e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60e5d0c79c7130c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.812+0000 7f6d80c00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6d80c00700 1 -- 172.21.15.86:0/2038771290 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84080060 con 0x55d78f85ddd0 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d8409d200 con 0x55d78f85ddd0 2024-04-25T19:18:56.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6d80c00700 1 -- 172.21.15.86:0/2038771290 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f6d8401fd90 con 0x55d78f85ddd0 2024-04-25T19:18:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6d80c00700 1 -- 172.21.15.86:0/2038771290 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d84067070 con 0x55d78f85ddd0 2024-04-25T19:18:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6da3d8ab80 10 client.27149 fetch_fsmap learned FSMap version 29 2024-04-25T19:18:56.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d78f8e8fd0 con 0x55d78f85ddd0 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6d80c00700 1 -- 172.21.15.86:0/2038771290 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d8409d200 con 0x55d78f85ddd0 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6d80c00700 1 -- 172.21.15.86:0/2038771290 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6d840a52e0 con 0x55d78f85ddd0 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.813+0000 7f6da3d8ab80 10 client.27149 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6da3d8ab80 10 client.27149 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6da3d8ab80 20 client.27149 populate_metadata read hostname 'smithi086' 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f8e8fd0 0x55d78f8eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6da3d8ab80 4 client.27149 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x55d78f7b88b0 con 0x55d78f8e8fd0 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6d89e00700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f8e8fd0 0x55d78f8eb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6d89e00700 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f8e8fd0 0x55d78f8eb3c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0a5daf52f19b53 server_cookie=41d17e913a541306 in_seq=0 out_seq=0 2024-04-25T19:18:56.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.814+0000 7f6d80c00700 10 client.27149 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:56.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.829+0000 7f6d80c00700 1 -- 172.21.15.86:0/2038771290 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d78f7b88b0 con 0x55d78f8e8fd0 2024-04-25T19:18:56.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.829+0000 7f6d80c00700 10 client.27149 handle_command_reply: tid=0 2024-04-25T19:18:56.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.829+0000 7f6da3d8ab80 1 client.27149 shutdown 2024-04-25T19:18:56.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.829+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 msgr2=0x55d78f86c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.829+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f86c420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c5b0 tx=0x7f6d84098e20 comp rx=0 tx=0).stop 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 shutdown_connections 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d78f8e8fd0 0x55d78f8eb3c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d78f869560 0x55d78f86cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d78f85ddd0 0x55d78f86c420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 --2- 172.21.15.86:0/2038771290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d78f85e6f0 0x55d78f86c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 >> 172.21.15.86:0/2038771290 conn(0x55d78f78d0e0 msgr2=0x55d78f867800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6d80c00700 0 client.27149 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 shutdown_connections 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 1 -- 172.21.15.86:0/2038771290 wait complete. 2024-04-25T19:18:56.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.830+0000 7f6da3d8ab80 20 client.27149 trim_cache size 0 max 16384 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 46, 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "5b016426": { 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 46, 2024-04-25T19:18:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 70.0, 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d780a3e50 msgr2=0x7f6d780a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d780a3e50 0x7f6d780a6310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55d78f85f7a0 tx=0x7f6d8c00bf60 comp rx=0 tx=0).stop 2024-04-25T19:18:56.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 msgr2=0x7f6d9419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 0x7f6d9419b1e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400eb30 tx=0x7f6d8400ee40 comp rx=0 tx=0).stop 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 shutdown_connections 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d9410f4a0 0x7f6d9419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f6d780a3e50 0x7f6d780a6310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d94074c70 0x7f6d9419b1e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 --2- 172.21.15.86:0/2824906976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d94075590 0x7f6d9419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 >> 172.21.15.86:0/2824906976 conn(0x7f6d94079b10 msgr2=0x7f6d940ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 shutdown_connections 2024-04-25T19:18:56.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:18:56.831+0000 7f6d99400700 1 -- 172.21.15.86:0/2824906976 wait complete. 2024-04-25T19:18:57.766 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5b016426', quiesce took 7.6 seconds. Will release after: 10 2024-04-25T19:18:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:18:58 smithi183 ceph-mon[32344]: pgmap v503: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:18:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:18:58 smithi194 ceph-mon[32740]: pgmap v503: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:18:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:18:58 smithi086 ceph-mon[27783]: pgmap v503: 97 pgs: 97 active+clean; 3.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T19:19:00.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:00 smithi183 ceph-mon[32344]: pgmap v504: 97 pgs: 97 active+clean; 3.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:19:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:00 smithi194 ceph-mon[32740]: pgmap v504: 97 pgs: 97 active+clean; 3.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:19:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:00 smithi086 ceph-mon[27783]: pgmap v504: 97 pgs: 97 active+clean; 3.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T19:19:02.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:02 smithi183 ceph-mon[32344]: pgmap v505: 97 pgs: 97 active+clean; 3.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:19:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:02 smithi194 ceph-mon[32740]: pgmap v505: 97 pgs: 97 active+clean; 3.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:19:02.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:02 smithi086 ceph-mon[27783]: pgmap v505: 97 pgs: 97 active+clean; 3.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:19:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.428+0000 7f0215000640 1 -- 172.21.15.86:0/302235135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0210104d70 msgr2=0x7f0210108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.428+0000 7f0215000640 1 --2- 172.21.15.86:0/302235135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0210104d70 0x7f0210108db0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 -- 172.21.15.86:0/302235135 shutdown_connections 2024-04-25T19:19:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 --2- 172.21.15.86:0/302235135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0210109490 0x7f0210111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 --2- 172.21.15.86:0/302235135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0210104d70 0x7f0210108db0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 --2- 172.21.15.86:0/302235135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 0x7f02101047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 -- 172.21.15.86:0/302235135 >> 172.21.15.86:0/302235135 conn(0x7f02100ffc80 msgr2=0x7f02101020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 -- 172.21.15.86:0/302235135 shutdown_connections 2024-04-25T19:19:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.429+0000 7f0215000640 1 -- 172.21.15.86:0/302235135 wait complete. 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 Processor -- start 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 -- start start 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 0x7f02101a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0210109490 0x7f02101a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 0x7f021019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02101143d0 con 0x7f021019a770 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0210114250 con 0x7f0210109490 2024-04-25T19:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f0215000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0210114550 con 0x7f02101043c0 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.430+0000 7f020e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 0x7f021019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 0x7f021019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55090/0 (socket says 172.21.15.86:55090) 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 0x7f02101a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 -- 172.21.15.86:0/3988541867 learned_addr learned my addr 172.21.15.86:0/3988541867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f0205400640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0210109490 0x7f02101a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 -- 172.21.15.86:0/3988541867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 msgr2=0x7f02101a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 0x7f02101a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 -- 172.21.15.86:0/3988541867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0210109490 msgr2=0x7f02101a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0210109490 0x7f02101a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 -- 172.21.15.86:0/3988541867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021019b370 con 0x7f021019a770 2024-04-25T19:19:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020de00640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 0x7f02101a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f020e800640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 0x7f021019abf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f020800efd0 tx=0x7f020800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f0207400640 1 -- 172.21.15.86:0/3988541867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208003b70 con 0x7f021019a770 2024-04-25T19:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f0207400640 1 -- 172.21.15.86:0/3988541867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0208003d10 con 0x7f021019a770 2024-04-25T19:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f0207400640 1 -- 172.21.15.86:0/3988541867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080108a0 con 0x7f021019a770 2024-04-25T19:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.431+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02101a7450 con 0x7f021019a770 2024-04-25T19:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.432+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101a7800 con 0x7f021019a770 2024-04-25T19:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 Processor -- start 2024-04-25T19:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0207400640 1 -- 172.21.15.86:0/3988541867 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0208004020 con 0x7f021019a770 2024-04-25T19:19:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 -- start start 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d30820 0x55d2a8d30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d31140 0x55d2a8d3b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8c96900 con 0x55d2a8d3ba00 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8c3cef0 con 0x55d2a8d30820 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.435+0000 7f0217fd9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8c928c0 con 0x55d2a8d31140 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d30820 0x55d2a8d30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d3ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55102/0 (socket says 172.21.15.86:55102) 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 -- 172.21.15.86:0/1744827093 learned_addr learned my addr 172.21.15.86:0/1744827093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f020de00640 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d31140 0x55d2a8d3b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 -- 172.21.15.86:0/1744827093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d31140 msgr2=0x55d2a8d3b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d31140 0x55d2a8d3b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 -- 172.21.15.86:0/1744827093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d30820 msgr2=0x55d2a8d30c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d30820 0x55d2a8d30c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 -- 172.21.15.86:0/1744827093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a89a9210 con 0x55d2a8d3ba00 2024-04-25T19:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f020de00640 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d31140 0x55d2a8d3b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0207400640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f01f40a3e40 0x7f01f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.436+0000 7f0205400640 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d3ddf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cca6727a570b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f0207400640 1 -- 172.21.15.86:0/3988541867 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f02080d0bc0 con 0x7f021019a770 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f01ff400640 1 -- 172.21.15.86:0/1744827093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8035070 con 0x55d2a8d3ba00 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f020de00640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f01f40a3e40 0x7f01f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f01ff400640 1 -- 172.21.15.86:0/1744827093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f80317e0 con 0x55d2a8d3ba00 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f01ff400640 1 -- 172.21.15.86:0/1744827093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8003910 con 0x55d2a8d3ba00 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f0217fd9740 1 -- 172.21.15.86:0/1744827093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 msgr2=0x55d2a8d3ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f020de00640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f01f40a3e40 0x7f01f40a6300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0200009fd0 tx=0x7f0200005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.437+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d3ddf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f8031400 comp rx=0 tx=0).stop 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 -- 172.21.15.86:0/1744827093 shutdown_connections 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d3ddf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d31140 0x55d2a8d3b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1744827093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d30820 0x55d2a8d30c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 -- 172.21.15.86:0/1744827093 >> 172.21.15.86:0/1744827093 conn(0x55d2a8d2cd30 msgr2=0x55d2a8d2d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 -- 172.21.15.86:0/1744827093 shutdown_connections 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 -- 172.21.15.86:0/1744827093 wait complete. 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 -- start start 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.438+0000 7f0217fd9740 1 Processor -- start 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8d38920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d38e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8d3e5d0 con 0x55d2a8d3ba00 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8c50310 con 0x55d2a8d31140 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8d3e750 con 0x55d2a8d30820 2024-04-25T19:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0217fd9740 10 client.0 _ll_register_callbacks cb 0x55d2a8c1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f020de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8d38920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f020de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8d38920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:42572/0 (socket says 172.21.15.86:42572) 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f020de00640 1 -- 172.21.15.86:0/1076349625 learned_addr learned my addr 172.21.15.86:0/1076349625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f020e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.439+0000 7f0205400640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d38e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0205400640 1 -- 172.21.15.86:0/1076349625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 msgr2=0x55d2a8d38920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0205400640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8d38920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0205400640 1 -- 172.21.15.86:0/1076349625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 msgr2=0x55d2a8d393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0205400640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0205400640 1 -- 172.21.15.86:0/1076349625 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a8a75620 con 0x55d2a8d31140 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f020e800640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0205400640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d38e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000c00 tx=0x7f01f8032d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a5cda2e42a4030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f01f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f01f3e00640 1 -- 172.21.15.86:0/1076349625 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f804e070 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f01f3e00640 1 -- 172.21.15.86:0/1076349625 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f01f8032e00 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f80317e0 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f01f3e00640 1 -- 172.21.15.86:0/1076349625 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01f8056050 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0217fd9740 10 client.27161 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.440+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d2a8d4a810 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f01f3e00640 1 -- 172.21.15.86:0/1076349625 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f80317e0 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f01f3e00640 1 -- 172.21.15.86:0/1076349625 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01f8051030 con 0x55d2a8d31140 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f0217fd9740 10 client.27161 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f0217fd9740 10 client.27161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f0217fd9740 20 client.27161 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8d4bb60 0x55d2a8d4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f0217fd9740 4 client.27161 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a8caa8b0 con 0x55d2a8d4bb60 2024-04-25T19:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.441+0000 7f020de00640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8d4bb60 0x55d2a8d4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.442+0000 7f020de00640 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8d4bb60 0x55d2a8d4df50 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d3200a8e6e0024 server_cookie=171137727a59f7e9 in_seq=0 out_seq=0 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.442+0000 7f01f3e00640 10 client.27161 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f01f3e00640 1 -- 172.21.15.86:0/1076349625 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2a8caa8b0 con 0x55d2a8d4bb60 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f01f3e00640 10 client.27161 handle_command_reply: tid=0 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 client.27161 shutdown 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 msgr2=0x55d2a8d38e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d38e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000c00 tx=0x7f01f8032d60 comp rx=0 tx=0).stop 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 shutdown_connections 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8d4bb60 0x55d2a8d4df50 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d38e60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 --2- 172.21.15.86:0/1076349625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8d38920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f01f3e00640 0 client.27161 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 >> 172.21.15.86:0/1076349625 conn(0x55d2a8d2cd30 msgr2=0x55d2a8d32a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 shutdown_connections 2024-04-25T19:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.443+0000 7f0217fd9740 1 -- 172.21.15.86:0/1076349625 wait complete. 2024-04-25T19:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.444+0000 7f0217fd9740 20 client.27161 trim_cache size 0 max 16384 2024-04-25T19:19:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.459+0000 7f0217fd9740 1 Processor -- start 2024-04-25T19:19:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 -- start start 2024-04-25T19:19:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 0x55d2a8d50f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02101affa0 con 0x55d2a8d30820 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02101b1ae0 con 0x55d2a8d31140 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0217fd9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02101b0670 con 0x55d2a8d3ba00 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 0x55d2a8d50f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 0x55d2a8d50f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55128/0 (socket says 172.21.15.86:55128) 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 -- 172.21.15.86:0/3085983125 learned_addr learned my addr 172.21.15.86:0/3085983125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f0205400640 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020e800640 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 -- 172.21.15.86:0/3085983125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d3ba00 msgr2=0x55d2a8d469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 -- 172.21.15.86:0/3085983125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 msgr2=0x55d2a8d514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020de00640 1 -- 172.21.15.86:0/3085983125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a8d36e90 con 0x55d2a8d30820 2024-04-25T19:19:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.460+0000 7f020e800640 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.461+0000 7f020de00640 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 0x55d2a8d50f70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0200012fe0 tx=0x7f020001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1968a2bde67954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.461+0000 7f01fea00640 1 -- 172.21.15.86:0/3085983125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000122d0 con 0x55d2a8d30820 2024-04-25T19:19:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.461+0000 7f01fea00640 1 -- 172.21.15.86:0/3085983125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f020002b8d0 con 0x55d2a8d30820 2024-04-25T19:19:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.461+0000 7f01fea00640 1 -- 172.21.15.86:0/3085983125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200004a00 con 0x55d2a8d30820 2024-04-25T19:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 -- 172.21.15.86:0/3085983125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 msgr2=0x55d2a8d50f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 0x55d2a8d50f70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0200012fe0 tx=0x7f020001a000 comp rx=0 tx=0).stop 2024-04-25T19:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 -- 172.21.15.86:0/3085983125 shutdown_connections 2024-04-25T19:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8d514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 --2- 172.21.15.86:0/3085983125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d30820 0x55d2a8d50f70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 -- 172.21.15.86:0/3085983125 >> 172.21.15.86:0/3085983125 conn(0x55d2a8d2cd30 msgr2=0x55d2a8d37210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 -- 172.21.15.86:0/3085983125 shutdown_connections 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 -- 172.21.15.86:0/3085983125 wait complete. 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 -- start start 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.462+0000 7f0217fd9740 1 Processor -- start 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8db2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8db2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d50260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02101b1410 con 0x55d2a8d3ba00 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02101b0d40 con 0x55d2a8d31140 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02101adfa0 con 0x55d2a8d30820 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0217fd9740 10 client.0 _ll_register_callbacks cb 0x7f01f8031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d50260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d50260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55130/0 (socket says 172.21.15.86:55130) 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 -- 172.21.15.86:0/4027790251 learned_addr learned my addr 172.21.15.86:0/4027790251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0205400640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8db2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020de00640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8db2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 -- 172.21.15.86:0/4027790251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 msgr2=0x55d2a8db2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8db2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 -- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 msgr2=0x55d2a8db2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8db2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f020e800640 1 -- 172.21.15.86:0/4027790251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a8d50950 con 0x55d2a8d3ba00 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.463+0000 7f0205400640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8db2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f020e800640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d50260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f02080ce2a0 tx=0x7f0208009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f898e8301b1a376 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f01f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f01f3e00640 1 -- 172.21.15.86:0/4027790251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080a0e40 con 0x55d2a8d3ba00 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f01f3e00640 1 -- 172.21.15.86:0/4027790251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8637+0+0 (secure 0 0 0) 0x7f0208004560 con 0x55d2a8d3ba00 2024-04-25T19:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020002b8d0 con 0x55d2a8d3ba00 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f01f3e00640 1 -- 172.21.15.86:0/4027790251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02080a5700 con 0x55d2a8d3ba00 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f0217fd9740 10 client.17076 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d2a8dba0c0 con 0x55d2a8d3ba00 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.464+0000 7f01f3e00640 1 -- 172.21.15.86:0/4027790251 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f020002b8d0 con 0x55d2a8d3ba00 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f01f3e00640 1 -- 172.21.15.86:0/4027790251 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f020801f020 con 0x55d2a8d3ba00 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f0217fd9740 10 client.17076 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f0217fd9740 10 client.17076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f0217fd9740 20 client.17076 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f0217fd9740 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8dba790 0x55d2a8dbcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f0217fd9740 4 client.17076 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2a8ca4320 con 0x55d2a8dba790 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.465+0000 7f020de00640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8dba790 0x55d2a8dbcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.466+0000 7f020de00640 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8dba790 0x55d2a8dbcbb0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea03068b316dc24 server_cookie=f37c47cd7aaeb5cb in_seq=0 out_seq=0 2024-04-25T19:19:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.466+0000 7f01f3e00640 10 client.17076 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.481+0000 7f01f3e00640 1 -- 172.21.15.86:0/4027790251 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2a8ca4320 con 0x55d2a8dba790 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.481+0000 7f01f3e00640 10 client.17076 handle_command_reply: tid=0 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.481+0000 7f0217fd9740 1 client.17076 shutdown 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.481+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 msgr2=0x55d2a8d50260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.481+0000 7f0217fd9740 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d50260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f02080ce2a0 tx=0x7f0208009820 comp rx=0 tx=0).stop 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 shutdown_connections 2024-04-25T19:19:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55d2a8dba790 0x55d2a8dbcbb0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2a8d3ba00 0x55d2a8d50260 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2a8d31140 0x55d2a8db2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 --2- 172.21.15.86:0/4027790251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2a8d30820 0x55d2a8db2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f01f3e00640 0 client.17076 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 >> 172.21.15.86:0/4027790251 conn(0x55d2a8d2cd30 msgr2=0x55d2a8d469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 shutdown_connections 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 1 -- 172.21.15.86:0/4027790251 wait complete. 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.482+0000 7f0217fd9740 20 client.17076 trim_cache size 0 max 16384 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:19:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:19:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f01f40a3e40 msgr2=0x7f01f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f01f40a3e40 0x7f01f40a6300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0200009fd0 tx=0x7f0200005f90 comp rx=0 tx=0).stop 2024-04-25T19:19:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 msgr2=0x7f021019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 0x7f021019abf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f020800efd0 tx=0x7f020800c6a0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 shutdown_connections 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f01f40a3e40 0x7f01f40a6300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f021019a770 0x7f021019abf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0210109490 0x7f02101a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 --2- 172.21.15.86:0/3988541867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02101043c0 0x7f02101a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 >> 172.21.15.86:0/3988541867 conn(0x7f02100ffc80 msgr2=0x7f0210100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 shutdown_connections 2024-04-25T19:19:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:03.483+0000 7f0215000640 1 -- 172.21.15.86:0/3988541867 wait complete. 2024-04-25T19:19:04.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:04 smithi183 ceph-mon[32344]: pgmap v506: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 82 KiB/s wr, 36 op/s 2024-04-25T19:19:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:04 smithi194 ceph-mon[32740]: pgmap v506: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 82 KiB/s wr, 36 op/s 2024-04-25T19:19:04.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:04 smithi086 ceph-mon[27783]: pgmap v506: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 82 KiB/s wr, 36 op/s 2024-04-25T19:19:06.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:06 smithi183 ceph-mon[32344]: pgmap v507: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-25T19:19:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:06 smithi194 ceph-mon[32740]: pgmap v507: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-25T19:19:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:06 smithi086 ceph-mon[27783]: pgmap v507: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-25T19:19:07.667 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:07 smithi183 ceph-mon[32344]: osdmap e115: 12 total, 12 up, 12 in 2024-04-25T19:19:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:07 smithi194 ceph-mon[32740]: osdmap e115: 12 total, 12 up, 12 in 2024-04-25T19:19:07.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:19:07.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:07 smithi086 ceph-mon[27783]: osdmap e115: 12 total, 12 up, 12 in 2024-04-25T19:19:08.212 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:19:08.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:08 smithi183 ceph-mon[32344]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 32 op/s 2024-04-25T19:19:08.733 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:08 smithi194 ceph-mon[32740]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 32 op/s 2024-04-25T19:19:08.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:08 smithi086 ceph-mon[27783]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 32 op/s 2024-04-25T19:19:10.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 -- 172.21.15.86:0/4160244508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc108f80 msgr2=0x7f0bbc109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:10.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 --2- 172.21.15.86:0/4160244508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc108f80 0x7f0bbc109400 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009990 tx=0x7f0ba8009ca0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 -- 172.21.15.86:0/4160244508 shutdown_connections 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 --2- 172.21.15.86:0/4160244508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc109940 0x7f0bbc115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 --2- 172.21.15.86:0/4160244508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc10f030 0x7f0bbc10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 --2- 172.21.15.86:0/4160244508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc108f80 0x7f0bbc109400 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.540+0000 7f0bbb200700 1 -- 172.21.15.86:0/4160244508 >> 172.21.15.86:0/4160244508 conn(0x7f0bbc0787d0 msgr2=0x7f0bbc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.541+0000 7f0bbb200700 1 -- 172.21.15.86:0/4160244508 shutdown_connections 2024-04-25T19:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.541+0000 7f0bbb200700 1 -- 172.21.15.86:0/4160244508 wait complete. 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 Processor -- start 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 -- start start 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc108f80 0x7f0bbc19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 0x7f0bbc19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc10f030 0x7f0bbc1a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1190f0 con 0x7f0bbc109940 2024-04-25T19:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc118f70 con 0x7f0bbc108f80 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bbb200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc119270 con 0x7f0bbc10f030 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.542+0000 7f0bb9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 0x7f0bbc19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc108f80 0x7f0bbc19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 0x7f0bbc19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49790/0 (socket says 172.21.15.86:49790) 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9400700 1 -- 172.21.15.86:0/3983659914 learned_addr learned my addr 172.21.15.86:0/3983659914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bba800700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc10f030 0x7f0bbc1a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9e00700 1 -- 172.21.15.86:0/3983659914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc10f030 msgr2=0x7f0bbc1a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9e00700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc10f030 0x7f0bbc1a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9e00700 1 -- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 msgr2=0x7f0bbc19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9e00700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 0x7f0bbc19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9e00700 1 -- 172.21.15.86:0/3983659914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc1a4590 con 0x7f0bbc108f80 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.543+0000 7f0bb9400700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 0x7f0bbc19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.544+0000 7f0bb9e00700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc108f80 0x7f0bbc19f580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00e9f0 tx=0x7f0bac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.544+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00c9c0 con 0x7f0bbc108f80 2024-04-25T19:19:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.544+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bac00cb60 con 0x7f0bbc108f80 2024-04-25T19:19:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.544+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc1a4880 con 0x7f0bbc108f80 2024-04-25T19:19:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.544+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac018680 con 0x7f0bbc108f80 2024-04-25T19:19:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.544+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1ac100 con 0x7f0bbc108f80 2024-04-25T19:19:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.545+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc069730 con 0x7f0bbc108f80 2024-04-25T19:19:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.547+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0bac018a30 con 0x7f0bbc108f80 2024-04-25T19:19:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.548+0000 7f0bb2a00700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ba00a3e30 0x7f0ba00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.549+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8681+0+0 (secure 0 0 0) 0x7f0bac0d0d50 con 0x7f0bbc108f80 2024-04-25T19:19:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.549+0000 7f0bb9400700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ba00a3e30 0x7f0ba00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.550+0000 7f0bb9400700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ba00a3e30 0x7f0ba00a62f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc10e750 tx=0x7f0ba8009510 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.551+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bac09d2f0 con 0x7f0bbc108f80 2024-04-25T19:19:10.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:10 smithi183 ceph-mon[32344]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 34 op/s 2024-04-25T19:19:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:10 smithi194 ceph-mon[32740]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 34 op/s 2024-04-25T19:19:10.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:10 smithi086 ceph-mon[27783]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 34 op/s 2024-04-25T19:19:10.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.822+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bbc1a08e0 con 0x7f0bbc108f80 2024-04-25T19:19:10.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.823+0000 7f0bb2a00700 1 -- 172.21.15.86:0/3983659914 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f0bac09cbf0 con 0x7f0bbc108f80 2024-04-25T19:19:10.825 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:19:10.826 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ba00a3e30 msgr2=0x7f0ba00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ba00a3e30 0x7f0ba00a62f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc10e750 tx=0x7f0ba8009510 comp rx=0 tx=0).stop 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc108f80 msgr2=0x7f0bbc19f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc108f80 0x7f0bbc19f580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00e9f0 tx=0x7f0bac00ed00 comp rx=0 tx=0).stop 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 shutdown_connections 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ba00a3e30 0x7f0ba00a62f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc10f030 0x7f0bbc1a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc108f80 0x7f0bbc19f580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 --2- 172.21.15.86:0/3983659914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bbc109940 0x7f0bbc19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 >> 172.21.15.86:0/3983659914 conn(0x7f0bbc0787d0 msgr2=0x7f0bbc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 shutdown_connections 2024-04-25T19:19:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:10.827+0000 7f0bbb200700 1 -- 172.21.15.86:0/3983659914 wait complete. 2024-04-25T19:19:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:19:11.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:11 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3983659914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:11.612 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 5b016426 --release --await' 2024-04-25T19:19:11.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 5b016426 --release --await 2024-04-25T19:19:11.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:11 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3983659914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:11.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:11 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3983659914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:12.056 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:19:12.561 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:12 smithi086 ceph-mon[27783]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 34 op/s 2024-04-25T19:19:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:12 smithi194 ceph-mon[32740]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 34 op/s 2024-04-25T19:19:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:12 smithi183 ceph-mon[32344]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 34 op/s 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.311+0000 7f07bae00700 1 -- 172.21.15.86:0/3263983274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b4075590 msgr2=0x7f07b410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.311+0000 7f07bae00700 1 --2- 172.21.15.86:0/3263983274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b4075590 0x7f07b410ef00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009990 tx=0x7f07a0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 -- 172.21.15.86:0/3263983274 shutdown_connections 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 --2- 172.21.15.86:0/3263983274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410f440 0x7f07b4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 --2- 172.21.15.86:0/3263983274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4074c70 0x7f07b4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 --2- 172.21.15.86:0/3263983274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b4075590 0x7f07b410ef00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 -- 172.21.15.86:0/3263983274 >> 172.21.15.86:0/3263983274 conn(0x7f07b4079ba0 msgr2=0x7f07b4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 -- 172.21.15.86:0/3263983274 shutdown_connections 2024-04-25T19:19:14.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.312+0000 7f07bae00700 1 -- 172.21.15.86:0/3263983274 wait complete. 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 Processor -- start 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 -- start start 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b4074c70 0x7f07b419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 0x7f07b419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b410f440 0x7f07b419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4114e70 con 0x7f07b410f440 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4114cf0 con 0x7f07b4074c70 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.313+0000 7f07bae00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4114ff0 con 0x7f07b4075590 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 0x7f07b419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b4074c70 0x7f07b419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 0x7f07b419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33538/0 (socket says 172.21.15.86:33538) 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 -- 172.21.15.86:0/2315654977 learned_addr learned my addr 172.21.15.86:0/2315654977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b8e00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b410f440 0x7f07b419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 -- 172.21.15.86:0/2315654977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b4074c70 msgr2=0x7f07b419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b4074c70 0x7f07b419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 -- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b410f440 msgr2=0x7f07b419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b410f440 0x7f07b419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.314+0000 7f07b3400700 1 -- 172.21.15.86:0/2315654977 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b41a0370 con 0x7f07b4075590 2024-04-25T19:19:14.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.315+0000 7f07b3400700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 0x7f07b419b8a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07a0005910 tx=0x7f07a000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.315+0000 7f07b0c00700 1 -- 172.21.15.86:0/2315654977 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0027070 con 0x7f07b4075590 2024-04-25T19:19:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.315+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b41a0600 con 0x7f07b4075590 2024-04-25T19:19:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.315+0000 7f07b0c00700 1 -- 172.21.15.86:0/2315654977 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a0022470 con 0x7f07b4075590 2024-04-25T19:19:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.315+0000 7f07b0c00700 1 -- 172.21.15.86:0/2315654977 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0017650 con 0x7f07b4075590 2024-04-25T19:19:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.315+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41a7ee0 con 0x7f07b4075590 2024-04-25T19:19:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.319+0000 7f07b0c00700 1 -- 172.21.15.86:0/2315654977 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f07a002a070 con 0x7f07b4075590 2024-04-25T19:19:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.319+0000 7f07c5803b80 1 Processor -- start 2024-04-25T19:19:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 -- start start 2024-04-25T19:19:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a016303d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a0163114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a016256f90 con 0x55a016303920 2024-04-25T19:19:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0161fd830 con 0x55a016304240 2024-04-25T19:19:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07c5803b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a016252f50 con 0x55a01630f0f0 2024-04-25T19:19:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07b8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a016303d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07b8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a016303d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49822/0 (socket says 172.21.15.86:49822) 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.320+0000 7f07b8e00700 1 -- 172.21.15.86:0/2281893160 learned_addr learned my addr 172.21.15.86:0/2281893160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b0c00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07980a3e30 0x7f07980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3400700 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a0163114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 -- 172.21.15.86:0/2281893160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 msgr2=0x55a0163114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a0163114e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 -- 172.21.15.86:0/2281893160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 msgr2=0x55a016303d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a016303d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 -- 172.21.15.86:0/2281893160 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0161f3780 con 0x55a016304240 2024-04-25T19:19:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b0c00700 1 -- 172.21.15.86:0/2315654977 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f07a00d1520 con 0x7f07b4075590 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b8e00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07980a3e30 0x7f07980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.321+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630ebb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009cb0 tx=0x7f07a4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3156ac51a0c3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.322+0000 7f07a9600700 1 -- 172.21.15.86:0/2281893160 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4011d20 con 0x55a016304240 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.322+0000 7f07a9600700 1 -- 172.21.15.86:0/2281893160 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a40044e0 con 0x55a016304240 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.322+0000 7f07a9600700 1 -- 172.21.15.86:0/2281893160 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4005280 con 0x55a016304240 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281893160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 msgr2=0x55a01630ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07b8e00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07980a3e30 0x7f07980a62f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0075c0 tx=0x7f07ac009040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630ebb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009cb0 tx=0x7f07a4011440 comp rx=0 tx=0).stop 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281893160 shutdown_connections 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a0163114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a016303d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281893160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630ebb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281893160 >> 172.21.15.86:0/2281893160 conn(0x55a0162326a0 msgr2=0x55a0162ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281893160 shutdown_connections 2024-04-25T19:19:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281893160 wait complete. 2024-04-25T19:19:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.323+0000 7f07c5803b80 1 -- start start 2024-04-25T19:19:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 Processor -- start 2024-04-25T19:19:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a01630e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a01630c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a016311c10 con 0x55a016303920 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a016210960 con 0x55a016304240 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a016311d90 con 0x55a01630f0f0 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.324+0000 7f07c5803b80 10 client.0 _ll_register_callbacks cb 0x55a0161ef820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a01630e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a01630e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49836/0 (socket says 172.21.15.86:49836) 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3e00700 1 -- 172.21.15.86:0/2281670097 learned_addr learned my addr 172.21.15.86:0/2281670097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b8e00700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a01630c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 -- 172.21.15.86:0/2281670097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 msgr2=0x55a01630c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a01630c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 -- 172.21.15.86:0/2281670097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 msgr2=0x55a01630e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a01630e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 -- 172.21.15.86:0/2281670097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0161a49d0 con 0x55a016304240 2024-04-25T19:19:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b8e00700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a01630c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f07b3400700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630bac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07a00ceba0 tx=0x7f07a009d2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7c68bcbb90727c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f079e000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.325+0000 7f079e000700 1 -- 172.21.15.86:0/2281670097 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a009db80 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f079e000700 1 -- 172.21.15.86:0/2281670097 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f07a00cee20 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a40044e0 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f079e000700 1 -- 172.21.15.86:0/2281670097 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07a00cf020 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f07c5803b80 10 client.27185 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0161edbf0 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f079e000700 1 -- 172.21.15.86:0/2281670097 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a40044e0 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.326+0000 7f079e000700 1 -- 172.21.15.86:0/2281670097 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07a00d0070 con 0x55a016304240 2024-04-25T19:19:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07c5803b80 10 client.27185 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07c5803b80 10 client.27185 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07c5803b80 20 client.27185 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a016327010 0x55a016329430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07c5803b80 4 client.27185 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a016264780 con 0x55a016327010 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.327+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a016327010 0x55a016329430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.328+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a016327010 0x55a016329430 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4e5be347ac58f3 server_cookie=98fe6f4e6c86d48 in_seq=0 out_seq=0 2024-04-25T19:19:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.328+0000 7f079e000700 10 client.27185 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.328+0000 7f079e000700 1 -- 172.21.15.86:0/2281670097 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a016264780 con 0x55a016327010 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.328+0000 7f079e000700 10 client.27185 handle_command_reply: tid=0 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.328+0000 7f07c5803b80 1 client.27185 shutdown 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 msgr2=0x55a01630bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630bac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07a00ceba0 tx=0x7f07a009d2d0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 shutdown_connections 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a016327010 0x55a016329430 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a01630f0f0 0x55a01630c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a01630e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f079e000700 0 client.27185 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2281670097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016304240 0x55a01630bac0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 >> 172.21.15.86:0/2281670097 conn(0x55a0162326a0 msgr2=0x55a0162ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 shutdown_connections 2024-04-25T19:19:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 1 -- 172.21.15.86:0/2281670097 wait complete. 2024-04-25T19:19:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.329+0000 7f07c5803b80 20 client.27185 trim_cache size 0 max 16384 2024-04-25T19:19:14.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 Processor -- start 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 -- start start 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a0163233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a01630f0f0 0x55a01631fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41ab290 con 0x55a016303920 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41acdd0 con 0x55a01630f0f0 2024-04-25T19:19:14.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.351+0000 7f07c5803b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41ab960 con 0x55a016304240 2024-04-25T19:19:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a0163233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:33564/0 (socket says 172.21.15.86:33564) 2024-04-25T19:19:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 -- 172.21.15.86:0/2567280022 learned_addr learned my addr 172.21.15.86:0/2567280022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b8e00700 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a01630f0f0 0x55a01631fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 -- 172.21.15.86:0/2567280022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a01630f0f0 msgr2=0x55a01631fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a01630f0f0 0x55a01631fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 -- 172.21.15.86:0/2567280022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 msgr2=0x55a0163233a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a0163233a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3400700 1 -- 172.21.15.86:0/2567280022 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a016329a90 con 0x55a016304240 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.352+0000 7f07b3e00700 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a0163233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.353+0000 7f07b3400700 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631f8b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07a0005f50 tx=0x7f07a0005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a6773ae1219d708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.353+0000 7f07a8c00700 1 -- 172.21.15.86:0/2567280022 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00a1830 con 0x55a016304240 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.353+0000 7f07a8c00700 1 -- 172.21.15.86:0/2567280022 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a00a6060 con 0x55a016304240 2024-04-25T19:19:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.353+0000 7f07a8c00700 1 -- 172.21.15.86:0/2567280022 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00991d0 con 0x55a016304240 2024-04-25T19:19:14.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 -- 172.21.15.86:0/2567280022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 msgr2=0x55a01631f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631f8b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07a0005f50 tx=0x7f07a0005c40 comp rx=0 tx=0).stop 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 -- 172.21.15.86:0/2567280022 shutdown_connections 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a01630f0f0 0x55a01631fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a016303920 0x55a0163233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 --2- 172.21.15.86:0/2567280022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631f8b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 -- 172.21.15.86:0/2567280022 >> 172.21.15.86:0/2567280022 conn(0x55a0162326a0 msgr2=0x55a0162fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 -- 172.21.15.86:0/2567280022 shutdown_connections 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 -- 172.21.15.86:0/2567280022 wait complete. 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 -- start start 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.354+0000 7f07c5803b80 1 Processor -- start 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016303920 0x55a01631e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 0x55a01638d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41ac700 con 0x55a01630f0f0 2024-04-25T19:19:14.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41ac030 con 0x55a016303920 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41a92c0 con 0x55a016304240 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07c5803b80 10 client.0 _ll_register_callbacks cb 0x55a0161ef820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07b8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 0x55a01638d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07b3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016303920 0x55a01631e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07b3400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07b8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 0x55a01638d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49850/0 (socket says 172.21.15.86:49850) 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07b8e00700 1 -- 172.21.15.86:0/1453784905 learned_addr learned my addr 172.21.15.86:0/1453784905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:14.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.355+0000 7f07b3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016303920 0x55a01631e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55580/0 (socket says 172.21.15.86:55580) 2024-04-25T19:19:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b3400700 1 -- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016303920 msgr2=0x55a01631e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b3400700 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016303920 0x55a01631e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b3400700 1 -- 172.21.15.86:0/1453784905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 msgr2=0x55a01638d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b3400700 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 0x55a01638d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b3400700 1 -- 172.21.15.86:0/1453784905 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01631f3a0 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b8e00700 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 0x55a01638d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07b3400700 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631eac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f07a0099e50 tx=0x7f07a000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb3404965b65b7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f079e000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f079e000700 1 -- 172.21.15.86:0/1453784905 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a009cfc0 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a00a6060 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f079e000700 1 -- 172.21.15.86:0/1453784905 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f07a00b6d70 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f079e000700 1 -- 172.21.15.86:0/1453784905 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07a00bc350 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07c5803b80 10 client.27187 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.356+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a01631f3a0 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f079e000700 1 -- 172.21.15.86:0/1453784905 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a00a6060 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f079e000700 1 -- 172.21.15.86:0/1453784905 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07a00bc030 con 0x55a016304240 2024-04-25T19:19:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f07c5803b80 10 client.27187 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f07c5803b80 10 client.27187 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f07c5803b80 20 client.27187 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f07c5803b80 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a01638eed0 0x55a0163912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f07c5803b80 4 client.27187 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5b016426", "release": true, "await": true}] 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.357+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5b016426", "release": true, "await": true}) -- 0x55a01625e820 con 0x55a01638eed0 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.358+0000 7f07b3e00700 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a01638eed0 0x55a0163912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.358+0000 7f07b3e00700 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a01638eed0 0x55a0163912c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f6927c9e3e8fa server_cookie=3adfddf6134bcacc in_seq=0 out_seq=0 2024-04-25T19:19:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.358+0000 7f079e000700 10 client.27187 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f079e000700 1 -- 172.21.15.86:0/1453784905 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a01625e820 con 0x55a01638eed0 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f079e000700 10 client.27187 handle_command_reply: tid=0 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 client.27187 shutdown 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 msgr2=0x55a01631eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631eac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f07a0099e50 tx=0x7f07a000bf60 comp rx=0 tx=0).stop 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 shutdown_connections 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55a01638eed0 0x55a0163912c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a01630f0f0 0x55a01638d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a016303920 0x55a01631e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 --2- 172.21.15.86:0/1453784905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a016304240 0x55a01631eac0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 >> 172.21.15.86:0/1453784905 conn(0x55a0162326a0 msgr2=0x55a0162ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f079e000700 0 client.27187 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:14.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.361+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 shutdown_connections 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07c5803b80 1 -- 172.21.15.86:0/1453784905 wait complete. 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07c5803b80 20 client.27187 trim_cache size 0 max 16384 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 48, 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "5b016426": { 2024-04-25T19:19:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 48, 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 70.0, 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:14.366 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:19:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07980a3e30 msgr2=0x7f07980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07980a3e30 0x7f07980a62f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0075c0 tx=0x7f07ac009040 comp rx=0 tx=0).stop 2024-04-25T19:19:14.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 msgr2=0x7f07b419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 0x7f07b419b8a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07a0005910 tx=0x7f07a000b670 comp rx=0 tx=0).stop 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 shutdown_connections 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b410f440 0x7f07b419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b4074c70 0x7f07b419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f07980a3e30 0x7f07980a62f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.362+0000 7f07bae00700 1 --2- 172.21.15.86:0/2315654977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07b4075590 0x7f07b419b8a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.363+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 >> 172.21.15.86:0/2315654977 conn(0x7f07b4079ba0 msgr2=0x7f07b40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.363+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 shutdown_connections 2024-04-25T19:19:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:14.363+0000 7f07bae00700 1 -- 172.21.15.86:0/2315654977 wait complete. 2024-04-25T19:19:14.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:14 smithi086 ceph-mon[27783]: pgmap v512: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T19:19:14.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:14 smithi086 ceph-mon[27783]: osdmap e116: 12 total, 12 up, 12 in 2024-04-25T19:19:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:14 smithi183 ceph-mon[32344]: pgmap v512: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T19:19:14.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:14 smithi183 ceph-mon[32344]: osdmap e116: 12 total, 12 up, 12 in 2024-04-25T19:19:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:14 smithi194 ceph-mon[32740]: pgmap v512: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T19:19:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:14 smithi194 ceph-mon[32740]: osdmap e116: 12 total, 12 up, 12 in 2024-04-25T19:19:15.304 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5b016426', seconds elapsed: 25.2 2024-04-25T19:19:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.626+0000 7f893fe00640 1 -- 172.21.15.86:0/1574459052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108490 msgr2=0x7f89401088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.626+0000 7f893fe00640 1 --2- 172.21.15.86:0/1574459052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108490 0x7f89401088d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8928009930 tx=0x7f892802a420 comp rx=0 tx=0).stop 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 -- 172.21.15.86:0/1574459052 shutdown_connections 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 --2- 172.21.15.86:0/1574459052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108e10 0x7f8940115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 --2- 172.21.15.86:0/1574459052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108490 0x7f89401088d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 --2- 172.21.15.86:0/1574459052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894010e420 0x7f894010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 -- 172.21.15.86:0/1574459052 >> 172.21.15.86:0/1574459052 conn(0x7f8940077d30 msgr2=0x7f8940106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 -- 172.21.15.86:0/1574459052 shutdown_connections 2024-04-25T19:19:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.627+0000 7f893fe00640 1 -- 172.21.15.86:0/1574459052 wait complete. 2024-04-25T19:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 Processor -- start 2024-04-25T19:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 -- start start 2024-04-25T19:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108490 0x7f89401a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108e10 0x7f89401a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894010e420 0x7f894019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89401185d0 con 0x7f8940108e10 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8940118450 con 0x7f8940108490 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.628+0000 7f893fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8940118750 con 0x7f894010e420 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f8936000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108e10 0x7f89401a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108490 0x7f89401a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f8936000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108e10 0x7f89401a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49870/0 (socket says 172.21.15.86:49870) 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f8936000640 1 -- 172.21.15.86:0/3679269468 learned_addr learned my addr 172.21.15.86:0/3679269468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893f400640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894010e420 0x7f894019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 -- 172.21.15.86:0/3679269468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894010e420 msgr2=0x7f894019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894010e420 0x7f894019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 -- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108e10 msgr2=0x7f89401a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108e10 0x7f89401a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 -- 172.21.15.86:0/3679269468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894019f0f0 con 0x7f8940108490 2024-04-25T19:19:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f893ea00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108490 0x7f89401a47c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f893000e9f0 tx=0x7f893000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.629+0000 7f8937e00640 1 -- 172.21.15.86:0/3679269468 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893000cd80 con 0x7f8940108490 2024-04-25T19:19:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.630+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894019f3e0 con 0x7f8940108490 2024-04-25T19:19:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.630+0000 7f8937e00640 1 -- 172.21.15.86:0/3679269468 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8930011920 con 0x7f8940108490 2024-04-25T19:19:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.630+0000 7f8937e00640 1 -- 172.21.15.86:0/3679269468 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930010970 con 0x7f8940108490 2024-04-25T19:19:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.630+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89401ab680 con 0x7f8940108490 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 Processor -- start 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 -- start start 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f8937e00640 1 -- 172.21.15.86:0/3679269468 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8930011470 con 0x7f8940108490 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c59728d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59733650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c59735f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c5968e900 con 0x563c597289b0 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c59634ef0 con 0x563c59733b90 2024-04-25T19:19:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.633+0000 7f894844c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c5968a8c0 con 0x563c597292d0 2024-04-25T19:19:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c59728d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c59728d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49874/0 (socket says 172.21.15.86:49874) 2024-04-25T19:19:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 -- 172.21.15.86:0/2755608712 learned_addr learned my addr 172.21.15.86:0/2755608712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f8936000640 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c59735f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893ea00640 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59733650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 -- 172.21.15.86:0/2755608712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 msgr2=0x563c59733650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59733650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 -- 172.21.15.86:0/2755608712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 msgr2=0x563c59735f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c59735f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 -- 172.21.15.86:0/2755608712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c593a1210 con 0x563c597289b0 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893ea00640 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59733650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.634+0000 7f893f400640 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c59728d90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ca0 tx=0x7f8938011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65d9afa7d60ba58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f892f400640 1 -- 172.21.15.86:0/2755608712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893801e070 con 0x563c597289b0 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f892f400640 1 -- 172.21.15.86:0/2755608712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8938009e50 con 0x563c597289b0 2024-04-25T19:19:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f892f400640 1 -- 172.21.15.86:0/2755608712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893800ea10 con 0x563c597289b0 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f8937e00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89240a3d90 0x7f89240a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f8937e00640 1 -- 172.21.15.86:0/3679269468 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f89300d0ca0 con 0x7f8940108490 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f894844c740 1 -- 172.21.15.86:0/2755608712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 msgr2=0x563c59728d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f894844c740 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c59728d90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ca0 tx=0x7f8938011810 comp rx=0 tx=0).stop 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.635+0000 7f893ea00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89240a3d90 0x7f89240a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 -- 172.21.15.86:0/2755608712 shutdown_connections 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c59735f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59733650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 --2- 172.21.15.86:0/2755608712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c59728d90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 -- 172.21.15.86:0/2755608712 >> 172.21.15.86:0/2755608712 conn(0x563c59724ec0 msgr2=0x563c597252a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 -- 172.21.15.86:0/2755608712 shutdown_connections 2024-04-25T19:19:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 -- 172.21.15.86:0/2755608712 wait complete. 2024-04-25T19:19:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f893ea00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89240a3d90 0x7f89240a6250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x563c59729f50 tx=0x7f89300bd040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.636+0000 7f894844c740 1 -- start start 2024-04-25T19:19:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 Processor -- start 2024-04-25T19:19:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 0x563c59730b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59731050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c59733b90 0x563c59731590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c597367c0 con 0x563c59733b90 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c59648310 con 0x563c597289b0 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c59736940 con 0x563c597292d0 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.637+0000 7f894844c740 10 client.0 _ll_register_callbacks cb 0x563c59616cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f893f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 0x563c59730b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f8936000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59731050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f893f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 0x563c59730b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:55608/0 (socket says 172.21.15.86:55608) 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f893f400640 1 -- 172.21.15.86:0/730403628 learned_addr learned my addr 172.21.15.86:0/730403628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f893f400640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c59733b90 0x563c59731590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f8936000640 1 -- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 msgr2=0x563c59730b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f8936000640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 0x563c59730b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f8936000640 1 -- 172.21.15.86:0/730403628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c59733b90 msgr2=0x563c59731590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f8936000640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c59733b90 0x563c59731590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f8936000640 1 -- 172.21.15.86:0/730403628 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5946d620 con 0x563c597292d0 2024-04-25T19:19:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.638+0000 7f893f400640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 0x563c59730b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.639+0000 7f8936000640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59731050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8928004770 tx=0x7f892802a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41b30e459e48a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.639+0000 7f8923e00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:19:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.644+0000 7f8923e00640 1 -- 172.21.15.86:0/730403628 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928035040 con 0x563c597292d0 2024-04-25T19:19:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.644+0000 7f8923e00640 1 -- 172.21.15.86:0/730403628 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f8928031860 con 0x563c597292d0 2024-04-25T19:19:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.644+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8938009e50 con 0x563c597292d0 2024-04-25T19:19:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.644+0000 7f8923e00640 1 -- 172.21.15.86:0/730403628 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8928004040 con 0x563c597292d0 2024-04-25T19:19:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 10 client.27199 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c59742a00 con 0x563c597292d0 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f8923e00640 1 -- 172.21.15.86:0/730403628 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8938009e50 con 0x563c597292d0 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f8923e00640 1 -- 172.21.15.86:0/730403628 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f892803d450 con 0x563c597292d0 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 10 client.27199 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 20 client.27199 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c59743f10 0x563c59746300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 4 client.27199 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.645+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c596a28b0 con 0x563c59743f10 2024-04-25T19:19:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.646+0000 7f893f400640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c59743f10 0x563c59746300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.646+0000 7f893f400640 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c59743f10 0x563c59746300 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7711fe2ea0729f9 server_cookie=c8f27c622bc178f0 in_seq=0 out_seq=0 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.646+0000 7f8923e00640 10 client.27199 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.647+0000 7f8923e00640 1 -- 172.21.15.86:0/730403628 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c596a28b0 con 0x563c59743f10 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.647+0000 7f8923e00640 10 client.27199 handle_command_reply: tid=0 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.647+0000 7f894844c740 1 client.27199 shutdown 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.647+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 msgr2=0x563c59731050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.647+0000 7f894844c740 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59731050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8928004770 tx=0x7f892802a9c0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 shutdown_connections 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c59743f10 0x563c59746300 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c59733b90 0x563c59731590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597292d0 0x563c59731050 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 --2- 172.21.15.86:0/730403628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597289b0 0x563c59730b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f8923e00640 0 client.27199 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 >> 172.21.15.86:0/730403628 conn(0x563c59724ec0 msgr2=0x563c5972abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 shutdown_connections 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 1 -- 172.21.15.86:0/730403628 wait complete. 2024-04-25T19:19:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.648+0000 7f894844c740 20 client.27199 trim_cache size 0 max 16384 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.664+0000 7f894844c740 1 Processor -- start 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.664+0000 7f894844c740 1 -- start start 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f894844c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597289b0 0x563c5973f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f894844c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 0x563c5972b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f894844c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c5974bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f894844c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89401b4220 con 0x563c597292d0 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f894844c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89401b5d60 con 0x563c59733b90 2024-04-25T19:19:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f894844c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89401b48f0 con 0x563c597289b0 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 0x563c5972b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 0x563c5972b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49886/0 (socket says 172.21.15.86:49886) 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 -- 172.21.15.86:0/287181403 learned_addr learned my addr 172.21.15.86:0/287181403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f893f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c5974bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f893f400640 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597289b0 0x563c5973f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 -- 172.21.15.86:0/287181403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597289b0 msgr2=0x563c5973f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597289b0 0x563c5973f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 -- 172.21.15.86:0/287181403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 msgr2=0x563c5974bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c5974bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f8936000640 1 -- 172.21.15.86:0/287181403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c59746670 con 0x563c597292d0 2024-04-25T19:19:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.665+0000 7f893f400640 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597289b0 0x563c5973f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.666+0000 7f8936000640 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 0x563c5972b370 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f892803d9e0 tx=0x7f892802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146cac7e83aeeeab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.666+0000 7f892ea00640 1 -- 172.21.15.86:0/287181403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892805b070 con 0x563c597292d0 2024-04-25T19:19:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.666+0000 7f892ea00640 1 -- 172.21.15.86:0/287181403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892803dc60 con 0x563c597292d0 2024-04-25T19:19:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.666+0000 7f892ea00640 1 -- 172.21.15.86:0/287181403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928035030 con 0x563c597292d0 2024-04-25T19:19:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 -- 172.21.15.86:0/287181403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 msgr2=0x563c5972b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 0x563c5972b370 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f892803d9e0 tx=0x7f892802a420 comp rx=0 tx=0).stop 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 -- 172.21.15.86:0/287181403 shutdown_connections 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c59733b90 0x563c5974bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597292d0 0x563c5972b370 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 --2- 172.21.15.86:0/287181403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c597289b0 0x563c5973f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 -- 172.21.15.86:0/287181403 >> 172.21.15.86:0/287181403 conn(0x563c59724ec0 msgr2=0x563c5972bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 -- 172.21.15.86:0/287181403 shutdown_connections 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.667+0000 7f894844c740 1 -- 172.21.15.86:0/287181403 wait complete. 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 -- start start 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 Processor -- start 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c5973b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597292d0 0x563c5973bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c59733b90 0x563c59738d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89401b5690 con 0x563c597289b0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89401b4fc0 con 0x563c597292d0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89401b2220 con 0x563c59733b90 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.668+0000 7f894844c740 10 client.0 _ll_register_callbacks cb 0x7f8928031c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c5973b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c5973b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49898/0 (socket says 172.21.15.86:49898) 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 -- 172.21.15.86:0/3788612289 learned_addr learned my addr 172.21.15.86:0/3788612289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f8936000640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597292d0 0x563c5973bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c59733b90 0x563c59738d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 -- 172.21.15.86:0/3788612289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c59733b90 msgr2=0x563c59738d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c59733b90 0x563c59738d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 -- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597292d0 msgr2=0x563c5973bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597292d0 0x563c5973bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 -- 172.21.15.86:0/3788612289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c59739470 con 0x563c597289b0 2024-04-25T19:19:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f8936000640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597292d0 0x563c5973bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f893f400640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c5973b860 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ab0 tx=0x7f8938011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a658485e428fbcfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.669+0000 7f8923e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f8923e00640 1 -- 172.21.15.86:0/3788612289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893801e040 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f8923e00640 1 -- 172.21.15.86:0/3788612289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f8938003800 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892803dc60 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f8923e00640 1 -- 172.21.15.86:0/3788612289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f893803e050 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 10 client.17118 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c597a9f20 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f8923e00640 1 -- 172.21.15.86:0/3788612289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892803dc60 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f8923e00640 1 -- 172.21.15.86:0/3788612289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f893800ea10 con 0x563c597289b0 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 10 client.17118 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 20 client.17118 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c597aa4e0 0x563c597ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 4 client.17118 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.670+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c5969c320 con 0x563c597aa4e0 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.671+0000 7f8936000640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c597aa4e0 0x563c597ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.671+0000 7f8923e00640 10 client.17118 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.671+0000 7f8936000640 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c597aa4e0 0x563c597ac900 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a2e60166caa066 server_cookie=a697dea0ded533f7 in_seq=0 out_seq=0 2024-04-25T19:19:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.686+0000 7f8923e00640 1 -- 172.21.15.86:0/3788612289 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c5969c320 con 0x563c597aa4e0 2024-04-25T19:19:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.686+0000 7f8923e00640 10 client.17118 handle_command_reply: tid=0 2024-04-25T19:19:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.686+0000 7f894844c740 1 client.17118 shutdown 2024-04-25T19:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.686+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 msgr2=0x563c5973b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.686+0000 7f894844c740 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c5973b860 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ab0 tx=0x7f8938011810 comp rx=0 tx=0).stop 2024-04-25T19:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 shutdown_connections 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563c597aa4e0 0x563c597ac900 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c59733b90 0x563c59738d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c597292d0 0x563c5973bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f8923e00640 0 client.17118 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 --2- 172.21.15.86:0/3788612289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c597289b0 0x563c5973b860 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 >> 172.21.15.86:0/3788612289 conn(0x563c59724ec0 msgr2=0x563c59730ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 shutdown_connections 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 1 -- 172.21.15.86:0/3788612289 wait complete. 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.687+0000 7f894844c740 20 client.17118 trim_cache size 0 max 16384 2024-04-25T19:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:19:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:19:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:19:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89240a3d90 msgr2=0x7f89240a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89240a3d90 0x7f89240a6250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x563c59729f50 tx=0x7f89300bd040 comp rx=0 tx=0).stop 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108490 msgr2=0x7f89401a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108490 0x7f89401a47c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f893000e9f0 tx=0x7f893000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 shutdown_connections 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89240a3d90 0x7f89240a6250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f894010e420 0x7f894019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8940108e10 0x7f89401a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 --2- 172.21.15.86:0/3679269468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8940108490 0x7f89401a47c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 >> 172.21.15.86:0/3679269468 conn(0x7f8940077d30 msgr2=0x7f8940078f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 shutdown_connections 2024-04-25T19:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:16.688+0000 7f893fe00640 1 -- 172.21.15.86:0/3679269468 wait complete. 2024-04-25T19:19:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:16 smithi183 ceph-mon[32344]: pgmap v514: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T19:19:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:16 smithi194 ceph-mon[32740]: pgmap v514: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T19:19:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:16 smithi086 ceph-mon[27783]: pgmap v514: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-25T19:19:18.903 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:18 smithi086 ceph-mon[27783]: pgmap v515: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T19:19:18.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:18 smithi183 ceph-mon[32344]: pgmap v515: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T19:19:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:18 smithi194 ceph-mon[32740]: pgmap v515: 97 pgs: 97 active+clean; 3.3 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T19:19:20.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:20 smithi086 ceph-mon[27783]: pgmap v516: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:19:20.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:20 smithi183 ceph-mon[32344]: pgmap v516: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:19:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:20 smithi194 ceph-mon[32740]: pgmap v516: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:19:22.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:22 smithi183 ceph-mon[32344]: pgmap v517: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:19:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:22 smithi194 ceph-mon[32740]: pgmap v517: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:19:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:22 smithi086 ceph-mon[27783]: pgmap v517: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-25T19:19:24.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:24 smithi183 ceph-mon[32344]: pgmap v518: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T19:19:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:24 smithi194 ceph-mon[32740]: pgmap v518: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T19:19:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:24 smithi086 ceph-mon[27783]: pgmap v518: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T19:19:26.528 INFO:tasks.workunit.client.0.smithi086.stdout:499/600: open fd = 3 2024-04-25T19:19:26.528 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:500/600: open fd = 3 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:501/600: open fd = 3 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:502/600: open fd = 3 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.529 INFO:tasks.workunit.client.0.smithi086.stdout:503/600: open fd = 3 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:504/600: open fd = 3 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:505/600: open fd = 3 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:506/600: open fd = 3 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.530 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:507/600: open fd = 3 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:508/600: open fd = 3 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:509/600: open fd = 3 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:510/600: open fd = 3 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.531 INFO:tasks.workunit.client.0.smithi086.stdout:511/600: open fd = 3 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:512/600: open fd = 3 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:513/600: open fd = 3 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:514/600: open fd = 3 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:515/600: open fd = 3 2024-04-25T19:19:26.532 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:516/600: open fd = 3 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:517/600: open fd = 3 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:518/600: open fd = 3 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.533 INFO:tasks.workunit.client.0.smithi086.stdout:519/600: open fd = 3 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:520/600: open fd = 3 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:521/600: open fd = 3 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:522/600: open fd = 3 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.534 INFO:tasks.workunit.client.0.smithi086.stdout:523/600: open fd = 3 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:524/600: open fd = 3 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:525/600: open fd = 3 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:526/600: open fd = 3 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.535 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:527/600: open fd = 3 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:528/600: open fd = 3 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:529/600: open fd = 3 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.536 INFO:tasks.workunit.client.0.smithi086.stdout:530/600: open fd = 3 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:531/600: open fd = 3 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:532/600: open fd = 3 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:533/600: open fd = 3 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.537 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:534/600: open fd = 3 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:535/600: open fd = 3 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:536/600: open fd = 3 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.538 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:537/600: open fd = 3 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:538/600: open fd = 3 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:539/600: open fd = 3 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:540/600: open fd = 3 2024-04-25T19:19:26.539 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:541/600: open fd = 3 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:542/600: open fd = 3 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:543/600: open fd = 3 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.540 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:544/600: open fd = 3 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:545/600: open fd = 3 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:546/600: open fd = 3 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.541 INFO:tasks.workunit.client.0.smithi086.stdout:547/600: open fd = 3 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:548/600: open fd = 3 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:549/600: open fd = 3 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:550/600: open fd = 3 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.542 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:551/600: open fd = 3 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:552/600: open fd = 3 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:553/600: open fd = 3 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:554/600: open fd = 3 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:555/600: open fd = 3 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:556/600: open fd = 3 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:557/600: open fd = 3 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:558/600: open fd = 3 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:559/600: open fd = 3 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:560/600: open fd = 3 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:19:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:19:26.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:26 smithi183 ceph-mon[32344]: pgmap v519: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:19:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:26 smithi194 ceph-mon[32740]: pgmap v519: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:19:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:26 smithi086 ceph-mon[27783]: pgmap v519: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T19:19:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:28 smithi194 ceph-mon[32740]: pgmap v520: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:19:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:28 smithi086 ceph-mon[27783]: pgmap v520: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:19:29.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:28 smithi183 ceph-mon[32344]: pgmap v520: 97 pgs: 97 active+clean; 3.6 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 -- 172.21.15.86:0/1744407046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c810e600 msgr2=0x7f80c8110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 --2- 172.21.15.86:0/1744407046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c810e600 0x7f80c8110a10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009ca0 tx=0x7f80c002a940 comp rx=0 tx=0).stop 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 -- 172.21.15.86:0/1744407046 shutdown_connections 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 --2- 172.21.15.86:0/1744407046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c810e600 0x7f80c8110a10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 --2- 172.21.15.86:0/1744407046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c8074c40 0x7f80c810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 --2- 172.21.15.86:0/1744407046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80c8074320 0x7f80c8074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.829+0000 7f80cdc00640 1 -- 172.21.15.86:0/1744407046 >> 172.21.15.86:0/1744407046 conn(0x7f80c80fd890 msgr2=0x7f80c80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.830+0000 7f80cdc00640 1 -- 172.21.15.86:0/1744407046 shutdown_connections 2024-04-25T19:19:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.830+0000 7f80cdc00640 1 -- 172.21.15.86:0/1744407046 wait complete. 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.830+0000 7f80cdc00640 1 Processor -- start 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 -- start start 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c8074320 0x7f80c81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 0x7f80c81a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80c810e600 0x7f80c819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81141f0 con 0x7f80c8074c40 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8114070 con 0x7f80c8074320 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cdc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8114370 con 0x7f80c810e600 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 0x7f80c81a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 0x7f80c81a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37790/0 (socket says 172.21.15.86:37790) 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80c7400640 1 -- 172.21.15.86:0/3694654584 learned_addr learned my addr 172.21.15.86:0/3694654584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80c7e00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c8074320 0x7f80c81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.831+0000 7f80cd200640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80c810e600 0x7f80c819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c7400640 1 -- 172.21.15.86:0/3694654584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80c810e600 msgr2=0x7f80c819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c7400640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80c810e600 0x7f80c819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c7400640 1 -- 172.21.15.86:0/3694654584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c8074320 msgr2=0x7f80c81a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c7400640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c8074320 0x7f80c81a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c7400640 1 -- 172.21.15.86:0/3694654584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c819ad50 con 0x7f80c8074c40 2024-04-25T19:19:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c7400640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 0x7f80c81a0960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e9f0 tx=0x7f80b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c4c00640 1 -- 172.21.15.86:0/3694654584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b400cd80 con 0x7f80c8074c40 2024-04-25T19:19:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c4c00640 1 -- 172.21.15.86:0/3694654584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b40108e0 con 0x7f80c8074c40 2024-04-25T19:19:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c819b040 con 0x7f80c8074c40 2024-04-25T19:19:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.832+0000 7f80c4c00640 1 -- 172.21.15.86:0/3694654584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4018980 con 0x7f80c8074c40 2024-04-25T19:19:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.833+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c81a72e0 con 0x7f80c8074c40 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.836+0000 7f80d1bfd740 1 Processor -- start 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.836+0000 7f80d1bfd740 1 -- start start 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.836+0000 7f80c4c00640 1 -- 172.21.15.86:0/3694654584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80b4010430 con 0x7f80c8074c40 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cca4311900 con 0x55cca43b6b90 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cca42b7ef0 con 0x55cca43ac2d0 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cca430d8c0 con 0x55cca43ab9b0 2024-04-25T19:19:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:40958/0 (socket says 172.21.15.86:40958) 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 -- 172.21.15.86:0/343819290 learned_addr learned my addr 172.21.15.86:0/343819290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80c7400640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80c7e00640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 -- 172.21.15.86:0/343819290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 msgr2=0x55cca43b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 -- 172.21.15.86:0/343819290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 msgr2=0x55cca43b8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80cd200640 1 -- 172.21.15.86:0/343819290 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca4024210 con 0x55cca43ab9b0 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80c7e00640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.837+0000 7f80c7400640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.838+0000 7f80cd200640 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43abd90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f80c8075270 tx=0x7f80c00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=355826528718055d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.838+0000 7f80bd600640 1 -- 172.21.15.86:0/343819290 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0034040 con 0x55cca43ab9b0 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.838+0000 7f80bd600640 1 -- 172.21.15.86:0/343819290 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c00098e0 con 0x55cca43ab9b0 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.838+0000 7f80bd600640 1 -- 172.21.15.86:0/343819290 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0003d70 con 0x55cca43ab9b0 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.838+0000 7f80c4c00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f80ac0a3d10 0x7f80ac0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.838+0000 7f80c4c00640 1 -- 172.21.15.86:0/3694654584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f80b40d0b30 con 0x7f80c8074c40 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80c7e00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f80ac0a3d10 0x7f80ac0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 -- 172.21.15.86:0/343819290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 msgr2=0x55cca43abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43abd90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f80c8075270 tx=0x7f80c00093e0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 -- 172.21.15.86:0/343819290 shutdown_connections 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/343819290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43abd90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 -- 172.21.15.86:0/343819290 >> 172.21.15.86:0/343819290 conn(0x55cca43a7ec0 msgr2=0x55cca43a82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80c7e00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f80ac0a3d10 0x7f80ac0a61d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55cca43acf50 tx=0x7f80b8005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 -- 172.21.15.86:0/343819290 shutdown_connections 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.839+0000 7f80d1bfd740 1 -- 172.21.15.86:0/343819290 wait complete. 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 -- start start 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 Processor -- start 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cca43b9760 con 0x55cca43b6b90 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cca42cb310 con 0x55cca43ac2d0 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cca43b98e0 con 0x55cca43ab9b0 2024-04-25T19:19:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.840+0000 7f80d1bfd740 10 client.0 _ll_register_callbacks cb 0x55cca4299cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37808/0 (socket says 172.21.15.86:37808) 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 -- 172.21.15.86:0/1378066552 learned_addr learned my addr 172.21.15.86:0/1378066552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80c7e00640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80c7400640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 -- 172.21.15.86:0/1378066552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 msgr2=0x55cca43b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 -- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 msgr2=0x55cca43b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 -- 172.21.15.86:0/1378066552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca40f0620 con 0x55cca43b6b90 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80cd200640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b4570 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80c002ae60 tx=0x7f80c00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92678601cb82244c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.841+0000 7f80b2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80b2000640 1 -- 172.21.15.86:0/1378066552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0051060 con 0x55cca43b6b90 2024-04-25T19:19:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80b2000640 1 -- 172.21.15.86:0/1378066552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f80c0030850 con 0x55cca43b6b90 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c00098e0 con 0x55cca43b6b90 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80b2000640 1 -- 172.21.15.86:0/1378066552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80c0034070 con 0x55cca43b6b90 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80d1bfd740 10 client.17130 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cca43c5980 con 0x55cca43b6b90 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.842+0000 7f80b2000640 1 -- 172.21.15.86:0/1378066552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c00098e0 con 0x55cca43b6b90 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80b2000640 1 -- 172.21.15.86:0/1378066552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f80c002f070 con 0x55cca43b6b90 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80d1bfd740 10 client.17130 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80d1bfd740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80d1bfd740 20 client.17130 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca43c6cd0 0x55cca43c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80d1bfd740 4 client.17130 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cca43258b0 con 0x55cca43c6cd0 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.843+0000 7f80c7e00640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca43c6cd0 0x55cca43c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.844+0000 7f80c7e00640 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca43c6cd0 0x55cca43c90c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1291e314065628 server_cookie=d1848a43e3142b0f in_seq=0 out_seq=0 2024-04-25T19:19:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.844+0000 7f80b2000640 10 client.17130 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.844+0000 7f80b2000640 1 -- 172.21.15.86:0/1378066552 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cca43258b0 con 0x55cca43c6cd0 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.844+0000 7f80b2000640 10 client.17130 handle_command_reply: tid=0 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 client.17130 shutdown 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 msgr2=0x55cca43b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b4570 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80c002ae60 tx=0x7f80c00095e0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 shutdown_connections 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca43c6cd0 0x55cca43c90c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43b4570 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1378066552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80b2000640 0 client.17130 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 >> 172.21.15.86:0/1378066552 conn(0x55cca43a7ec0 msgr2=0x55cca43adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 shutdown_connections 2024-04-25T19:19:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1378066552 wait complete. 2024-04-25T19:19:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.845+0000 7f80d1bfd740 20 client.17130 trim_cache size 0 max 16384 2024-04-25T19:19:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 Processor -- start 2024-04-25T19:19:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 -- start start 2024-04-25T19:19:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81afd20 con 0x55cca43b6b90 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b1860 con 0x55cca43ac2d0 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.861+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b03f0 con 0x55cca43ab9b0 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80cd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52780/0 (socket says 172.21.15.86:52780) 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:42716/0 (socket says 172.21.15.86:42716) 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80cd200640 1 -- 172.21.15.86:0/3786749285 learned_addr learned my addr 172.21.15.86:0/3786749285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7400640 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 -- 172.21.15.86:0/3786749285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 msgr2=0x55cca43b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 -- 172.21.15.86:0/3786749285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 msgr2=0x55cca43ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 -- 172.21.15.86:0/3786749285 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca43a7730 con 0x55cca43ab9b0 2024-04-25T19:19:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.862+0000 7f80c7e00640 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43c2620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55cca43b98e0 tx=0x7f80b8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d08c94383886981 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80bcc00640 1 -- 172.21.15.86:0/3786749285 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b801dbb0 con 0x55cca43ab9b0 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80bcc00640 1 -- 172.21.15.86:0/3786749285 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b801dd50 con 0x55cca43ab9b0 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80bcc00640 1 -- 172.21.15.86:0/3786749285 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003e20 con 0x55cca43ab9b0 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 -- 172.21.15.86:0/3786749285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 msgr2=0x55cca43c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43c2620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55cca43b98e0 tx=0x7f80b8019000 comp rx=0 tx=0).stop 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 -- 172.21.15.86:0/3786749285 shutdown_connections 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43b6b90 0x55cca43ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43ac2d0 0x55cca43b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/3786749285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ab9b0 0x55cca43c2620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 -- 172.21.15.86:0/3786749285 >> 172.21.15.86:0/3786749285 conn(0x55cca43a7ec0 msgr2=0x55cca43a7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 -- 172.21.15.86:0/3786749285 shutdown_connections 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.863+0000 7f80d1bfd740 1 -- 172.21.15.86:0/3786749285 wait complete. 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 -- start start 2024-04-25T19:19:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 Processor -- start 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 0x55cca43bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ac2d0 0x55cca43bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43b6b90 0x55cca43bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b1190 con 0x55cca43ab9b0 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b0ac0 con 0x55cca43b6b90 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.864+0000 7f80d1bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81add20 con 0x55cca43ac2d0 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80d1bfd740 10 client.0 _ll_register_callbacks cb 0x7f80c002f860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 0x55cca43bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 0x55cca43bf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52784/0 (socket says 172.21.15.86:52784) 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 -- 172.21.15.86:0/1315519252 learned_addr learned my addr 172.21.15.86:0/1315519252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80cd200640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43b6b90 0x55cca43bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7400640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ac2d0 0x55cca43bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 -- 172.21.15.86:0/1315519252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ac2d0 msgr2=0x55cca43bc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ac2d0 0x55cca43bc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 -- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43b6b90 msgr2=0x55cca43bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43b6b90 0x55cca43bcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 -- 172.21.15.86:0/1315519252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca442bf70 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80c7e00640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 0x55cca43bf0c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f80b801efd0 tx=0x7f80b801dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a416e6012a1247 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.865+0000 7f80b2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80b2000640 1 -- 172.21.15.86:0/1315519252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003770 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80b2000640 1 -- 172.21.15.86:0/1315519252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f80b8005040 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b801dd50 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80b2000640 1 -- 172.21.15.86:0/1315519252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80b8041050 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80d1bfd740 10 client.17136 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cca442d140 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80b2000640 1 -- 172.21.15.86:0/1315519252 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b801dd50 con 0x55cca43ab9b0 2024-04-25T19:19:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80b2000640 1 -- 172.21.15.86:0/1315519252 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f80b802b070 con 0x55cca43ab9b0 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80d1bfd740 10 client.17136 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80d1bfd740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.866+0000 7f80d1bfd740 20 client.17136 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.867+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca442d850 0x55cca442fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.867+0000 7f80d1bfd740 4 client.17136 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.867+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cca431f320 con 0x55cca442d850 2024-04-25T19:19:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.867+0000 7f80c7400640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca442d850 0x55cca442fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.867+0000 7f80c7400640 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca442d850 0x55cca442fc70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473f3e97f5512f9e server_cookie=3ec1520210d9bfe5 in_seq=0 out_seq=0 2024-04-25T19:19:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.867+0000 7f80b2000640 10 client.17136 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.881+0000 7f80b2000640 1 -- 172.21.15.86:0/1315519252 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cca431f320 con 0x55cca442d850 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.881+0000 7f80b2000640 10 client.17136 handle_command_reply: tid=0 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.881+0000 7f80d1bfd740 1 client.17136 shutdown 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.881+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 msgr2=0x55cca43bf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.881+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 0x55cca43bf0c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f80b801efd0 tx=0x7f80b801dcc0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 shutdown_connections 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cca442d850 0x55cca442fc70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca43b6b90 0x55cca43bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cca43ac2d0 0x55cca43bc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 --2- 172.21.15.86:0/1315519252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cca43ab9b0 0x55cca43bf0c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80b2000640 0 client.17136 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 >> 172.21.15.86:0/1315519252 conn(0x55cca43a7ec0 msgr2=0x55cca43ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 shutdown_connections 2024-04-25T19:19:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 1 -- 172.21.15.86:0/1315519252 wait complete. 2024-04-25T19:19:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.882+0000 7f80d1bfd740 20 client.17136 trim_cache size 0 max 16384 2024-04-25T19:19:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:19:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:19:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:19:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:19:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f80ac0a3d10 msgr2=0x7f80ac0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f80ac0a3d10 0x7f80ac0a61d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55cca43acf50 tx=0x7f80b8005f90 comp rx=0 tx=0).stop 2024-04-25T19:19:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 msgr2=0x7f80c81a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 0x7f80c81a0960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e9f0 tx=0x7f80b400eeb0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 shutdown_connections 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f80ac0a3d10 0x7f80ac0a61d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80c810e600 0x7f80c819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8074c40 0x7f80c81a0960 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 --2- 172.21.15.86:0/3694654584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80c8074320 0x7f80c81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 >> 172.21.15.86:0/3694654584 conn(0x7f80c80fd890 msgr2=0x7f80c80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 shutdown_connections 2024-04-25T19:19:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:29.883+0000 7f80cdc00640 1 -- 172.21.15.86:0/3694654584 wait complete. 2024-04-25T19:19:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:30 smithi194 ceph-mon[32740]: pgmap v521: 97 pgs: 97 active+clean; 3.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T19:19:31.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:30 smithi086 ceph-mon[27783]: pgmap v521: 97 pgs: 97 active+clean; 3.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T19:19:31.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:30 smithi183 ceph-mon[32344]: pgmap v521: 97 pgs: 97 active+clean; 3.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T19:19:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:32 smithi194 ceph-mon[32740]: pgmap v522: 97 pgs: 97 active+clean; 3.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:19:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:32 smithi086 ceph-mon[27783]: pgmap v522: 97 pgs: 97 active+clean; 3.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:19:33.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:32 smithi183 ceph-mon[32344]: pgmap v522: 97 pgs: 97 active+clean; 3.8 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T19:19:33.877 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-25T19:19:33.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:19:34.331 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:19:34.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:34 smithi086 ceph-mon[27783]: pgmap v523: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:19:34.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:34 smithi194 ceph-mon[32740]: pgmap v523: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:19:35.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:34 smithi183 ceph-mon[32344]: pgmap v523: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T19:19:35.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:35 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:19:36.067 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:35 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:19:36.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:35 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:19:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 -- 172.21.15.86:0/4194531172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 msgr2=0x7f4278115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 --2- 172.21.15.86:0/4194531172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 0x7f4278115f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f427000b0d0 tx=0x7f427000b3e0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 -- 172.21.15.86:0/4194531172 shutdown_connections 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 --2- 172.21.15.86:0/4194531172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 0x7f4278115f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 --2- 172.21.15.86:0/4194531172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42781092d0 0x7f42781096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 --2- 172.21.15.86:0/4194531172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4278109c80 0x7f427810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 -- 172.21.15.86:0/4194531172 >> 172.21.15.86:0/4194531172 conn(0x7f4278076210 msgr2=0x7f4278107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.744+0000 7f4280600700 1 -- 172.21.15.86:0/4194531172 shutdown_connections 2024-04-25T19:19:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.745+0000 7f4280600700 1 -- 172.21.15.86:0/4194531172 wait complete. 2024-04-25T19:19:36.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.745+0000 7f4280600700 1 Processor -- start 2024-04-25T19:19:36.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 -- start start 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 0x7f427819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4278109c80 0x7f427819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 0x7f42781a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4278119250 con 0x7f4278109c80 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42781190d0 con 0x7f42781092d0 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f4280600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42781193d0 con 0x7f427810e3b0 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.746+0000 7f427dc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 0x7f427819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 0x7f427819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:58106/0 (socket says 172.21.15.86:58106) 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 -- 172.21.15.86:0/2978416122 learned_addr learned my addr 172.21.15.86:0/2978416122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:36.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427e600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 0x7f42781a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 -- 172.21.15.86:0/2978416122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 msgr2=0x7f42781a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 0x7f42781a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 -- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4278109c80 msgr2=0x7f427819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4278109c80 0x7f427819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 -- 172.21.15.86:0/2978416122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42781a46e0 con 0x7f42781092d0 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.747+0000 7f427dc00700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 0x7f427819f720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f426c00eb30 tx=0x7f426c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.748+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c00cc90 con 0x7f42781092d0 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.748+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781a49d0 con 0x7f42781092d0 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.748+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f426c0044d0 con 0x7f42781092d0 2024-04-25T19:19:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.748+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c010430 con 0x7f42781092d0 2024-04-25T19:19:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.748+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781ac250 con 0x7f42781092d0 2024-04-25T19:19:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.749+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4278069730 con 0x7f42781092d0 2024-04-25T19:19:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.751+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f426c003ea0 con 0x7f42781092d0 2024-04-25T19:19:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.752+0000 7f4276000700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f42600a3db0 0x7f42600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.752+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f426c0d0b50 con 0x7f42781092d0 2024-04-25T19:19:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.752+0000 7f427d200700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f42600a3db0 0x7f42600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.753+0000 7f427d200700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f42600a3db0 0x7f42600a6270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4278108980 tx=0x7f4264009620 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:36.757+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f426c09d120 con 0x7f42781092d0 2024-04-25T19:19:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:36 smithi194 ceph-mon[32740]: pgmap v524: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:19:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:36 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:19:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:36 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:19:36.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:36 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:19:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.030+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f427802df80 con 0x7f42781092d0 2024-04-25T19:19:37.032 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:36 smithi086 ceph-mon[27783]: pgmap v524: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:19:37.032 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:36 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:19:37.032 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:36 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:19:37.032 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:36 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:19:37.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.031+0000 7f4276000700 1 -- 172.21.15.86:0/2978416122 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f426c09ca20 con 0x7f42781092d0 2024-04-25T19:19:37.033 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:19:37.033 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f42600a3db0 msgr2=0x7f42600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f42600a3db0 0x7f42600a6270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4278108980 tx=0x7f4264009620 comp rx=0 tx=0).stop 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 msgr2=0x7f427819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 0x7f427819f720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f426c00eb30 tx=0x7f426c00ee40 comp rx=0 tx=0).stop 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 shutdown_connections 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f42600a3db0 0x7f42600a6270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f427810e3b0 0x7f42781a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42781092d0 0x7f427819f720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 --2- 172.21.15.86:0/2978416122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4278109c80 0x7f427819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.034+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 >> 172.21.15.86:0/2978416122 conn(0x7f4278076210 msgr2=0x7f4278076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.035+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 shutdown_connections 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:37.035+0000 7f4280600700 1 -- 172.21.15.86:0/2978416122 wait complete. 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:19:37.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:36 smithi183 ceph-mon[32344]: pgmap v524: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:19:37.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:36 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:19:37.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:36 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:19:37.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:36 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:19:37.913 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-25T19:19:37.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-25T19:19:37.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:37 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2978416122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:37.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:37 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2978416122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:38.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:37 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2978416122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:38.362 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:19:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:38 smithi086 ceph-mon[27783]: pgmap v525: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:19:38.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:38 smithi194 ceph-mon[32740]: pgmap v525: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:19:39.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:38 smithi183 ceph-mon[32344]: pgmap v525: 97 pgs: 97 active+clean; 4.1 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T19:19:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 -- 172.21.15.86:0/2124690886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 msgr2=0x7f462c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 --2- 172.21.15.86:0/2124690886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 0x7f462c10ef60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4618009990 tx=0x7f4618009ca0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 -- 172.21.15.86:0/2124690886 shutdown_connections 2024-04-25T19:19:40.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 --2- 172.21.15.86:0/2124690886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c10f4a0 0x7f462c111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 --2- 172.21.15.86:0/2124690886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c074c70 0x7f462c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 --2- 172.21.15.86:0/2124690886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 0x7f462c10ef60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 -- 172.21.15.86:0/2124690886 >> 172.21.15.86:0/2124690886 conn(0x7f462c079b10 msgr2=0x7f462c079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 -- 172.21.15.86:0/2124690886 shutdown_connections 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.698+0000 7f4633000700 1 -- 172.21.15.86:0/2124690886 wait complete. 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 Processor -- start 2024-04-25T19:19:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 -- start start 2024-04-25T19:19:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 0x7f462c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 0x7f462c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c10f4a0 0x7f462c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f462c114e10 con 0x7f462c074c70 2024-04-25T19:19:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f462c114c90 con 0x7f462c10f4a0 2024-04-25T19:19:40.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.700+0000 7f4633000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f462c114f90 con 0x7f462c075590 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 0x7f462c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 0x7f462c19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52732/0 (socket says 172.21.15.86:52732) 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 -- 172.21.15.86:0/1986051712 learned_addr learned my addr 172.21.15.86:0/1986051712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462b400700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 0x7f462c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f4631000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c10f4a0 0x7f462c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 -- 172.21.15.86:0/1986051712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 msgr2=0x7f462c19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 0x7f462c19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 -- 172.21.15.86:0/1986051712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c10f4a0 msgr2=0x7f462c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c10f4a0 0x7f462c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.701+0000 7f462be00700 1 -- 172.21.15.86:0/1986051712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1a0270 con 0x7f462c074c70 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.702+0000 7f462be00700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 0x7f462c19b260 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f461c00e9c0 tx=0x7f461c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.702+0000 7f4628c00700 1 -- 172.21.15.86:0/1986051712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c00cbd0 con 0x7f462c074c70 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.702+0000 7f4628c00700 1 -- 172.21.15.86:0/1986051712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c00ee60 con 0x7f462c074c70 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.702+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1a0560 con 0x7f462c074c70 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.702+0000 7f4628c00700 1 -- 172.21.15.86:0/1986051712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0055c0 con 0x7f462c074c70 2024-04-25T19:19:40.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.702+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1a7de0 con 0x7f462c074c70 2024-04-25T19:19:40.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.707+0000 7f4628c00700 1 -- 172.21.15.86:0/1986051712 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f461c004750 con 0x7f462c074c70 2024-04-25T19:19:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.708+0000 7f463d898b80 1 Processor -- start 2024-04-25T19:19:40.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.708+0000 7f463d898b80 1 -- start start 2024-04-25T19:19:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.708+0000 7f4628c00700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f46100a3e50 0x7f46100a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f463d898b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 0x5591bee45d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f4628c00700 1 -- 172.21.15.86:0/1986051712 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f461c0d0b90 con 0x7f462c074c70 2024-04-25T19:19:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f463d898b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee50c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f463d898b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee51180 0x5591bee53570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f463d898b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591bed99020 con 0x5591bee462d0 2024-04-25T19:19:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f463d898b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591bed3f8c0 con 0x5591bee51180 2024-04-25T19:19:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f463d898b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5591bed94fe0 con 0x5591bee459b0 2024-04-25T19:19:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f4631000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f46100a3e50 0x7f46100a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f462be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 0x5591bee45d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f462b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee50c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f462be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 0x5591bee45d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58646/0 (socket says 172.21.15.86:58646) 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.709+0000 7f462be00700 1 -- 172.21.15.86:0/2235125559 learned_addr learned my addr 172.21.15.86:0/2235125559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f462be00700 1 -- 172.21.15.86:0/2235125559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee51180 msgr2=0x5591bee53570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f462be00700 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee51180 0x5591bee53570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f462be00700 1 -- 172.21.15.86:0/2235125559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 msgr2=0x5591bee50c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f462be00700 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee50c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f462be00700 1 -- 172.21.15.86:0/2235125559 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bed35810 con 0x5591bee459b0 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f462be00700 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 0x5591bee45d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f461c098ec0 tx=0x7f461c09c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41edf773d46aec1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f4631000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f46100a3e50 0x7f46100a6310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f462400f540 tx=0x7f4624009920 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:40.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f4621600700 1 -- 172.21.15.86:0/2235125559 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0bd0d0 con 0x5591bee459b0 2024-04-25T19:19:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f4621600700 1 -- 172.21.15.86:0/2235125559 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c09ca60 con 0x5591bee459b0 2024-04-25T19:19:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.710+0000 7f4621600700 1 -- 172.21.15.86:0/2235125559 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c09cf20 con 0x5591bee459b0 2024-04-25T19:19:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.711+0000 7f463d898b80 1 -- 172.21.15.86:0/2235125559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 msgr2=0x5591bee45d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.711+0000 7f463d898b80 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 0x5591bee45d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f461c098ec0 tx=0x7f461c09c690 comp rx=0 tx=0).stop 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 -- 172.21.15.86:0/2235125559 shutdown_connections 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee51180 0x5591bee53570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee459b0 0x5591bee45d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 --2- 172.21.15.86:0/2235125559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee50c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 -- 172.21.15.86:0/2235125559 >> 172.21.15.86:0/2235125559 conn(0x5591bed74730 msgr2=0x5591bee41e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 -- 172.21.15.86:0/2235125559 shutdown_connections 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 -- 172.21.15.86:0/2235125559 wait complete. 2024-04-25T19:19:40.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.712+0000 7f463d898b80 1 -- start start 2024-04-25T19:19:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 Processor -- start 2024-04-25T19:19:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee50950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee462d0 0x5591bee4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee51180 0x5591bee4e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591bee53ca0 con 0x5591bee51180 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591bed529f0 con 0x5591bee459b0 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5591bee53e20 con 0x5591bee462d0 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f463d898b80 10 client.0 _ll_register_callbacks cb 0x5591bed318b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.713+0000 7f462be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee50950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee462d0 0x5591bee4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee50950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40598/0 (socket says 172.21.15.86:40598) 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee462d0 0x5591bee4dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:58650/0 (socket says 172.21.15.86:58650) 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 -- 172.21.15.86:0/3529041709 learned_addr learned my addr 172.21.15.86:0/3529041709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f4631000700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee51180 0x5591bee4e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 -- 172.21.15.86:0/3529041709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee462d0 msgr2=0x5591bee4dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee462d0 0x5591bee4dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 -- 172.21.15.86:0/3529041709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee51180 msgr2=0x5591bee4e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee51180 0x5591bee4e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 -- 172.21.15.86:0/3529041709 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bece6a60 con 0x5591bee459b0 2024-04-25T19:19:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f4631000700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee51180 0x5591bee4e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f462be00700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee50950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f461c098e70 tx=0x7f461c0ce180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342b6535c9569c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.714+0000 7f4616000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.715+0000 7f4616000700 1 -- 172.21.15.86:0/3529041709 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0a56d0 con 0x5591bee459b0 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.716+0000 7f4616000700 1 -- 172.21.15.86:0/3529041709 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f461c067380 con 0x5591bee459b0 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.716+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c09ca60 con 0x5591bee459b0 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.716+0000 7f4616000700 1 -- 172.21.15.86:0/3529041709 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f461c067580 con 0x5591bee459b0 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.716+0000 7f463d898b80 10 client.27242 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591bed2fc80 con 0x5591bee459b0 2024-04-25T19:19:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f4616000700 1 -- 172.21.15.86:0/3529041709 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c09ca60 con 0x5591bee459b0 2024-04-25T19:19:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f4616000700 1 -- 172.21.15.86:0/3529041709 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f461c0987b0 con 0x5591bee459b0 2024-04-25T19:19:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 10 client.27242 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 10 client.27242 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:19:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 20 client.27242 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591bee62c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 4 client.27242 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:40.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.717+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591beda6810 con 0x5591bee608a0 2024-04-25T19:19:40.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.718+0000 7f462b400700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591bee62c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.718+0000 7f462b400700 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591bee62c90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad6b36affe2fe7b server_cookie=f4e46994ac114fc2 in_seq=0 out_seq=0 2024-04-25T19:19:40.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.718+0000 7f4616000700 10 client.27242 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:40.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.720+0000 7f4616000700 1 -- 172.21.15.86:0/3529041709 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591beda6810 con 0x5591bee608a0 2024-04-25T19:19:40.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.720+0000 7f4616000700 10 client.27242 handle_command_reply: tid=0 2024-04-25T19:19:40.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 client.27242 shutdown 2024-04-25T19:19:40.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 msgr2=0x5591bee50950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee50950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f461c098e70 tx=0x7f461c0ce180 comp rx=0 tx=0).stop 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 shutdown_connections 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591bee62c90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee51180 0x5591bee4e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f4616000700 0 client.27242 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee50950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 --2- 172.21.15.86:0/3529041709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee462d0 0x5591bee4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.721+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 >> 172.21.15.86:0/3529041709 conn(0x5591bed74730 msgr2=0x5591bee41a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.722+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 shutdown_connections 2024-04-25T19:19:40.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.722+0000 7f463d898b80 1 -- 172.21.15.86:0/3529041709 wait complete. 2024-04-25T19:19:40.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.722+0000 7f463d898b80 20 client.27242 trim_cache size 0 max 16384 2024-04-25T19:19:40.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 Processor -- start 2024-04-25T19:19:40.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 -- start start 2024-04-25T19:19:40.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee4fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee4af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee4b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1af610 con 0x5591bee462d0 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1b1150 con 0x5591bee459b0 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.745+0000 7f463d898b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1afce0 con 0x5591bee51180 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee4af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee4fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee4af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52756/0 (socket says 172.21.15.86:52756) 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462b400700 1 -- 172.21.15.86:0/2795679528 learned_addr learned my addr 172.21.15.86:0/2795679528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f4631000700 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee4b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 -- 172.21.15.86:0/2795679528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 msgr2=0x5591bee4b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee4b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 -- 172.21.15.86:0/2795679528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 msgr2=0x5591bee4af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee4af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 -- 172.21.15.86:0/2795679528 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bee40ba0 con 0x5591bee459b0 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462b400700 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.746+0000 7f462be00700 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee4fb90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f461c098ec0 tx=0x7f461c09c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ce324b2759a0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.747+0000 7f4620c00700 1 -- 172.21.15.86:0/2795679528 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0a56d0 con 0x5591bee459b0 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.747+0000 7f4620c00700 1 -- 172.21.15.86:0/2795679528 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c09ca60 con 0x5591bee459b0 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.747+0000 7f4620c00700 1 -- 172.21.15.86:0/2795679528 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0987b0 con 0x5591bee459b0 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 -- 172.21.15.86:0/2795679528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 msgr2=0x5591bee4fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee4fb90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f461c098ec0 tx=0x7f461c09c5a0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 -- 172.21.15.86:0/2795679528 shutdown_connections 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee4b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee459b0 0x5591bee4fb90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 --2- 172.21.15.86:0/2795679528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee462d0 0x5591bee4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 -- 172.21.15.86:0/2795679528 >> 172.21.15.86:0/2795679528 conn(0x5591bed74730 msgr2=0x5591bee4f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 -- 172.21.15.86:0/2795679528 shutdown_connections 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 -- 172.21.15.86:0/2795679528 wait complete. 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.748+0000 7f463d898b80 1 -- start start 2024-04-25T19:19:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 Processor -- start 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee459b0 0x5591bee66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee462d0 0x5591bee64510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee64b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1b0a80 con 0x5591bee459b0 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1b03b0 con 0x5591bee462d0 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ad5d0 con 0x5591bee51180 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f463d898b80 10 client.0 _ll_register_callbacks cb 0x5591bed318b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f462be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee459b0 0x5591bee66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f462be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee459b0 0x5591bee66160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52772/0 (socket says 172.21.15.86:52772) 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f4631000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee64b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f462be00700 1 -- 172.21.15.86:0/995693337 learned_addr learned my addr 172.21.15.86:0/995693337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.749+0000 7f462b400700 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee462d0 0x5591bee64510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4631000700 1 -- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee462d0 msgr2=0x5591bee64510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4631000700 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee462d0 0x5591bee64510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4631000700 1 -- 172.21.15.86:0/995693337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee459b0 msgr2=0x5591bee66160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4631000700 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee459b0 0x5591bee66160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4631000700 1 -- 172.21.15.86:0/995693337 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bee65320 con 0x5591bee51180 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4631000700 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee64b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5591bee424e0 tx=0x7f4624004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6746da1c1c191246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4616000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4616000700 1 -- 172.21.15.86:0/995693337 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462401f050 con 0x5591bee51180 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c09ca60 con 0x5591bee51180 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.750+0000 7f4616000700 1 -- 172.21.15.86:0/995693337 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f4624004bb0 con 0x5591bee51180 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f4616000700 1 -- 172.21.15.86:0/995693337 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4624033050 con 0x5591bee51180 2024-04-25T19:19:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f463d898b80 10 client.27247 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:40.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591beec7bc0 con 0x5591bee51180 2024-04-25T19:19:40.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f4616000700 1 -- 172.21.15.86:0/995693337 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c09ca60 con 0x5591bee51180 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f4616000700 1 -- 172.21.15.86:0/995693337 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f46240268a0 con 0x5591bee51180 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f463d898b80 10 client.27247 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f463d898b80 10 client.27247 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.751+0000 7f463d898b80 20 client.27247 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.752+0000 7f463d898b80 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591beeca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.752+0000 7f463d898b80 4 client.27247 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.752+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x5591beda08b0 con 0x5591bee608a0 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.752+0000 7f462be00700 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591beeca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.753+0000 7f462be00700 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591beeca330 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde8213dee225a9 server_cookie=42b1b3fdb67a0ac9 in_seq=0 out_seq=0 2024-04-25T19:19:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.753+0000 7f4616000700 10 client.27247 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:40.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.767+0000 7f4616000700 1 -- 172.21.15.86:0/995693337 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5591beda08b0 con 0x5591bee608a0 2024-04-25T19:19:40.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.767+0000 7f4616000700 10 client.27247 handle_command_reply: tid=0 2024-04-25T19:19:40.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 client.27247 shutdown 2024-04-25T19:19:40.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:40 smithi086 ceph-mon[27783]: pgmap v526: 97 pgs: 97 active+clean; 4.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 msgr2=0x5591bee64b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee64b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5591bee424e0 tx=0x7f4624004b40 comp rx=0 tx=0).stop 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 shutdown_connections 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5591bee608a0 0x5591beeca330 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5591bee51180 0x5591bee64b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591bee459b0 0x5591bee66160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 --2- 172.21.15.86:0/995693337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591bee462d0 0x5591bee64510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 >> 172.21.15.86:0/995693337 conn(0x5591bed74730 msgr2=0x5591bee4fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f4616000700 0 client.27247 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 shutdown_connections 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.768+0000 7f463d898b80 1 -- 172.21.15.86:0/995693337 wait complete. 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f463d898b80 20 client.27247 trim_cache size 0 max 16384 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 50, 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "6c2d0f17": { 2024-04-25T19:19:40.771 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 50, 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 70.0, 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:40.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f46100a3e50 msgr2=0x7f46100a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f46100a3e50 0x7f46100a6310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f462400f540 tx=0x7f4624009920 comp rx=0 tx=0).stop 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 msgr2=0x7f462c19b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 0x7f462c19b260 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f461c00e9c0 tx=0x7f461c00ecd0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 shutdown_connections 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f462c10f4a0 0x7f462c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f462c074c70 0x7f462c19b260 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f46100a3e50 0x7f46100a6310 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.769+0000 7f4633000700 1 --2- 172.21.15.86:0/1986051712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c075590 0x7f462c19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.770+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 >> 172.21.15.86:0/1986051712 conn(0x7f462c079b10 msgr2=0x7f462c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.770+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 shutdown_connections 2024-04-25T19:19:40.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:40.770+0000 7f4633000700 1 -- 172.21.15.86:0/1986051712 wait complete. 2024-04-25T19:19:40.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:40 smithi194 ceph-mon[32740]: pgmap v526: 97 pgs: 97 active+clean; 4.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:19:41.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:40 smithi183 ceph-mon[32344]: pgmap v526: 97 pgs: 97 active+clean; 4.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T19:19:41.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6c2d0f17', quiesce took 7.9 seconds. Will release after: 10 2024-04-25T19:19:42.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:42 smithi194 ceph-mon[32740]: pgmap v527: 97 pgs: 97 active+clean; 4.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:19:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 -- 172.21.15.86:0/475836315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c104940 msgr2=0x7f474c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 --2- 172.21.15.86:0/475836315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c104940 0x7f474c104dc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4748009ca0 tx=0x7f474802a8c0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 -- 172.21.15.86:0/475836315 shutdown_connections 2024-04-25T19:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 --2- 172.21.15.86:0/475836315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c104940 0x7f474c104dc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 --2- 172.21.15.86:0/475836315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474c06acc0 0x7f474c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 --2- 172.21.15.86:0/475836315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 0x7f474c06a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 -- 172.21.15.86:0/475836315 >> 172.21.15.86:0/475836315 conn(0x7f474c076c50 msgr2=0x7f474c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 -- 172.21.15.86:0/475836315 shutdown_connections 2024-04-25T19:19:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.027+0000 7f4754800640 1 -- 172.21.15.86:0/475836315 wait complete. 2024-04-25T19:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 Processor -- start 2024-04-25T19:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 -- start start 2024-04-25T19:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 0x7f474c1a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474c104940 0x7f474c1a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 0x7f474c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f474c118790 con 0x7f474c06acc0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f474c118610 con 0x7f474c104940 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.028+0000 7f4754800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f474c118910 con 0x7f474c106fa0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 0x7f474c1a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 0x7f474c1a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52788/0 (socket says 172.21.15.86:52788) 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 -- 172.21.15.86:0/707908784 learned_addr learned my addr 172.21.15.86:0/707908784 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4752800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 0x7f474c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751400640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474c104940 0x7f474c1a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 -- 172.21.15.86:0/707908784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 msgr2=0x7f474c19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 0x7f474c19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 -- 172.21.15.86:0/707908784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474c104940 msgr2=0x7f474c1a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474c104940 0x7f474c1a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4751e00640 1 -- 172.21.15.86:0/707908784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c19f130 con 0x7f474c06acc0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.029+0000 7f4752800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 0x7f474c19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.030+0000 7f4751e00640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 0x7f474c1a47b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f474000e9f0 tx=0x7f474000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.030+0000 7f4746a00640 1 -- 172.21.15.86:0/707908784 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474000cd80 con 0x7f474c06acc0 2024-04-25T19:19:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.030+0000 7f4746a00640 1 -- 172.21.15.86:0/707908784 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47400054d0 con 0x7f474c06acc0 2024-04-25T19:19:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.030+0000 7f4746a00640 1 -- 172.21.15.86:0/707908784 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740010690 con 0x7f474c06acc0 2024-04-25T19:19:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.030+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c19f420 con 0x7f474c06acc0 2024-04-25T19:19:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.030+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1ab670 con 0x7f474c06acc0 2024-04-25T19:19:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.033+0000 7f4746a00640 1 -- 172.21.15.86:0/707908784 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4740004e90 con 0x7f474c06acc0 2024-04-25T19:19:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.033+0000 7f4757575740 1 Processor -- start 2024-04-25T19:19:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.033+0000 7f4757575740 1 -- start start 2024-04-25T19:19:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4757575740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f70fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4757575740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f71a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4757575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f71aa00 0x559a3f71cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4757575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a3f675900 con 0x559a3f71aa00 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4757575740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a3f61bef0 con 0x559a3f70f820 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4757575740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a3f6718c0 con 0x559a3f710140 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4752800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f70fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4752800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f70fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40640/0 (socket says 172.21.15.86:40640) 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4752800640 1 -- 172.21.15.86:0/927185097 learned_addr learned my addr 172.21.15.86:0/927185097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4751400640 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f71aa00 0x559a3f71cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.034+0000 7f4751e00640 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f71a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4752800640 1 -- 172.21.15.86:0/927185097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 msgr2=0x559a3f71a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4752800640 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f71a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4752800640 1 -- 172.21.15.86:0/927185097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f71aa00 msgr2=0x559a3f71cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4752800640 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f71aa00 0x559a3f71cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4752800640 1 -- 172.21.15.86:0/927185097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3f388210 con 0x559a3f70f820 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4746a00640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47340a3dc0 0x7f47340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4752800640 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f70fc00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4748035000 tx=0x7f4748005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10fa2762eaa1aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4746a00640 1 -- 172.21.15.86:0/707908784 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f47400d0cc0 con 0x7f474c06acc0 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f4751e00640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47340a3dc0 0x7f47340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f473b000640 1 -- 172.21.15.86:0/927185097 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748031450 con 0x559a3f70f820 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f473b000640 1 -- 172.21.15.86:0/927185097 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4748036040 con 0x559a3f70f820 2024-04-25T19:19:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.035+0000 7f473b000640 1 -- 172.21.15.86:0/927185097 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748004380 con 0x559a3f70f820 2024-04-25T19:19:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 -- 172.21.15.86:0/927185097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 msgr2=0x559a3f70fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4751e00640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47340a3dc0 0x7f47340a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x559a3f710dc0 tx=0x7f47400d4770 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f70fc00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4748035000 tx=0x7f4748005f80 comp rx=0 tx=0).stop 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 -- 172.21.15.86:0/927185097 shutdown_connections 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f71aa00 0x559a3f71cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f71a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 --2- 172.21.15.86:0/927185097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f70fc00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 -- 172.21.15.86:0/927185097 >> 172.21.15.86:0/927185097 conn(0x559a3f70bd30 msgr2=0x559a3f70c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 -- 172.21.15.86:0/927185097 shutdown_connections 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.036+0000 7f4757575740 1 -- 172.21.15.86:0/927185097 wait complete. 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.037+0000 7f4757575740 1 -- start start 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.037+0000 7f4757575740 1 Processor -- start 2024-04-25T19:19:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.037+0000 7f4757575740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f70f820 0x559a3f717990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4757575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f717ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4757575740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f718410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4757575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a3f71d450 con 0x559a3f710140 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4757575740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a3f62f310 con 0x559a3f71aa00 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4757575740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a3f71d5d0 con 0x559a3f70f820 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4757575740 10 client.0 _ll_register_callbacks cb 0x559a3f5fdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f717ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4752800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f70f820 0x559a3f717990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f717ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52802/0 (socket says 172.21.15.86:52802) 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 -- 172.21.15.86:0/948989121 learned_addr learned my addr 172.21.15.86:0/948989121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4752800640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f718410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 -- 172.21.15.86:0/948989121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f70f820 msgr2=0x559a3f717990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f70f820 0x559a3f717990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 -- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 msgr2=0x559a3f718410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f718410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 -- 172.21.15.86:0/948989121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3f454620 con 0x559a3f710140 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4752800640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f70f820 0x559a3f717990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f4751400640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f717ed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f473c012730 tx=0x7f473c012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db45bee612f0528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.038+0000 7f472f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f472f400640 1 -- 172.21.15.86:0/948989121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c00f920 con 0x559a3f710140 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f472f400640 1 -- 172.21.15.86:0/948989121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f473c00fac0 con 0x559a3f710140 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4748036040 con 0x559a3f710140 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f472f400640 1 -- 172.21.15.86:0/948989121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f473c00fcc0 con 0x559a3f710140 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f4757575740 10 client.17169 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a3f729750 con 0x559a3f710140 2024-04-25T19:19:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.039+0000 7f472f400640 1 -- 172.21.15.86:0/948989121 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4748036040 con 0x559a3f710140 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f472f400640 1 -- 172.21.15.86:0/948989121 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f473c01c420 con 0x559a3f710140 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4757575740 10 client.17169 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4757575740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4757575740 20 client.17169 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4757575740 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f72acb0 0x559a3f72d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4757575740 4 client.17169 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a3f6898b0 con 0x559a3f72acb0 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.040+0000 7f4752800640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f72acb0 0x559a3f72d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.041+0000 7f4752800640 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f72acb0 0x559a3f72d0c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a7eb9b1a1d240 server_cookie=1b9ab8b1a73cd76c in_seq=0 out_seq=0 2024-04-25T19:19:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.041+0000 7f472f400640 10 client.17169 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.041+0000 7f472f400640 1 -- 172.21.15.86:0/948989121 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a3f6898b0 con 0x559a3f72acb0 2024-04-25T19:19:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.041+0000 7f472f400640 10 client.17169 handle_command_reply: tid=0 2024-04-25T19:19:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.041+0000 7f4757575740 1 client.17169 shutdown 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 msgr2=0x559a3f717ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f717ed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f473c012730 tx=0x7f473c012bf0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 shutdown_connections 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f72acb0 0x559a3f72d0c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f718410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f717ed0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f472f400640 0 client.17169 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 --2- 172.21.15.86:0/948989121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f70f820 0x559a3f717990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 >> 172.21.15.86:0/948989121 conn(0x559a3f70bd30 msgr2=0x559a3f711a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 shutdown_connections 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 1 -- 172.21.15.86:0/948989121 wait complete. 2024-04-25T19:19:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.042+0000 7f4757575740 20 client.17169 trim_cache size 0 max 16384 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.058+0000 7f4757575740 1 Processor -- start 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.058+0000 7f4757575740 1 -- start start 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4757575740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f726310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4757575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f70ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4757575740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f71aa00 0x559a3f7329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4757575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b4180 con 0x559a3f710140 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4757575740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b5cc0 con 0x559a3f70f820 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4757575740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b4850 con 0x559a3f71aa00 2024-04-25T19:19:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4752800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f726310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4751400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f70ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4752800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f726310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:40660/0 (socket says 172.21.15.86:40660) 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4752800640 1 -- 172.21.15.86:0/2097518472 learned_addr learned my addr 172.21.15.86:0/2097518472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.059+0000 7f4752800640 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f71aa00 0x559a3f7329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4751400640 1 -- 172.21.15.86:0/2097518472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f71aa00 msgr2=0x559a3f7329d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4751400640 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f71aa00 0x559a3f7329d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4751400640 1 -- 172.21.15.86:0/2097518472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 msgr2=0x559a3f726310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4751400640 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f726310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4751400640 1 -- 172.21.15.86:0/2097518472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3f715f00 con 0x559a3f710140 2024-04-25T19:19:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4752800640 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f726310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f4751400640 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f70ad10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f473c000c00 tx=0x7f473c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55a1997c7279946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f473a600640 1 -- 172.21.15.86:0/2097518472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c00f920 con 0x559a3f710140 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f473a600640 1 -- 172.21.15.86:0/2097518472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c00fac0 con 0x559a3f710140 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.060+0000 7f473a600640 1 -- 172.21.15.86:0/2097518472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0035f0 con 0x559a3f710140 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 -- 172.21.15.86:0/2097518472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 msgr2=0x559a3f70ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f70ad10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f473c000c00 tx=0x7f473c009640 comp rx=0 tx=0).stop 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 -- 172.21.15.86:0/2097518472 shutdown_connections 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f71aa00 0x559a3f7329d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f710140 0x559a3f70ad10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 --2- 172.21.15.86:0/2097518472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f70f820 0x559a3f726310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 -- 172.21.15.86:0/2097518472 >> 172.21.15.86:0/2097518472 conn(0x559a3f70bd30 msgr2=0x559a3f70a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 -- 172.21.15.86:0/2097518472 shutdown_connections 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.061+0000 7f4757575740 1 -- 172.21.15.86:0/2097518472 wait complete. 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 -- start start 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 Processor -- start 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 0x559a3f7208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f720de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f790250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b55f0 con 0x559a3f70f820 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b4f20 con 0x559a3f71aa00 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b2180 con 0x559a3f710140 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.062+0000 7f4757575740 10 client.0 _ll_register_callbacks cb 0x7f473c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 0x559a3f7208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 0x559a3f7208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52824/0 (socket says 172.21.15.86:52824) 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 -- 172.21.15.86:0/1242915905 learned_addr learned my addr 172.21.15.86:0/1242915905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4751400640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f720de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f790250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 -- 172.21.15.86:0/1242915905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 msgr2=0x559a3f720de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f720de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 -- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 msgr2=0x559a3f790250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f790250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 -- 172.21.15.86:0/1242915905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3f790790 con 0x559a3f70f820 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f4752800640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 0x559a3f7208a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4748005e30 tx=0x7f4748057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afa60266b95e924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.063+0000 7f472f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f472f400640 1 -- 172.21.15.86:0/1242915905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748003cb0 con 0x559a3f70f820 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f472f400640 1 -- 172.21.15.86:0/1242915905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7f474804b040 con 0x559a3f70f820 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473c00fac0 con 0x559a3f70f820 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f472f400640 1 -- 172.21.15.86:0/1242915905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4748031450 con 0x559a3f70f820 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f4757575740 10 client.17175 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a3f7919a0 con 0x559a3f70f820 2024-04-25T19:19:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.064+0000 7f472f400640 1 -- 172.21.15.86:0/1242915905 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c00fac0 con 0x559a3f70f820 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f472f400640 1 -- 172.21.15.86:0/1242915905 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4748051040 con 0x559a3f70f820 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4757575740 10 client.17175 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4757575740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4757575740 20 client.17175 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4757575740 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f791e40 0x559a3f794260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4757575740 4 client.17175 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:19:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a3f683320 con 0x559a3f791e40 2024-04-25T19:19:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.065+0000 7f4751400640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f791e40 0x559a3f794260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.066+0000 7f4751400640 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f791e40 0x559a3f794260 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c98d725fa4701f8 server_cookie=a6b3a44a1d021393 in_seq=0 out_seq=0 2024-04-25T19:19:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.066+0000 7f472f400640 10 client.17175 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:42 smithi086 ceph-mon[27783]: pgmap v527: 97 pgs: 97 active+clean; 4.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:19:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f472f400640 1 -- 172.21.15.86:0/1242915905 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a3f683320 con 0x559a3f791e40 2024-04-25T19:19:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f472f400640 10 client.17175 handle_command_reply: tid=0 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 client.17175 shutdown 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 msgr2=0x559a3f7208a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 0x559a3f7208a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4748005e30 tx=0x7f4748057040 comp rx=0 tx=0).stop 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 shutdown_connections 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x559a3f791e40 0x559a3f794260 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a3f71aa00 0x559a3f790250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3f710140 0x559a3f720de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 --2- 172.21.15.86:0/1242915905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a3f70f820 0x559a3f7208a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f472f400640 0 client.17175 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 >> 172.21.15.86:0/1242915905 conn(0x559a3f70bd30 msgr2=0x559a3f70ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 shutdown_connections 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 1 -- 172.21.15.86:0/1242915905 wait complete. 2024-04-25T19:19:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.082+0000 7f4757575740 20 client.17175 trim_cache size 0 max 16384 2024-04-25T19:19:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:19:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:19:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:19:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47340a3dc0 msgr2=0x7f47340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47340a3dc0 0x7f47340a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x559a3f710dc0 tx=0x7f47400d4770 comp rx=0 tx=0).stop 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 msgr2=0x7f474c1a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 0x7f474c1a47b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f474000e9f0 tx=0x7f474000edf0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 shutdown_connections 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f47340a3dc0 0x7f47340a6280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f474c106fa0 0x7f474c19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f474c104940 0x7f474c1a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 --2- 172.21.15.86:0/707908784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f474c06acc0 0x7f474c1a47b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 >> 172.21.15.86:0/707908784 conn(0x7f474c076c50 msgr2=0x7f474c103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 shutdown_connections 2024-04-25T19:19:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:43.083+0000 7f4754800640 1 -- 172.21.15.86:0/707908784 wait complete. 2024-04-25T19:19:43.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:42 smithi183 ceph-mon[32344]: pgmap v527: 97 pgs: 97 active+clean; 4.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T19:19:44.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:44 smithi194 ceph-mon[32740]: pgmap v528: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-25T19:19:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:44 smithi086 ceph-mon[27783]: pgmap v528: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-25T19:19:45.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:44 smithi183 ceph-mon[32344]: pgmap v528: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-25T19:19:46.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:46 smithi194 ceph-mon[32740]: pgmap v529: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T19:19:47.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:46 smithi086 ceph-mon[27783]: pgmap v529: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T19:19:47.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:46 smithi183 ceph-mon[32344]: pgmap v529: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T19:19:49.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:48 smithi086 ceph-mon[27783]: pgmap v530: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T19:19:49.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:48 smithi183 ceph-mon[32344]: pgmap v530: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T19:19:49.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:48 smithi194 ceph-mon[32740]: pgmap v530: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T19:19:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:50 smithi086 ceph-mon[27783]: pgmap v531: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T19:19:51.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:50 smithi183 ceph-mon[32344]: pgmap v531: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T19:19:51.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:50 smithi194 ceph-mon[32740]: pgmap v531: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 19 op/s 2024-04-25T19:19:51.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:19:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:19:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:52 smithi086 ceph-mon[27783]: pgmap v532: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-25T19:19:53.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:52 smithi183 ceph-mon[32344]: pgmap v532: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-25T19:19:53.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:52 smithi194 ceph-mon[32740]: pgmap v532: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-25T19:19:54.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 -- 172.21.15.86:0/3350319386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138102a90 msgr2=0x7fa138102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:54.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3350319386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138102a90 0x7fa138102f10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa124009990 tx=0x7fa124009ca0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 -- 172.21.15.86:0/3350319386 shutdown_connections 2024-04-25T19:19:54.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3350319386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138103450 0x7fa13810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3350319386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa138108a50 0x7fa138108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3350319386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138102a90 0x7fa138102f10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 -- 172.21.15.86:0/3350319386 >> 172.21.15.86:0/3350319386 conn(0x7fa138079ba0 msgr2=0x7fa138079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 -- 172.21.15.86:0/3350319386 shutdown_connections 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.433+0000 7fa13ca00700 1 -- 172.21.15.86:0/3350319386 wait complete. 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.434+0000 7fa13ca00700 1 Processor -- start 2024-04-25T19:19:54.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 -- start start 2024-04-25T19:19:54.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138102a90 0x7fa13819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:54.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 0x7fa13819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:54.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa138108a50 0x7fa13819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:54.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa138112c00 con 0x7fa138102a90 2024-04-25T19:19:54.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa138112a80 con 0x7fa138103450 2024-04-25T19:19:54.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa13ca00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa138112d80 con 0x7fa138108a50 2024-04-25T19:19:54.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.435+0000 7fa135e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138102a90 0x7fa13819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:54.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 0x7fa13819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:54.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 0x7fa13819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37956/0 (socket says 172.21.15.86:37956) 2024-04-25T19:19:54.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 -- 172.21.15.86:0/3299225062 learned_addr learned my addr 172.21.15.86:0/3299225062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:54.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa136800700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa138108a50 0x7fa13819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:54.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 -- 172.21.15.86:0/3299225062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa138108a50 msgr2=0x7fa13819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa138108a50 0x7fa13819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 -- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138102a90 msgr2=0x7fa13819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138102a90 0x7fa13819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 -- 172.21.15.86:0/3299225062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1381a0380 con 0x7fa138103450 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135e00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138102a90 0x7fa13819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.436+0000 7fa135400700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 0x7fa13819b8b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa124000c00 tx=0x7fa12400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:54.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.437+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124027070 con 0x7fa138103450 2024-04-25T19:19:54.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.437+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa124022470 con 0x7fa138103450 2024-04-25T19:19:54.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.437+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124017650 con 0x7fa138103450 2024-04-25T19:19:54.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.437+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1381a0610 con 0x7fa138103450 2024-04-25T19:19:54.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.437+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1381a7ef0 con 0x7fa138103450 2024-04-25T19:19:54.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.438+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa138069730 con 0x7fa138103450 2024-04-25T19:19:54.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.441+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa12402a070 con 0x7fa138103450 2024-04-25T19:19:54.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.443+0000 7fa12ea00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa11c0a3e30 0x7fa11c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:54.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.443+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7fa1240d1940 con 0x7fa138103450 2024-04-25T19:19:54.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.443+0000 7fa135e00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa11c0a3e30 0x7fa11c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:54.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.444+0000 7fa135e00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa11c0a3e30 0x7fa11c0a62f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa128009fd0 tx=0x7fa128009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:54.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.444+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa124013070 con 0x7fa138103450 2024-04-25T19:19:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.720+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13819c6d0 con 0x7fa138103450 2024-04-25T19:19:54.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.721+0000 7fa12ea00700 1 -- 172.21.15.86:0/3299225062 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fa12409dfa0 con 0x7fa138103450 2024-04-25T19:19:54.723 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:19:54.723 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:19:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.724+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa11c0a3e30 msgr2=0x7fa11c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.724+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa11c0a3e30 0x7fa11c0a62f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa128009fd0 tx=0x7fa128009260 comp rx=0 tx=0).stop 2024-04-25T19:19:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.724+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 msgr2=0x7fa13819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.724+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 0x7fa13819b8b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa124000c00 tx=0x7fa12400bfd0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 shutdown_connections 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa11c0a3e30 0x7fa11c0a62f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa138108a50 0x7fa13819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa138102a90 0x7fa13819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 --2- 172.21.15.86:0/3299225062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa138103450 0x7fa13819b8b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 >> 172.21.15.86:0/3299225062 conn(0x7fa138079ba0 msgr2=0x7fa13810df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 shutdown_connections 2024-04-25T19:19:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:54.725+0000 7fa13ca00700 1 -- 172.21.15.86:0/3299225062 wait complete. 2024-04-25T19:19:54.728 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:19:55.033 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:54 smithi194 ceph-mon[32740]: pgmap v533: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T19:19:55.034 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:54 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3299225062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:54 smithi086 ceph-mon[27783]: pgmap v533: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T19:19:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:54 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3299225062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:55.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:54 smithi183 ceph-mon[32344]: pgmap v533: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T19:19:55.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:54 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3299225062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:19:55.611 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id 6c2d0f17 --release --await' 2024-04-25T19:19:55.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id 6c2d0f17 --release --await 2024-04-25T19:19:56.060 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.214+0000 7fa829000640 1 -- 172.21.15.86:0/442312459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa82410e420 msgr2=0x7fa82410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.214+0000 7fa829000640 1 --2- 172.21.15.86:0/442312459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa82410e420 0x7fa82410e800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa814009930 tx=0x7fa81402a440 comp rx=0 tx=0).stop 2024-04-25T19:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 -- 172.21.15.86:0/442312459 shutdown_connections 2024-04-25T19:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 --2- 172.21.15.86:0/442312459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa824108e10 0x7fa824115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 --2- 172.21.15.86:0/442312459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa824108490 0x7fa8241088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 --2- 172.21.15.86:0/442312459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa82410e420 0x7fa82410e800 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 -- 172.21.15.86:0/442312459 >> 172.21.15.86:0/442312459 conn(0x7fa824077d30 msgr2=0x7fa824106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 -- 172.21.15.86:0/442312459 shutdown_connections 2024-04-25T19:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.215+0000 7fa829000640 1 -- 172.21.15.86:0/442312459 wait complete. 2024-04-25T19:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.216+0000 7fa829000640 1 Processor -- start 2024-04-25T19:19:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.216+0000 7fa829000640 1 -- start start 2024-04-25T19:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa829000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa824108490 0x7fa8241a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa829000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 0x7fa8241a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa829000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa82410e420 0x7fa82419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa829000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa824118600 con 0x7fa82410e420 2024-04-25T19:19:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa829000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa824118480 con 0x7fa824108490 2024-04-25T19:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa829000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa824118780 con 0x7fa824108e10 2024-04-25T19:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa81be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 0x7fa8241a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa81be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 0x7fa8241a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48352/0 (socket says 172.21.15.86:48352) 2024-04-25T19:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa81be00640 1 -- 172.21.15.86:0/1250612409 learned_addr learned my addr 172.21.15.86:0/1250612409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa823400640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa824108490 0x7fa8241a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.217+0000 7fa823e00640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa82410e420 0x7fa82419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa81be00640 1 -- 172.21.15.86:0/1250612409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa824108490 msgr2=0x7fa8241a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa81be00640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa824108490 0x7fa8241a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa81be00640 1 -- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa82410e420 msgr2=0x7fa82419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa81be00640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa82410e420 0x7fa82419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa81be00640 1 -- 172.21.15.86:0/1250612409 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419f120 con 0x7fa824108e10 2024-04-25T19:19:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa823e00640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa82410e420 0x7fa82419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa81be00640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 0x7fa8241a4d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00ea60 tx=0x7fa80c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.218+0000 7fa820c00640 1 -- 172.21.15.86:0/1250612409 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00cde0 con 0x7fa824108e10 2024-04-25T19:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.219+0000 7fa820c00640 1 -- 172.21.15.86:0/1250612409 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c0035f0 con 0x7fa824108e10 2024-04-25T19:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.219+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419f410 con 0x7fa824108e10 2024-04-25T19:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.219+0000 7fa820c00640 1 -- 172.21.15.86:0/1250612409 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c010690 con 0x7fa824108e10 2024-04-25T19:19:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.219+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241ab6b0 con 0x7fa824108e10 2024-04-25T19:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.222+0000 7fa82d406740 1 Processor -- start 2024-04-25T19:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.222+0000 7fa820c00640 1 -- 172.21.15.86:0/1250612409 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa80c0037a0 con 0x7fa824108e10 2024-04-25T19:19:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.222+0000 7fa82d406740 1 -- start start 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa82d406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89becd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa82d406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89bf7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa82d406740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bf9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa82d406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a89b52900 con 0x560a89bed2d0 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa82d406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a89af8ef0 con 0x560a89bec9b0 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa82d406740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560a89b4e8c0 con 0x560a89bf7b90 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa823400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89bf7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa823e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89becd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa823e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89becd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:37984/0 (socket says 172.21.15.86:37984) 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa823400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89bf7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50042/0 (socket says 172.21.15.86:50042) 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa823e00640 1 -- 172.21.15.86:0/3837364345 learned_addr learned my addr 172.21.15.86:0/3837364345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.223+0000 7fa81be00640 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bf9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823e00640 1 -- 172.21.15.86:0/3837364345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 msgr2=0x560a89bf9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823e00640 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bf9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823e00640 1 -- 172.21.15.86:0/3837364345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 msgr2=0x560a89bf7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823e00640 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89bf7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823e00640 1 -- 172.21.15.86:0/3837364345 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a89865210 con 0x560a89bec9b0 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823400640 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89bf7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823e00640 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89becd90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009fd0 tx=0x7fa81c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4baf774aaac4ef44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa820c00640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa8080a3d10 0x7fa8080a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa818c00640 1 -- 172.21.15.86:0/3837364345 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c011700 con 0x560a89bec9b0 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa820c00640 1 -- 172.21.15.86:0/1250612409 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7fa80c0d0c20 con 0x7fa824108e10 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa818c00640 1 -- 172.21.15.86:0/3837364345 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa81c011d50 con 0x560a89bec9b0 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa818c00640 1 -- 172.21.15.86:0/3837364345 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00e3c0 con 0x560a89bec9b0 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.224+0000 7fa823400640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa8080a3d10 0x7fa8080a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 -- 172.21.15.86:0/3837364345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 msgr2=0x560a89becd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89becd90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009fd0 tx=0x7fa81c010860 comp rx=0 tx=0).stop 2024-04-25T19:19:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa823400640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa8080a3d10 0x7fa8080a61d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa82419fae0 tx=0x7fa814005fd0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 -- 172.21.15.86:0/3837364345 shutdown_connections 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bf9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89bf7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 --2- 172.21.15.86:0/3837364345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89becd90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 -- 172.21.15.86:0/3837364345 >> 172.21.15.86:0/3837364345 conn(0x560a89be8ec0 msgr2=0x560a89be92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 -- 172.21.15.86:0/3837364345 shutdown_connections 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.225+0000 7fa82d406740 1 -- 172.21.15.86:0/3837364345 wait complete. 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.226+0000 7fa82d406740 1 -- start start 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.226+0000 7fa82d406740 1 Processor -- start 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.226+0000 7fa82d406740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bec9b0 0x560a89bf4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.226+0000 7fa82d406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89bf50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa82d406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bf7b90 0x560a89bf55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa82d406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a89bfa7c0 con 0x560a89bf7b90 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa82d406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a89b0c310 con 0x560a89bed2d0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa82d406740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560a89bfa940 con 0x560a89bec9b0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa82d406740 10 client.0 _ll_register_callbacks cb 0x560a89adacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa823400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bec9b0 0x560a89bf4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa823e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bf7b90 0x560a89bf55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa81be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89bf50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa823400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bec9b0 0x560a89bf4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48376/0 (socket says 172.21.15.86:48376) 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa823400640 1 -- 172.21.15.86:0/1513939753 learned_addr learned my addr 172.21.15.86:0/1513939753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa81be00640 1 -- 172.21.15.86:0/1513939753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bec9b0 msgr2=0x560a89bf4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa81be00640 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bec9b0 0x560a89bf4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa81be00640 1 -- 172.21.15.86:0/1513939753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bf7b90 msgr2=0x560a89bf55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa81be00640 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bf7b90 0x560a89bf55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa81be00640 1 -- 172.21.15.86:0/1513939753 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a89931620 con 0x560a89bed2d0 2024-04-25T19:19:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.227+0000 7fa823e00640 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bf7b90 0x560a89bf55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa81be00640 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89bf50a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0ce270 tx=0x7fa80c0095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f940b52e296efcc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa811600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa811600640 1 -- 172.21.15.86:0/1513939753 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c09c630 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81c011d50 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa811600640 1 -- 172.21.15.86:0/1513939753 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7fa80c004000 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa811600640 1 -- 172.21.15.86:0/1513939753 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa80c09c450 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa82d406740 10 client.27284 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a89c06a70 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.228+0000 7fa811600640 1 -- 172.21.15.86:0/1513939753 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa81c011d50 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa811600640 1 -- 172.21.15.86:0/1513939753 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa80c0b8070 con 0x560a89bed2d0 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa82d406740 10 client.27284 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa82d406740 10 client.27284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa82d406740 20 client.27284 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa82d406740 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c07fd0 0x560a89c0a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa82d406740 4 client.27284 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a89b668b0 con 0x560a89c07fd0 2024-04-25T19:19:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.229+0000 7fa823400640 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c07fd0 0x560a89c0a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.230+0000 7fa823400640 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c07fd0 0x560a89c0a3e0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937f9cbdf01aebc3 server_cookie=f2629f4308051d21 in_seq=0 out_seq=0 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.230+0000 7fa811600640 10 client.27284 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa811600640 1 -- 172.21.15.86:0/1513939753 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a89b668b0 con 0x560a89c07fd0 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa811600640 10 client.27284 handle_command_reply: tid=0 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 client.27284 shutdown 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 msgr2=0x560a89bf50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89bf50a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0ce270 tx=0x7fa80c0095b0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 shutdown_connections 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c07fd0 0x560a89c0a3e0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bf7b90 0x560a89bf55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89bf50a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 --2- 172.21.15.86:0/1513939753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bec9b0 0x560a89bf4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa811600640 0 client.27284 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 >> 172.21.15.86:0/1513939753 conn(0x560a89be8ec0 msgr2=0x560a89beec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 shutdown_connections 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 1 -- 172.21.15.86:0/1513939753 wait complete. 2024-04-25T19:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.231+0000 7fa82d406740 20 client.27284 trim_cache size 0 max 16384 2024-04-25T19:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.247+0000 7fa82d406740 1 Processor -- start 2024-04-25T19:19:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.247+0000 7fa82d406740 1 -- start start 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa82d406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89c03680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa82d406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89c0fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa82d406740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89c10230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa82d406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b4250 con 0x560a89bed2d0 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa82d406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b5d90 con 0x560a89bec9b0 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa82d406740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b4920 con 0x560a89bf7b90 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89c03680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89c03680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38006/0 (socket says 172.21.15.86:38006) 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa81be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89c0fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 -- 172.21.15.86:0/1203013405 learned_addr learned my addr 172.21.15.86:0/1203013405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823e00640 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89c10230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 -- 172.21.15.86:0/1203013405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 msgr2=0x560a89c10230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89c10230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 -- 172.21.15.86:0/1203013405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 msgr2=0x560a89c0fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89c0fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823400640 1 -- 172.21.15.86:0/1203013405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a89c0aa60 con 0x560a89bec9b0 2024-04-25T19:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.248+0000 7fa823e00640 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89c10230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.249+0000 7fa823400640 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89c03680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa814004770 tx=0x7fa81403f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b359809ca4b207e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.249+0000 7fa813e00640 1 -- 172.21.15.86:0/1203013405 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814035040 con 0x560a89bec9b0 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.249+0000 7fa813e00640 1 -- 172.21.15.86:0/1203013405 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa814058070 con 0x560a89bec9b0 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.249+0000 7fa813e00640 1 -- 172.21.15.86:0/1203013405 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81403d450 con 0x560a89bec9b0 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 -- 172.21.15.86:0/1203013405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 msgr2=0x560a89c03680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89c03680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa814004770 tx=0x7fa81403f070 comp rx=0 tx=0).stop 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 -- 172.21.15.86:0/1203013405 shutdown_connections 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89c10230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bed2d0 0x560a89c0fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 --2- 172.21.15.86:0/1203013405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bec9b0 0x560a89c03680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 -- 172.21.15.86:0/1203013405 >> 172.21.15.86:0/1203013405 conn(0x560a89be8ec0 msgr2=0x560a89befeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 -- 172.21.15.86:0/1203013405 shutdown_connections 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 -- 172.21.15.86:0/1203013405 wait complete. 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 -- start start 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.250+0000 7fa82d406740 1 Processor -- start 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bec9b0 0x560a89c66960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89c66ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bfcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b56c0 con 0x560a89bec9b0 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b4ff0 con 0x560a89bed2d0 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b2250 con 0x560a89bf7b90 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa82d406740 10 client.0 _ll_register_callbacks cb 0x7fa80c004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89c66ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa823400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bec9b0 0x560a89c66960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa823e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bfcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89c66ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:38020/0 (socket says 172.21.15.86:38020) 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 -- 172.21.15.86:0/2005568856 learned_addr learned my addr 172.21.15.86:0/2005568856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 -- 172.21.15.86:0/2005568856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 msgr2=0x560a89bfcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bfcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 -- 172.21.15.86:0/2005568856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bec9b0 msgr2=0x560a89c66960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bec9b0 0x560a89c66960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.251+0000 7fa81be00640 1 -- 172.21.15.86:0/2005568856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a89bfd660 con 0x560a89bed2d0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa81be00640 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89c66ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa80c09ada0 tx=0x7fa80c0d0ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee0ff38cfa5bbde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa811600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa811600640 1 -- 172.21.15.86:0/2005568856 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c09c630 con 0x560a89bed2d0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa814058070 con 0x560a89bed2d0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa811600640 1 -- 172.21.15.86:0/2005568856 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7fa80c098840 con 0x560a89bed2d0 2024-04-25T19:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa811600640 1 -- 172.21.15.86:0/2005568856 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa80c098d90 con 0x560a89bed2d0 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa82d406740 10 client.27290 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.252+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a89c6e0f0 con 0x560a89bed2d0 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa811600640 1 -- 172.21.15.86:0/2005568856 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa814058070 con 0x560a89bed2d0 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa811600640 1 -- 172.21.15.86:0/2005568856 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa80c0b8070 con 0x560a89bed2d0 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa82d406740 10 client.27290 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa82d406740 10 client.27290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa82d406740 20 client.27290 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa82d406740 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c6e660 0x560a89c70a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa82d406740 4 client.27290 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:19:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a89b60320 con 0x560a89c6e660 2024-04-25T19:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.253+0000 7fa823400640 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c6e660 0x560a89c70a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.254+0000 7fa823400640 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c6e660 0x560a89c70a80 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7d593f54eaaf27 server_cookie=7c970237c9cb143b in_seq=0 out_seq=0 2024-04-25T19:19:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.254+0000 7fa811600640 10 client.27290 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa811600640 1 -- 172.21.15.86:0/2005568856 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a89b60320 con 0x560a89c6e660 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa811600640 10 client.27290 handle_command_reply: tid=0 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 client.27290 shutdown 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 msgr2=0x560a89c66ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89c66ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa80c09ada0 tx=0x7fa80c0d0ea0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 shutdown_connections 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x560a89c6e660 0x560a89c70a80 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a89bf7b90 0x560a89bfcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a89bed2d0 0x560a89c66ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 --2- 172.21.15.86:0/2005568856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a89bec9b0 0x560a89c66960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa811600640 0 client.27290 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 >> 172.21.15.86:0/2005568856 conn(0x560a89be8ec0 msgr2=0x560a89bf4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 shutdown_connections 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 1 -- 172.21.15.86:0/2005568856 wait complete. 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.268+0000 7fa82d406740 20 client.27290 trim_cache size 0 max 16384 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:19:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:19:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa8080a3d10 msgr2=0x7fa8080a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa8080a3d10 0x7fa8080a61d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa82419fae0 tx=0x7fa814005fd0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 msgr2=0x7fa8241a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 0x7fa8241a4d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00ea60 tx=0x7fa80c00ef20 comp rx=0 tx=0).stop 2024-04-25T19:19:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 shutdown_connections 2024-04-25T19:19:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fa8080a3d10 0x7fa8080a61d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa82410e420 0x7fa82419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa824108e10 0x7fa8241a4d30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 --2- 172.21.15.86:0/1250612409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa824108490 0x7fa8241a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.269+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 >> 172.21.15.86:0/1250612409 conn(0x7fa824077d30 msgr2=0x7fa824078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.270+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 shutdown_connections 2024-04-25T19:19:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:19:56.270+0000 7fa829000640 1 -- 172.21.15.86:0/1250612409 wait complete. 2024-04-25T19:19:56.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:56 smithi086 ceph-mon[27783]: pgmap v534: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-25T19:19:57.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:56 smithi183 ceph-mon[32344]: pgmap v534: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-25T19:19:57.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:56 smithi194 ceph-mon[32740]: pgmap v534: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-25T19:19:58.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 -- 172.21.15.86:0/4169148288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 msgr2=0x7feb9c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 --2- 172.21.15.86:0/4169148288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c115f80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feb94009f90 tx=0x7feb9400b330 comp rx=0 tx=0).stop 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 -- 172.21.15.86:0/4169148288 shutdown_connections 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 --2- 172.21.15.86:0/4169148288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c115f80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 --2- 172.21.15.86:0/4169148288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb9c1092d0 0x7feb9c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 --2- 172.21.15.86:0/4169148288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c109c80 0x7feb9c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.238+0000 7feba2400700 1 -- 172.21.15.86:0/4169148288 >> 172.21.15.86:0/4169148288 conn(0x7feb9c076210 msgr2=0x7feb9c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.239+0000 7feba2400700 1 -- 172.21.15.86:0/4169148288 shutdown_connections 2024-04-25T19:19:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.239+0000 7feba2400700 1 -- 172.21.15.86:0/4169148288 wait complete. 2024-04-25T19:19:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 Processor -- start 2024-04-25T19:19:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 -- start start 2024-04-25T19:19:58.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 0x7feb9c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb9c109c80 0x7feb9c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c119250 con 0x7feb9c1092d0 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1190d0 con 0x7feb9c10e3b0 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.240+0000 7feba2400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1193d0 con 0x7feb9c109c80 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 0x7feb9c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 0x7feb9c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50072/0 (socket says 172.21.15.86:50072) 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 -- 172.21.15.86:0/785651131 learned_addr learned my addr 172.21.15.86:0/785651131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9aa00700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb9c109c80 0x7feb9c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9be00700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 -- 172.21.15.86:0/785651131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb9c109c80 msgr2=0x7feb9c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb9c109c80 0x7feb9c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 -- 172.21.15.86:0/785651131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 msgr2=0x7feb9c1a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c1a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9b400700 1 -- 172.21.15.86:0/785651131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c1a4710 con 0x7feb9c1092d0 2024-04-25T19:19:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.241+0000 7feb9be00700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.242+0000 7feb9b400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 0x7feb9c19f700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00eb30 tx=0x7feb8c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.242+0000 7feb93e00700 1 -- 172.21.15.86:0/785651131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00cbd0 con 0x7feb9c1092d0 2024-04-25T19:19:58.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.242+0000 7feb93e00700 1 -- 172.21.15.86:0/785651131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c0044d0 con 0x7feb9c1092d0 2024-04-25T19:19:58.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.242+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c1a4a00 con 0x7feb9c1092d0 2024-04-25T19:19:58.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.242+0000 7feb93e00700 1 -- 172.21.15.86:0/785651131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c003df0 con 0x7feb9c1092d0 2024-04-25T19:19:58.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.242+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c1ac280 con 0x7feb9c1092d0 2024-04-25T19:19:58.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.245+0000 7feb93e00700 1 -- 172.21.15.86:0/785651131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb8c004000 con 0x7feb9c1092d0 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.247+0000 7febacddeb80 1 Processor -- start 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 -- start start 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7feb93e00700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feb800a3e30 0x7feb800a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7feb93e00700 1 -- 172.21.15.86:0/785651131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7feb8c0d0ad0 con 0x7feb9c1092d0 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7feb9be00700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feb800a3e30 0x7feb800a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b72d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a88ac5f90 con 0x563a88b72920 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a88a6c830 con 0x563a88b73240 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7febacddeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a88ac1f50 con 0x563a88b7e0f0 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7feb9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b72d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7feb9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b72d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50082/0 (socket says 172.21.15.86:50082) 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.248+0000 7feb9b400700 1 -- 172.21.15.86:0/3026203242 learned_addr learned my addr 172.21.15.86:0/3026203242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9aa00700 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9be00700 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9b400700 1 -- 172.21.15.86:0/3026203242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 msgr2=0x563a88b804e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9b400700 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b804e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9b400700 1 -- 172.21.15.86:0/3026203242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 msgr2=0x563a88b7dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9b400700 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9b400700 1 -- 172.21.15.86:0/3026203242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a88a62780 con 0x563a88b72920 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9be00700 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9be00700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feb800a3e30 0x7feb800a62f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b6c0 tx=0x7feb94004f40 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.249+0000 7feb9b400700 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b72d00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feb8c099370 tx=0x7feb8c09c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb97d558cc51448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.250+0000 7feb90c00700 1 -- 172.21.15.86:0/3026203242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c009500 con 0x563a88b72920 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.250+0000 7feb90c00700 1 -- 172.21.15.86:0/3026203242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c0042f0 con 0x563a88b72920 2024-04-25T19:19:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.250+0000 7feb90c00700 1 -- 172.21.15.86:0/3026203242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c099010 con 0x563a88b72920 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.250+0000 7febacddeb80 1 -- 172.21.15.86:0/3026203242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 msgr2=0x563a88b72d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.250+0000 7febacddeb80 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b72d00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feb8c099370 tx=0x7feb8c09c770 comp rx=0 tx=0).stop 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 -- 172.21.15.86:0/3026203242 shutdown_connections 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b72d00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 --2- 172.21.15.86:0/3026203242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 -- 172.21.15.86:0/3026203242 >> 172.21.15.86:0/3026203242 conn(0x563a88aa16a0 msgr2=0x563a88b6edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 -- 172.21.15.86:0/3026203242 shutdown_connections 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 -- 172.21.15.86:0/3026203242 wait complete. 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.251+0000 7febacddeb80 1 -- start start 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 Processor -- start 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b7d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b7b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a88b80c10 con 0x563a88b72920 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a88a7f960 con 0x563a88b73240 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a88b80d90 con 0x563a88b7e0f0 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7febacddeb80 10 client.0 _ll_register_callbacks cb 0x563a88a5e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.252+0000 7feb9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b7d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b7d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50086/0 (socket says 172.21.15.86:50086) 2024-04-25T19:19:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 -- 172.21.15.86:0/831947129 learned_addr learned my addr 172.21.15.86:0/831947129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9be00700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b7b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9aa00700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 -- 172.21.15.86:0/831947129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 msgr2=0x563a88b7b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b7b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 -- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 msgr2=0x563a88b7abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 -- 172.21.15.86:0/831947129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a88a139d0 con 0x563a88b72920 2024-04-25T19:19:58.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb9b400700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b7d8c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00eb00 tx=0x7feb8c003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f4fb4573678a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:58.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb85600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:58.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.253+0000 7feb85600700 1 -- 172.21.15.86:0/831947129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c09d0a0 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7feb85600700 1 -- 172.21.15.86:0/831947129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7feb8c09cc00 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c0042f0 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7feb85600700 1 -- 172.21.15.86:0/831947129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb8c09c9a0 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 10 client.17211 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a88a5cbf0 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7feb85600700 1 -- 172.21.15.86:0/831947129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c0042f0 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7feb85600700 1 -- 172.21.15.86:0/831947129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feb8c0bbb60 con 0x563a88b72920 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 10 client.17211 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 10 client.17211 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 20 client.17211 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88b8d780 0x563a88b8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 4 client.17211 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:19:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.254+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a88ad3780 con 0x563a88b8d780 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.255+0000 7feb9aa00700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88b8d780 0x563a88b8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.255+0000 7feb9aa00700 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88b8d780 0x563a88b8fb70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a2979e8096972b server_cookie=7775bb62c6ed9666 in_seq=0 out_seq=0 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.255+0000 7feb85600700 10 client.17211 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.256+0000 7feb85600700 1 -- 172.21.15.86:0/831947129 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a88ad3780 con 0x563a88b8d780 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.256+0000 7feb85600700 10 client.17211 handle_command_reply: tid=0 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.256+0000 7febacddeb80 1 client.17211 shutdown 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 msgr2=0x563a88b7d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b7d8c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00eb00 tx=0x7feb8c003790 comp rx=0 tx=0).stop 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 shutdown_connections 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88b8d780 0x563a88b8fb70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b7e0f0 0x563a88b7b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b72920 0x563a88b7d8c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 --2- 172.21.15.86:0/831947129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b7abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7feb85600700 0 client.17211 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 >> 172.21.15.86:0/831947129 conn(0x563a88aa16a0 msgr2=0x563a88b6e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 shutdown_connections 2024-04-25T19:19:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 1 -- 172.21.15.86:0/831947129 wait complete. 2024-04-25T19:19:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.257+0000 7febacddeb80 20 client.17211 trim_cache size 0 max 16384 2024-04-25T19:19:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.278+0000 7febacddeb80 1 Processor -- start 2024-04-25T19:19:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.278+0000 7febacddeb80 1 -- start start 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.278+0000 7febacddeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 0x563a88b76320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7febacddeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b77e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7febacddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b78380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7febacddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1b3b70 con 0x563a88b7e0f0 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7febacddeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1b56b0 con 0x563a88b73240 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7febacddeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1b4240 con 0x563a88b72920 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b78380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 0x563a88b76320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 0x563a88b76320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48428/0 (socket says 172.21.15.86:48428) 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 -- 172.21.15.86:0/264565780 learned_addr learned my addr 172.21.15.86:0/264565780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9aa00700 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b77e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 -- 172.21.15.86:0/264565780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 msgr2=0x563a88b77e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b77e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 -- 172.21.15.86:0/264565780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 msgr2=0x563a88b78380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b78380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9b400700 1 -- 172.21.15.86:0/264565780 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a88b79130 con 0x563a88b72920 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.279+0000 7feb9aa00700 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b77e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.280+0000 7feb9b400700 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 0x563a88b76320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb8c099370 tx=0x7feb8c09aeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db13678c419c4552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.280+0000 7feb87e00700 1 -- 172.21.15.86:0/264565780 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c0042f0 con 0x563a88b72920 2024-04-25T19:19:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.280+0000 7feb87e00700 1 -- 172.21.15.86:0/264565780 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c09d0a0 con 0x563a88b72920 2024-04-25T19:19:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.280+0000 7feb87e00700 1 -- 172.21.15.86:0/264565780 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c09cc00 con 0x563a88b72920 2024-04-25T19:19:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 -- 172.21.15.86:0/264565780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 msgr2=0x563a88b76320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 0x563a88b76320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb8c099370 tx=0x7feb8c09aeb0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 -- 172.21.15.86:0/264565780 shutdown_connections 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b78380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b72920 0x563a88b76320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 --2- 172.21.15.86:0/264565780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b73240 0x563a88b77e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 -- 172.21.15.86:0/264565780 >> 172.21.15.86:0/264565780 conn(0x563a88aa16a0 msgr2=0x563a88b7b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 -- 172.21.15.86:0/264565780 shutdown_connections 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 -- 172.21.15.86:0/264565780 wait complete. 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.281+0000 7febacddeb80 1 -- start start 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 Processor -- start 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b72920 0x563a88b92c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 0x563a88b90fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b91510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1b4fe0 con 0x563a88b7e0f0 2024-04-25T19:19:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1b4910 con 0x563a88b72920 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1b1b30 con 0x563a88b73240 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.282+0000 7febacddeb80 10 client.0 _ll_register_callbacks cb 0x563a88a5e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b91510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 0x563a88b90fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 0x563a88b90fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:48438/0 (socket says 172.21.15.86:48438) 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b72920 0x563a88b92c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b91510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50092/0 (socket says 172.21.15.86:50092) 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9aa00700 1 -- 172.21.15.86:0/320926460 learned_addr learned my addr 172.21.15.86:0/320926460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 -- 172.21.15.86:0/320926460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 msgr2=0x563a88b90fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 0x563a88b90fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 -- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b72920 msgr2=0x563a88b92c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b72920 0x563a88b92c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 -- 172.21.15.86:0/320926460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a88b91a50 con 0x563a88b7e0f0 2024-04-25T19:19:58.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9aa00700 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 0x563a88b90fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb9be00700 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b91510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7feb94009790 tx=0x7feb9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601c8692611f12a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.283+0000 7feb85600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7feb85600700 1 -- 172.21.15.86:0/320926460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94030070 con 0x563a88b7e0f0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c09d0a0 con 0x563a88b7e0f0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7feb85600700 1 -- 172.21.15.86:0/320926460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8637+0+0 (secure 0 0 0) 0x7feb94007cb0 con 0x563a88b7e0f0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7feb85600700 1 -- 172.21.15.86:0/320926460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb94035460 con 0x563a88b7e0f0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7febacddeb80 10 client.17217 fetch_fsmap learned FSMap version 29 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a88bf4720 con 0x563a88b7e0f0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.284+0000 7feb85600700 1 -- 172.21.15.86:0/320926460 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c09d0a0 con 0x563a88b7e0f0 2024-04-25T19:19:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7feb85600700 1 -- 172.21.15.86:0/320926460 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feb94011070 con 0x563a88b7e0f0 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7febacddeb80 10 client.17217 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7febacddeb80 10 client.17217 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7febacddeb80 20 client.17217 populate_metadata read hostname 'smithi086' 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7febacddeb80 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88bf4e10 0x563a88bf7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7febacddeb80 4 client.17217 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6c2d0f17", "release": true, "await": true}] 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6c2d0f17", "release": true, "await": true}) -- 0x563a88acd820 con 0x563a88bf4e10 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7feb9b400700 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88bf4e10 0x563a88bf7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7feb9b400700 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88bf4e10 0x563a88bf7230 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bbcaaed700dcfa server_cookie=fc864331333595f9 in_seq=0 out_seq=0 2024-04-25T19:19:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.285+0000 7feb85600700 10 client.17217 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7feb85600700 1 -- 172.21.15.86:0/320926460 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563a88acd820 con 0x563a88bf4e10 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7feb85600700 10 client.17217 handle_command_reply: tid=0 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 client.17217 shutdown 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 msgr2=0x563a88b91510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b91510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7feb94009790 tx=0x7feb9400b330 comp rx=0 tx=0).stop 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 shutdown_connections 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x563a88bf4e10 0x563a88bf7230 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a88b7e0f0 0x563a88b91510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a88b72920 0x563a88b92c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 --2- 172.21.15.86:0/320926460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a88b73240 0x563a88b90fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 >> 172.21.15.86:0/320926460 conn(0x563a88aa16a0 msgr2=0x563a88b7c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7feb85600700 0 client.17217 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:19:58.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.289+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 shutdown_connections 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7febacddeb80 1 -- 172.21.15.86:0/320926460 wait complete. 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7febacddeb80 20 client.17217 trim_cache size 0 max 16384 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 52, 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout: "6c2d0f17": { 2024-04-25T19:19:58.292 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 52, 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 70.0, 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:58.293 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:58.294 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:19:58.294 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:19:58.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feb800a3e30 msgr2=0x7feb800a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feb800a3e30 0x7feb800a62f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b6c0 tx=0x7feb94004f40 comp rx=0 tx=0).stop 2024-04-25T19:19:58.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 msgr2=0x7feb9c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:58.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 0x7feb9c19f700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00eb30 tx=0x7feb8c00ee40 comp rx=0 tx=0).stop 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 shutdown_connections 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb9c10e3b0 0x7feb9c1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb9c1092d0 0x7feb9c19f700 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feb800a3e30 0x7feb800a62f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.290+0000 7feba2400700 1 --2- 172.21.15.86:0/785651131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb9c109c80 0x7feb9c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.291+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 >> 172.21.15.86:0/785651131 conn(0x7feb9c076210 msgr2=0x7feb9c0767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.291+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 shutdown_connections 2024-04-25T19:19:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:19:58.291+0000 7feba2400700 1 -- 172.21.15.86:0/785651131 wait complete. 2024-04-25T19:19:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:19:58 smithi086 ceph-mon[27783]: pgmap v535: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-25T19:19:59.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:19:58 smithi183 ceph-mon[32344]: pgmap v535: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-25T19:19:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:19:58 smithi194 ceph-mon[32740]: pgmap v535: 97 pgs: 97 active+clean; 4.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-25T19:19:59.248 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6c2d0f17', seconds elapsed: 25.4 2024-04-25T19:20:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:00 smithi086 ceph-mon[27783]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T19:20:00.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:00 smithi086 ceph-mon[27783]: overall HEALTH_OK 2024-04-25T19:20:01.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:00 smithi183 ceph-mon[32344]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T19:20:01.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:00 smithi183 ceph-mon[32344]: overall HEALTH_OK 2024-04-25T19:20:01.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:00 smithi194 ceph-mon[32740]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T19:20:01.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:00 smithi194 ceph-mon[32740]: overall HEALTH_OK 2024-04-25T19:20:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:02 smithi086 ceph-mon[27783]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-04-25T19:20:03.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:02 smithi183 ceph-mon[32344]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-04-25T19:20:03.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:02 smithi194 ceph-mon[32740]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-04-25T19:20:05.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:04 smithi086 ceph-mon[27783]: pgmap v538: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 34 op/s 2024-04-25T19:20:05.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:04 smithi183 ceph-mon[32344]: pgmap v538: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 34 op/s 2024-04-25T19:20:05.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:04 smithi194 ceph-mon[32740]: pgmap v538: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 34 op/s 2024-04-25T19:20:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:06 smithi086 ceph-mon[27783]: pgmap v539: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-04-25T19:20:07.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:06 smithi183 ceph-mon[32344]: pgmap v539: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-04-25T19:20:07.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:06 smithi194 ceph-mon[32740]: pgmap v539: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-04-25T19:20:08.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:07 smithi086 ceph-mon[27783]: osdmap e117: 12 total, 12 up, 12 in 2024-04-25T19:20:08.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:07 smithi183 ceph-mon[32344]: osdmap e117: 12 total, 12 up, 12 in 2024-04-25T19:20:08.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:07 smithi194 ceph-mon[32740]: osdmap e117: 12 total, 12 up, 12 in 2024-04-25T19:20:09.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:08 smithi086 ceph-mon[27783]: pgmap v541: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 35 op/s 2024-04-25T19:20:09.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:08 smithi183 ceph-mon[32344]: pgmap v541: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 35 op/s 2024-04-25T19:20:09.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:08 smithi194 ceph-mon[32740]: pgmap v541: 97 pgs: 97 active+clean; 4.7 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 35 op/s 2024-04-25T19:20:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 -- 172.21.15.86:0/917575664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104bb0 msgr2=0x7f5a98108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 --2- 172.21.15.86:0/917575664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104bb0 0x7f5a98108bf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009900 tx=0x7f5a8002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 -- 172.21.15.86:0/917575664 shutdown_connections 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 --2- 172.21.15.86:0/917575664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a981092d0 0x7f5a98110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 --2- 172.21.15.86:0/917575664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104bb0 0x7f5a98108bf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 --2- 172.21.15.86:0/917575664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a98104200 0x7f5a981045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 -- 172.21.15.86:0/917575664 >> 172.21.15.86:0/917575664 conn(0x7f5a980ffa60 msgr2=0x7f5a98101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 -- 172.21.15.86:0/917575664 shutdown_connections 2024-04-25T19:20:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.411+0000 7f5a97e00640 1 -- 172.21.15.86:0/917575664 wait complete. 2024-04-25T19:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.412+0000 7f5a97e00640 1 Processor -- start 2024-04-25T19:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.412+0000 7f5a97e00640 1 -- start start 2024-04-25T19:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.412+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 0x7f5a981a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.412+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a981092d0 0x7f5a981a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.412+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a9819a670 0x7f5a9819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.412+0000 7f5a97e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981141f0 con 0x7f5a98104200 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a97e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98114070 con 0x7f5a981092d0 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a97e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98114370 con 0x7f5a9819a670 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 0x7f5a981a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 0x7f5a981a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33530/0 (socket says 172.21.15.86:33530) 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 -- 172.21.15.86:0/1554415046 learned_addr learned my addr 172.21.15.86:0/1554415046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a8e000640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a981092d0 0x7f5a981a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a97400640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a9819a670 0x7f5a9819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 -- 172.21.15.86:0/1554415046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a9819a670 msgr2=0x7f5a9819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a9819a670 0x7f5a9819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 -- 172.21.15.86:0/1554415046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a981092d0 msgr2=0x7f5a981a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a981092d0 0x7f5a981a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 -- 172.21.15.86:0/1554415046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9819b270 con 0x7f5a98104200 2024-04-25T19:20:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.413+0000 7f5a96a00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 0x7f5a981a05b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e9f0 tx=0x7f5a8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.414+0000 7f5a8fe00640 1 -- 172.21.15.86:0/1554415046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800ccc0 con 0x7f5a98104200 2024-04-25T19:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.414+0000 7f5a8fe00640 1 -- 172.21.15.86:0/1554415046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a880108e0 con 0x7f5a98104200 2024-04-25T19:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.414+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a981a7350 con 0x7f5a98104200 2024-04-25T19:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.414+0000 7f5a8fe00640 1 -- 172.21.15.86:0/1554415046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88018880 con 0x7f5a98104200 2024-04-25T19:20:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.414+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a981a7700 con 0x7f5a98104200 2024-04-25T19:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.417+0000 7f5aa09a6740 1 Processor -- start 2024-04-25T19:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.417+0000 7f5a8fe00640 1 -- 172.21.15.86:0/1554415046 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a88010430 con 0x7f5a98104200 2024-04-25T19:20:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.417+0000 7f5aa09a6740 1 -- start start 2024-04-25T19:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653f4e93900 con 0x5653f4f2d9b0 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653f4e39ef0 con 0x5653f4f38b90 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5653f4e8f8c0 con 0x5653f4f2e2d0 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f2dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33532/0 (socket says 172.21.15.86:33532) 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 -- 172.21.15.86:0/3269089862 learned_addr learned my addr 172.21.15.86:0/3269089862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a8e000640 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a96a00640 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 -- 172.21.15.86:0/3269089862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 msgr2=0x5653f4f38650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f38650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 -- 172.21.15.86:0/3269089862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 msgr2=0x5653f4f3af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f3af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.418+0000 7f5a97400640 1 -- 172.21.15.86:0/3269089862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f4ba6210 con 0x5653f4f2d9b0 2024-04-25T19:20:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a97400640 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f2dd90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009f60 tx=0x7f5a90009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c08acfb8b25542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a8fe00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a7c0a3d10 0x7f5a7c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a87e00640 1 -- 172.21.15.86:0/3269089862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9001e070 con 0x5653f4f2d9b0 2024-04-25T19:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a87e00640 1 -- 172.21.15.86:0/3269089862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a90003c30 con 0x5653f4f2d9b0 2024-04-25T19:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a87e00640 1 -- 172.21.15.86:0/3269089862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000f820 con 0x5653f4f2d9b0 2024-04-25T19:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a8fe00640 1 -- 172.21.15.86:0/1554415046 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8681+0+0 (secure 0 0 0) 0x7f5a880d0cb0 con 0x7f5a98104200 2024-04-25T19:20:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.419+0000 7f5a96a00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a7c0a3d10 0x7f5a7c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3269089862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 msgr2=0x5653f4f2dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f2dd90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009f60 tx=0x7f5a90009f90 comp rx=0 tx=0).stop 2024-04-25T19:20:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5a96a00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a7c0a3d10 0x7f5a7c0a61d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5653f4f2ef50 tx=0x7f5a8800ce60 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3269089862 shutdown_connections 2024-04-25T19:20:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f3af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f38650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3269089862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f2dd90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3269089862 >> 172.21.15.86:0/3269089862 conn(0x5653f4f29ec0 msgr2=0x5653f4f2a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3269089862 shutdown_connections 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3269089862 wait complete. 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.420+0000 7f5aa09a6740 1 -- start start 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 Processor -- start 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f35b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f2e2d0 0x5653f4f36050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 0x5653f4f36590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653f4f3b7c0 con 0x5653f4f38b90 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653f4e4d310 con 0x5653f4f2e2d0 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5653f4f3b940 con 0x5653f4f2d9b0 2024-04-25T19:20:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5aa09a6740 10 client.0 _ll_register_callbacks cb 0x5653f4e1bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.421+0000 7f5a97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 0x5653f4f36590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 0x5653f4f36590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33540/0 (socket says 172.21.15.86:33540) 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 -- 172.21.15.86:0/2468249347 learned_addr learned my addr 172.21.15.86:0/2468249347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a8e000640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f2e2d0 0x5653f4f36050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f35b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 -- 172.21.15.86:0/2468249347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 msgr2=0x5653f4f35b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f35b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 -- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f2e2d0 msgr2=0x5653f4f36050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f2e2d0 0x5653f4f36050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 -- 172.21.15.86:0/2468249347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f4c72620 con 0x5653f4f38b90 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a97400640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 0x5653f4f36590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a90004880 tx=0x7f5a9000f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84079c598f375158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a84c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a84c00640 1 -- 172.21.15.86:0/2468249347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9002d060 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.422+0000 7f5a84c00640 1 -- 172.21.15.86:0/2468249347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8681+0+0 (secure 0 0 0) 0x7f5a9000f580 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a90003c30 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5a84c00640 1 -- 172.21.15.86:0/2468249347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a9001e030 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 10 client.17229 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653f4f47a00 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5a84c00640 1 -- 172.21.15.86:0/2468249347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a90003c30 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5a84c00640 1 -- 172.21.15.86:0/2468249347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a90019570 con 0x5653f4f38b90 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 10 client.17229 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 20 client.17229 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4f48e60 0x5653f4f4b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 4 client.17229 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:20:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.423+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f4ea78b0 con 0x5653f4f48e60 2024-04-25T19:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.424+0000 7f5a8e000640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4f48e60 0x5653f4f4b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.424+0000 7f5a8e000640 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4f48e60 0x5653f4f4b250 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35cf6711afb2bbb3 server_cookie=ee17fcf719654ac2 in_seq=0 out_seq=0 2024-04-25T19:20:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.424+0000 7f5a84c00640 10 client.17229 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.425+0000 7f5a84c00640 1 -- 172.21.15.86:0/2468249347 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653f4ea78b0 con 0x5653f4f48e60 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.425+0000 7f5a84c00640 10 client.17229 handle_command_reply: tid=0 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.425+0000 7f5aa09a6740 1 client.17229 shutdown 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.425+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 msgr2=0x5653f4f36590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.425+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 0x5653f4f36590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a90004880 tx=0x7f5a9000f480 comp rx=0 tx=0).stop 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 shutdown_connections 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4f48e60 0x5653f4f4b250 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f38b90 0x5653f4f36590 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f2e2d0 0x5653f4f36050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/2468249347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f35b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 >> 172.21.15.86:0/2468249347 conn(0x5653f4f29ec0 msgr2=0x5653f4f2fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5a84c00640 0 client.17229 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 shutdown_connections 2024-04-25T19:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 1 -- 172.21.15.86:0/2468249347 wait complete. 2024-04-25T19:20:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.426+0000 7f5aa09a6740 20 client.17229 trim_cache size 0 max 16384 2024-04-25T19:20:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 Processor -- start 2024-04-25T19:20:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 -- start start 2024-04-25T19:20:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f44680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f50910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981aff10 con 0x5653f4f2d9b0 2024-04-25T19:20:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981b1a50 con 0x5653f4f38b90 2024-04-25T19:20:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.442+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981b05e0 con 0x5653f4f2e2d0 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a8e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33556/0 (socket says 172.21.15.86:33556) 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 -- 172.21.15.86:0/3883462432 learned_addr learned my addr 172.21.15.86:0/3883462432 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f50910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 -- 172.21.15.86:0/3883462432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 msgr2=0x5653f4f353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 -- 172.21.15.86:0/3883462432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 msgr2=0x5653f4f50910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f50910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 -- 172.21.15.86:0/3883462432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f4f25820 con 0x5653f4f2d9b0 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.443+0000 7f5a97400640 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f44680 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5a90005d70 tx=0x7f5a9001bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943016de719cdb5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5a87400640 1 -- 172.21.15.86:0/3883462432 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9002d040 con 0x5653f4f2d9b0 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5a87400640 1 -- 172.21.15.86:0/3883462432 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a9001f040 con 0x5653f4f2d9b0 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5a87400640 1 -- 172.21.15.86:0/3883462432 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000f4f0 con 0x5653f4f2d9b0 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3883462432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 msgr2=0x5653f4f44680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f44680 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5a90005d70 tx=0x7f5a9001bfa0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3883462432 shutdown_connections 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4f50910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2e2d0 0x5653f4f353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/3883462432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2d9b0 0x5653f4f44680 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3883462432 >> 172.21.15.86:0/3883462432 conn(0x5653f4f29ec0 msgr2=0x5653f4f29020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3883462432 shutdown_connections 2024-04-25T19:20:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.444+0000 7f5aa09a6740 1 -- 172.21.15.86:0/3883462432 wait complete. 2024-04-25T19:20:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.445+0000 7f5aa09a6740 1 -- start start 2024-04-25T19:20:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.445+0000 7f5aa09a6740 1 Processor -- start 2024-04-25T19:20:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.445+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 0x5653f4f3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5aa09a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4fad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981b1380 con 0x5653f4f2e2d0 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981b0cb0 con 0x5653f4f38b90 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5aa09a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981adf10 con 0x5653f4f2d9b0 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5aa09a6740 10 client.0 _ll_register_callbacks cb 0x7f5a90004d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 0x5653f4f3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a97400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 0x5653f4f3e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33562/0 (socket says 172.21.15.86:33562) 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 -- 172.21.15.86:0/303323537 learned_addr learned my addr 172.21.15.86:0/303323537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a97400640 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4fad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 -- 172.21.15.86:0/303323537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 msgr2=0x5653f4f3dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f3dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 -- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 msgr2=0x5653f4fad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4fad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.446+0000 7f5a8e000640 1 -- 172.21.15.86:0/303323537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f4fad6c0 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5a8e000640 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 0x5653f4f3e290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005940 tx=0x7f5a80005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4996357326a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5a84c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5a84c00640 1 -- 172.21.15.86:0/303323537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80035040 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5a84c00640 1 -- 172.21.15.86:0/303323537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8681+0+0 (secure 0 0 0) 0x7f5a80004340 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a9001f040 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5a84c00640 1 -- 172.21.15.86:0/303323537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a80055050 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5aa09a6740 10 client.17235 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.447+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653f4fae960 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5a84c00640 1 -- 172.21.15.86:0/303323537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a9001f040 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5a84c00640 1 -- 172.21.15.86:0/303323537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a800466f0 con 0x5653f4f2e2d0 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5aa09a6740 10 client.17235 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5aa09a6740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5aa09a6740 20 client.17235 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4faf160 0x5653f4fb1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5aa09a6740 4 client.17235 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653f4ea1320 con 0x5653f4faf160 2024-04-25T19:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.448+0000 7f5a97400640 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4faf160 0x5653f4fb1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.449+0000 7f5a97400640 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4faf160 0x5653f4fb1580 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b25faf1286636f0 server_cookie=cb34b0a0b3a090d2 in_seq=0 out_seq=0 2024-04-25T19:20:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.449+0000 7f5a84c00640 10 client.17235 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5a84c00640 1 -- 172.21.15.86:0/303323537 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653f4ea1320 con 0x5653f4faf160 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5a84c00640 10 client.17235 handle_command_reply: tid=0 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 client.17235 shutdown 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 msgr2=0x5653f4f3e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 0x5653f4f3e290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005940 tx=0x7f5a80005c00 comp rx=0 tx=0).stop 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 shutdown_connections 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5653f4faf160 0x5653f4fb1580 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5653f4f38b90 0x5653f4fad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653f4f2e2d0 0x5653f4f3e290 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5a84c00640 0 client.17235 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 --2- 172.21.15.86:0/303323537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653f4f2d9b0 0x5653f4f3dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 >> 172.21.15.86:0/303323537 conn(0x5653f4f29ec0 msgr2=0x5653f4f35ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 shutdown_connections 2024-04-25T19:20:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 1 -- 172.21.15.86:0/303323537 wait complete. 2024-04-25T19:20:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.469+0000 7f5aa09a6740 20 client.17235 trim_cache size 0 max 16384 2024-04-25T19:20:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:20:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:20:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:20:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a7c0a3d10 msgr2=0x7f5a7c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a7c0a3d10 0x7f5a7c0a61d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5653f4f2ef50 tx=0x7f5a8800ce60 comp rx=0 tx=0).stop 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 msgr2=0x7f5a981a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 0x7f5a981a05b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e9f0 tx=0x7f5a8800edf0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 shutdown_connections 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5a7c0a3d10 0x7f5a7c0a61d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a9819a670 0x7f5a9819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a981092d0 0x7f5a981a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 --2- 172.21.15.86:0/1554415046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a98104200 0x7f5a981a05b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.470+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 >> 172.21.15.86:0/1554415046 conn(0x7f5a980ffa60 msgr2=0x7f5a98100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.471+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 shutdown_connections 2024-04-25T19:20:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:09.471+0000 7f5a97e00640 1 -- 172.21.15.86:0/1554415046 wait complete. 2024-04-25T19:20:11.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:10 smithi086 ceph-mon[27783]: pgmap v542: 97 pgs: 97 active+clean; 4.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-04-25T19:20:11.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:10 smithi183 ceph-mon[32344]: pgmap v542: 97 pgs: 97 active+clean; 4.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-04-25T19:20:11.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:10 smithi194 ceph-mon[32740]: pgmap v542: 97 pgs: 97 active+clean; 4.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-04-25T19:20:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:12 smithi183 ceph-mon[32344]: pgmap v543: 97 pgs: 97 active+clean; 4.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-04-25T19:20:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:12 smithi194 ceph-mon[32740]: pgmap v543: 97 pgs: 97 active+clean; 4.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-04-25T19:20:13.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:12 smithi086 ceph-mon[27783]: pgmap v543: 97 pgs: 97 active+clean; 4.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 96 KiB/s wr, 35 op/s 2024-04-25T19:20:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:14 smithi183 ceph-mon[32344]: pgmap v544: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:14.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:14 smithi183 ceph-mon[32344]: osdmap e118: 12 total, 12 up, 12 in 2024-04-25T19:20:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:14 smithi194 ceph-mon[32740]: pgmap v544: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:14.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:14 smithi194 ceph-mon[32740]: osdmap e118: 12 total, 12 up, 12 in 2024-04-25T19:20:15.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:14 smithi086 ceph-mon[27783]: pgmap v544: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:15.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:14 smithi086 ceph-mon[27783]: osdmap e118: 12 total, 12 up, 12 in 2024-04-25T19:20:16.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:16 smithi183 ceph-mon[32344]: pgmap v546: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:20:16.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:16 smithi194 ceph-mon[32740]: pgmap v546: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:20:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:16 smithi086 ceph-mon[27783]: pgmap v546: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T19:20:17.820 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-04-25T19:20:17.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:20:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:20:18.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:18 smithi086 ceph-mon[27783]: pgmap v547: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:18.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:18 smithi194 ceph-mon[32740]: pgmap v547: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:19.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:18 smithi183 ceph-mon[32344]: pgmap v547: 97 pgs: 97 active+clean; 5.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:20.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 -- 172.21.15.86:0/1866753011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06b0f0 msgr2=0x7fbc3c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:20.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 --2- 172.21.15.86:0/1866753011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c10f900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009f90 tx=0x7fbc3400b330 comp rx=0 tx=0).stop 2024-04-25T19:20:20.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 -- 172.21.15.86:0/1866753011 shutdown_connections 2024-04-25T19:20:20.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 --2- 172.21.15.86:0/1866753011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c10f900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 --2- 172.21.15.86:0/1866753011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c108a30 0x7fbc3c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 --2- 172.21.15.86:0/1866753011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc3c06a750 0x7fbc3c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 -- 172.21.15.86:0/1866753011 >> 172.21.15.86:0/1866753011 conn(0x7fbc3c0788f0 msgr2=0x7fbc3c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:20.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 -- 172.21.15.86:0/1866753011 shutdown_connections 2024-04-25T19:20:20.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.568+0000 7fbc43000700 1 -- 172.21.15.86:0/1866753011 wait complete. 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.569+0000 7fbc43000700 1 Processor -- start 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 -- start start 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06a750 0x7fbc3c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc3c108a30 0x7fbc3c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c112c10 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c112a90 con 0x7fbc3c06a750 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.570+0000 7fbc43000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c112d90 con 0x7fbc3c108a30 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60324/0 (socket says 172.21.15.86:60324) 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 -- 172.21.15.86:0/2246195211 learned_addr learned my addr 172.21.15.86:0/2246195211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3be00700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06a750 0x7fbc3c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc41000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc3c108a30 0x7fbc3c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 -- 172.21.15.86:0/2246195211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc3c108a30 msgr2=0x7fbc3c102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc3c108a30 0x7fbc3c102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 -- 172.21.15.86:0/2246195211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06a750 msgr2=0x7fbc3c101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06a750 0x7fbc3c101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 -- 172.21.15.86:0/2246195211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c1a7600 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3be00700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06a750 0x7fbc3c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:20.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.571+0000 7fbc3b400700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c101f50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e9c0 tx=0x7fbc2800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.572+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800cbd0 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.572+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc2800ee60 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.572+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280055c0 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.572+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c1a78f0 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.572+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a7de0 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.573+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c069730 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.575+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc28004750 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.577+0000 7fbc38c00700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbc200a3e30 0x7fbc200a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:20.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.577+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fbc280d0ac0 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.577+0000 7fbc3be00700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbc200a3e30 0x7fbc200a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:20.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.578+0000 7fbc3be00700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbc200a3e30 0x7fbc200a62f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005fd0 tx=0x7fbc2c009310 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:20.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.581+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc2809d090 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.860+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc3c02df80 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.861+0000 7fbc38c00700 1 -- 172.21.15.86:0/2246195211 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fbc2809c990 con 0x7fbc3c06b0f0 2024-04-25T19:20:20.866 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:20:20.867 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:20:20.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.864+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbc200a3e30 msgr2=0x7fbc200a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:20.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.864+0000 7fbc43000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbc200a3e30 0x7fbc200a62f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005fd0 tx=0x7fbc2c009310 comp rx=0 tx=0).stop 2024-04-25T19:20:20.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.864+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 msgr2=0x7fbc3c101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:20.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.864+0000 7fbc43000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c101f50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e9c0 tx=0x7fbc2800ecd0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 shutdown_connections 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fbc200a3e30 0x7fbc200a62f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc3c108a30 0x7fbc3c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc3c06a750 0x7fbc3c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 --2- 172.21.15.86:0/2246195211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc3c06b0f0 0x7fbc3c101f50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 >> 172.21.15.86:0/2246195211 conn(0x7fbc3c0788f0 msgr2=0x7fbc3c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 shutdown_connections 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:20.865+0000 7fbc43000700 1 -- 172.21.15.86:0/2246195211 wait complete. 2024-04-25T19:20:20.868 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:20:20.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:20 smithi194 ceph-mon[32740]: pgmap v548: 97 pgs: 97 active+clean; 5.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:20 smithi086 ceph-mon[27783]: pgmap v548: 97 pgs: 97 active+clean; 5.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:21.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:20 smithi183 ceph-mon[32344]: pgmap v548: 97 pgs: 97 active+clean; 5.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:21.720 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db / --timeout 60 --expiration 89.8 --await-for 42.0' 2024-04-25T19:20:21.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db / --timeout 60 --expiration 89.8 --await-for 42.0 2024-04-25T19:20:21.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:21 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2246195211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:20:22.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:21 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2246195211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:20:22.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:21 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2246195211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:20:22.169 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:20:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 -- 172.21.15.86:0/828391196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 msgr2=0x7fd1981045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 --2- 172.21.15.86:0/828391196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981045e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd188009900 tx=0x7fd18802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 -- 172.21.15.86:0/828391196 shutdown_connections 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 --2- 172.21.15.86:0/828391196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1981092d0 0x7fd198110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 --2- 172.21.15.86:0/828391196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd198104bb0 0x7fd198108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 --2- 172.21.15.86:0/828391196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981045e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 -- 172.21.15.86:0/828391196 >> 172.21.15.86:0/828391196 conn(0x7fd1980ffa60 msgr2=0x7fd198101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 -- 172.21.15.86:0/828391196 shutdown_connections 2024-04-25T19:20:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.614+0000 7fd197e00640 1 -- 172.21.15.86:0/828391196 wait complete. 2024-04-25T19:20:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.615+0000 7fd197e00640 1 Processor -- start 2024-04-25T19:20:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.615+0000 7fd197e00640 1 -- start start 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1981092d0 0x7fd1981a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd19819a660 0x7fd19819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd198114280 con 0x7fd1981092d0 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd198114100 con 0x7fd198104200 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd198114400 con 0x7fd19819a660 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd196a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd196000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1981092d0 0x7fd1981a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd19819a660 0x7fd19819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd196a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46652/0 (socket says 172.21.15.86:46652) 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd196a00640 1 -- 172.21.15.86:0/50416447 learned_addr learned my addr 172.21.15.86:0/50416447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197400640 1 -- 172.21.15.86:0/50416447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 msgr2=0x7fd1981a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197400640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197400640 1 -- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1981092d0 msgr2=0x7fd1981a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197400640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1981092d0 0x7fd1981a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.616+0000 7fd197400640 1 -- 172.21.15.86:0/50416447 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19819b180 con 0x7fd19819a660 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.617+0000 7fd197400640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd19819a660 0x7fd19819ab10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd19000ec50 tx=0x7fd19000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.617+0000 7fd19ca00640 1 -- 172.21.15.86:0/50416447 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190003b70 con 0x7fd19819a660 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.617+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19819b470 con 0x7fd19819a660 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.617+0000 7fd19ca00640 1 -- 172.21.15.86:0/50416447 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd190003d10 con 0x7fd19819a660 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.617+0000 7fd19ca00640 1 -- 172.21.15.86:0/50416447 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190010830 con 0x7fd19819a660 2024-04-25T19:20:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.617+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a7670 con 0x7fd19819a660 2024-04-25T19:20:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.620+0000 7fd1a0d80740 1 Processor -- start 2024-04-25T19:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd19ca00640 1 -- 172.21.15.86:0/50416447 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd190003ec0 con 0x7fd19819a660 2024-04-25T19:20:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 -- start start 2024-04-25T19:20:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d669d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d674650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d676f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb4d5cf900 con 0x55cb4d66a2d0 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb4d575ef0 con 0x55cb4d674b90 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.621+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb4d5cb8c0 con 0x55cb4d6699b0 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d674650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d669d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d669d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51290/0 (socket says 172.21.15.86:51290) 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d674650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60358/0 (socket says 172.21.15.86:60358) 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196000640 1 -- 172.21.15.86:0/139908522 learned_addr learned my addr 172.21.15.86:0/139908522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196000640 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d676f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 -- 172.21.15.86:0/139908522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 msgr2=0x55cb4d669d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d669d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 -- 172.21.15.86:0/139908522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 msgr2=0x55cb4d676f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d676f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196a00640 1 -- 172.21.15.86:0/139908522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4d2e2210 con 0x55cb4d66a2d0 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.622+0000 7fd196000640 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d669d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd196a00640 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d674650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd188000c00 tx=0x7fd188005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b0a587a9c7dcff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd19ca00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd17c0a3d90 0x7fd17c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd183e00640 1 -- 172.21.15.86:0/139908522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188040070 con 0x55cb4d66a2d0 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd183e00640 1 -- 172.21.15.86:0/139908522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1880054d0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd183e00640 1 -- 172.21.15.86:0/139908522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188035040 con 0x55cb4d66a2d0 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd19ca00640 1 -- 172.21.15.86:0/50416447 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fd1900d0cf0 con 0x7fd19819a660 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.623+0000 7fd196000640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd17c0a3d90 0x7fd17c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 -- 172.21.15.86:0/139908522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 msgr2=0x55cb4d674650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d674650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd188000c00 tx=0x7fd188005c00 comp rx=0 tx=0).stop 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd196000640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd17c0a3d90 0x7fd17c0a6250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55cb4d6672e0 tx=0x7fd184005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 -- 172.21.15.86:0/139908522 shutdown_connections 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d676f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d674650 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/139908522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d669d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 -- 172.21.15.86:0/139908522 >> 172.21.15.86:0/139908522 conn(0x55cb4d665ec0 msgr2=0x55cb4d6662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 -- 172.21.15.86:0/139908522 shutdown_connections 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.624+0000 7fd1a0d80740 1 -- 172.21.15.86:0/139908522 wait complete. 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 -- start start 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 Processor -- start 2024-04-25T19:20:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d671b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d6720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d6725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb4d677760 con 0x55cb4d66a2d0 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb4d589310 con 0x55cb4d674b90 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb4d6778e0 con 0x55cb4d6699b0 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.625+0000 7fd1a0d80740 10 client.0 _ll_register_callbacks cb 0x55cb4d557cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d6720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d6720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60364/0 (socket says 172.21.15.86:60364) 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 -- 172.21.15.86:0/2813222833 learned_addr learned my addr 172.21.15.86:0/2813222833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd197400640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d6725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196a00640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d671b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 -- 172.21.15.86:0/2813222833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 msgr2=0x55cb4d671b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d671b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 -- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 msgr2=0x55cb4d6725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d6725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd196000640 1 -- 172.21.15.86:0/2813222833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4d3ae620 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.626+0000 7fd197400640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d6725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd196000640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d6720b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd184011b20 tx=0x7fd1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233881581889aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd180c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd180c00640 1 -- 172.21.15.86:0/2813222833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184004480 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd180c00640 1 -- 172.21.15.86:0/2813222833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fd184022bc0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1880054d0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd180c00640 1 -- 172.21.15.86:0/2813222833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd184022dc0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd1a0d80740 10 client.17253 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb4d683a10 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.627+0000 7fd180c00640 1 -- 172.21.15.86:0/2813222833 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1880054d0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd180c00640 1 -- 172.21.15.86:0/2813222833 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd18402c3f0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd1a0d80740 10 client.17253 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd1a0d80740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd1a0d80740 20 client.17253 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d684f90 0x55cb4d687380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd1a0d80740 4 client.17253 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb4d5e38b0 con 0x55cb4d684f90 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.628+0000 7fd196a00640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d684f90 0x55cb4d687380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.629+0000 7fd196a00640 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d684f90 0x55cb4d687380 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b137de0cd0ef0a7 server_cookie=96328a7dc77ce808 in_seq=0 out_seq=0 2024-04-25T19:20:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.629+0000 7fd180c00640 10 client.17253 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd180c00640 1 -- 172.21.15.86:0/2813222833 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb4d5e38b0 con 0x55cb4d684f90 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd180c00640 10 client.17253 handle_command_reply: tid=0 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 client.17253 shutdown 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 msgr2=0x55cb4d6720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d6720b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd184011b20 tx=0x7fd1840046c0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 shutdown_connections 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d684f90 0x55cb4d687380 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d674b90 0x55cb4d6725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d66a2d0 0x55cb4d6720b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd180c00640 0 client.17253 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2813222833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d671b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 >> 172.21.15.86:0/2813222833 conn(0x55cb4d665ec0 msgr2=0x55cb4d66bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 shutdown_connections 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2813222833 wait complete. 2024-04-25T19:20:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.630+0000 7fd1a0d80740 20 client.17253 trim_cache size 0 max 16384 2024-04-25T19:20:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 Processor -- start 2024-04-25T19:20:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 -- start start 2024-04-25T19:20:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d680620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d68cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d68d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981afe10 con 0x55cb4d674b90 2024-04-25T19:20:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b1950 con 0x55cb4d66a2d0 2024-04-25T19:20:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b04e0 con 0x55cb4d6699b0 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd196000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d68cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd196000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d68cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46688/0 (socket says 172.21.15.86:46688) 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd196000640 1 -- 172.21.15.86:0/1768948017 learned_addr learned my addr 172.21.15.86:0/1768948017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.647+0000 7fd196a00640 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d680620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd197400640 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d68d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd196000640 1 -- 172.21.15.86:0/1768948017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 msgr2=0x55cb4d680620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd196000640 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d680620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd196000640 1 -- 172.21.15.86:0/1768948017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 msgr2=0x55cb4d68d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd196000640 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d68d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd196000640 1 -- 172.21.15.86:0/1768948017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4d66a100 con 0x55cb4d66a2d0 2024-04-25T19:20:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd196000640 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d68cc70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd184011af0 tx=0x7fd184009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b324711cee6d4e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd183400640 1 -- 172.21.15.86:0/1768948017 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184005d30 con 0x55cb4d66a2d0 2024-04-25T19:20:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd183400640 1 -- 172.21.15.86:0/1768948017 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd184022800 con 0x55cb4d66a2d0 2024-04-25T19:20:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.648+0000 7fd183400640 1 -- 172.21.15.86:0/1768948017 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184010430 con 0x55cb4d66a2d0 2024-04-25T19:20:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 -- 172.21.15.86:0/1768948017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 msgr2=0x55cb4d68cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d68cc70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd184011af0 tx=0x7fd184009620 comp rx=0 tx=0).stop 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 -- 172.21.15.86:0/1768948017 shutdown_connections 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d68d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d68cc70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/1768948017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d680620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 -- 172.21.15.86:0/1768948017 >> 172.21.15.86:0/1768948017 conn(0x55cb4d665ec0 msgr2=0x55cb4d66cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.649+0000 7fd1a0d80740 1 -- 172.21.15.86:0/1768948017 shutdown_connections 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.650+0000 7fd1a0d80740 1 -- 172.21.15.86:0/1768948017 wait complete. 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.650+0000 7fd1a0d80740 1 -- start start 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.650+0000 7fd1a0d80740 1 Processor -- start 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d67cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d67a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d67a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b1280 con 0x55cb4d674b90 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b0bb0 con 0x55cb4d66a2d0 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981ade10 con 0x55cb4d6699b0 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd1a0d80740 10 client.0 _ll_register_callbacks cb 0x7fd1840119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd196a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d67cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd197400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d67a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd196a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d67cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51306/0 (socket says 172.21.15.86:51306) 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd196a00640 1 -- 172.21.15.86:0/2286499181 learned_addr learned my addr 172.21.15.86:0/2286499181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd197400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d67a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60380/0 (socket says 172.21.15.86:60380) 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.651+0000 7fd196000640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d67a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd196a00640 1 -- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 msgr2=0x55cb4d67a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd196a00640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d67a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd196a00640 1 -- 172.21.15.86:0/2286499181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 msgr2=0x55cb4d67a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd196a00640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d67a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd196a00640 1 -- 172.21.15.86:0/2286499181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4d67ab60 con 0x55cb4d6699b0 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd197400640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d67a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd196a00640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d67cbe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd18802a9e0 tx=0x7fd188004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56741f2469820d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd180c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd180c00640 1 -- 172.21.15.86:0/2286499181 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188040040 con 0x55cb4d6699b0 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.652+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd184022800 con 0x55cb4d6699b0 2024-04-25T19:20:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.653+0000 7fd180c00640 1 -- 172.21.15.86:0/2286499181 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fd1880049a0 con 0x55cb4d6699b0 2024-04-25T19:20:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.653+0000 7fd180c00640 1 -- 172.21.15.86:0/2286499181 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd188035040 con 0x55cb4d6699b0 2024-04-25T19:20:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.653+0000 7fd1a0d80740 10 client.27346 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.653+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb4d6eb1f0 con 0x55cb4d6699b0 2024-04-25T19:20:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.653+0000 7fd180c00640 1 -- 172.21.15.86:0/2286499181 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd184022800 con 0x55cb4d6699b0 2024-04-25T19:20:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd180c00640 1 -- 172.21.15.86:0/2286499181 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd1880038f0 con 0x55cb4d6699b0 2024-04-25T19:20:22.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd1a0d80740 10 client.27346 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd1a0d80740 10 client.27346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd1a0d80740 20 client.27346 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d6eb4d0 0x55cb4d6ed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd1a0d80740 4 client.27346 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb4d5dd320 con 0x55cb4d6eb4d0 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.654+0000 7fd196000640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d6eb4d0 0x55cb4d6ed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.655+0000 7fd180c00640 10 client.27346 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.655+0000 7fd196000640 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d6eb4d0 0x55cb4d6ed8c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd975d054282b2d server_cookie=49f658b9709de496 in_seq=0 out_seq=0 2024-04-25T19:20:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.668+0000 7fd180c00640 1 -- 172.21.15.86:0/2286499181 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb4d5dd320 con 0x55cb4d6eb4d0 2024-04-25T19:20:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.668+0000 7fd180c00640 10 client.27346 handle_command_reply: tid=0 2024-04-25T19:20:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.668+0000 7fd1a0d80740 1 client.27346 shutdown 2024-04-25T19:20:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 msgr2=0x55cb4d67cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d67cbe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd18802a9e0 tx=0x7fd188004200 comp rx=0 tx=0).stop 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 shutdown_connections 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55cb4d6eb4d0 0x55cb4d6ed8c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb4d674b90 0x55cb4d67a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb4d66a2d0 0x55cb4d67a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 --2- 172.21.15.86:0/2286499181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb4d6699b0 0x55cb4d67cbe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 >> 172.21.15.86:0/2286499181 conn(0x55cb4d665ec0 msgr2=0x55cb4d66b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd180c00640 0 client.27346 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 shutdown_connections 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 1 -- 172.21.15.86:0/2286499181 wait complete. 2024-04-25T19:20:22.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.669+0000 7fd1a0d80740 20 client.27346 trim_cache size 0 max 16384 2024-04-25T19:20:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:20:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:20:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:20:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd17c0a3d90 msgr2=0x7fd17c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd17c0a3d90 0x7fd17c0a6250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55cb4d6672e0 tx=0x7fd184005f90 comp rx=0 tx=0).stop 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd19819a660 msgr2=0x7fd19819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd19819a660 0x7fd19819ab10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd19000ec50 tx=0x7fd19000c6a0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 shutdown_connections 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd17c0a3d90 0x7fd17c0a6250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd19819a660 0x7fd19819ab10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1981092d0 0x7fd1981a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 --2- 172.21.15.86:0/50416447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd198104200 0x7fd1981a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.671+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 >> 172.21.15.86:0/50416447 conn(0x7fd1980ffa60 msgr2=0x7fd198100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.672+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 shutdown_connections 2024-04-25T19:20:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:22.672+0000 7fd197e00640 1 -- 172.21.15.86:0/50416447 wait complete. 2024-04-25T19:20:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:22 smithi194 ceph-mon[32740]: pgmap v549: 97 pgs: 97 active+clean; 5.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:22 smithi086 ceph-mon[27783]: pgmap v549: 97 pgs: 97 active+clean; 5.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:23.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:22 smithi183 ceph-mon[32344]: pgmap v549: 97 pgs: 97 active+clean; 5.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T19:20:23.615 INFO:tasks.workunit.client.0.smithi086.stdout:561/600: open fd = 3 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:562/600: open fd = 3 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:563/600: open fd = 3 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:564/600: open fd = 3 2024-04-25T19:20:23.616 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:565/600: open fd = 3 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:566/600: open fd = 3 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:567/600: open fd = 3 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.617 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:568/600: open fd = 3 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:569/600: open fd = 3 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:570/600: open fd = 3 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:571/600: open fd = 3 2024-04-25T19:20:23.618 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:572/600: open fd = 3 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:573/600: open fd = 3 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:574/600: open fd = 3 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.619 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:575/600: open fd = 3 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:576/600: open fd = 3 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:577/600: open fd = 3 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:578/600: open fd = 3 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.620 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:579/600: open fd = 3 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:580/600: open fd = 3 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:581/600: open fd = 3 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:582/600: open fd = 3 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.621 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:583/600: open fd = 3 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:584/600: open fd = 3 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:585/600: open fd = 3 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.622 INFO:tasks.workunit.client.0.smithi086.stdout:586/600: open fd = 3 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:587/600: open fd = 3 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:588/600: open fd = 3 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:589/600: open fd = 3 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.623 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:590/600: open fd = 3 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:591/600: open fd = 3 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:592/600: open fd = 3 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:593/600: open fd = 3 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.624 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:594/600: open fd = 3 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:595/600: open fd = 3 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:596/600: open fd = 3 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.625 INFO:tasks.workunit.client.0.smithi086.stdout:597/600: open fd = 3 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:598/600: open fd = 3 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:599/600: open fd = 3 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:600/600: open fd = 3 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.626 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-25T19:20:23.627 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-25T19:20:23.627 INFO:tasks.workunit.client.0.smithi086.stderr:+ echo OK 2024-04-25T19:20:23.628 INFO:tasks.workunit.client.0.smithi086.stdout:OK 2024-04-25T19:20:23.628 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T19:20:23.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T19:20:23.727 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-25T19:20:23.728 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.551+0000 7fda8aa00700 1 -- 172.21.15.86:0/3098427904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102d90 msgr2=0x7fda8410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.551+0000 7fda8aa00700 1 --2- 172.21.15.86:0/3098427904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102d90 0x7fda8410efb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009990 tx=0x7fda6c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 -- 172.21.15.86:0/3098427904 shutdown_connections 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 --2- 172.21.15.86:0/3098427904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 0x7fda841118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 --2- 172.21.15.86:0/3098427904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102470 0x7fda84102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 --2- 172.21.15.86:0/3098427904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102d90 0x7fda8410efb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 -- 172.21.15.86:0/3098427904 >> 172.21.15.86:0/3098427904 conn(0x7fda840788e0 msgr2=0x7fda84078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 -- 172.21.15.86:0/3098427904 shutdown_connections 2024-04-25T19:20:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.552+0000 7fda8aa00700 1 -- 172.21.15.86:0/3098427904 wait complete. 2024-04-25T19:20:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.553+0000 7fda8aa00700 1 Processor -- start 2024-04-25T19:20:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.553+0000 7fda8aa00700 1 -- start start 2024-04-25T19:20:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102470 0x7fda8419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda8aa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 0x7fda8419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda8aa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 0x7fda8419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda8aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda84114e90 con 0x7fda84102470 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda8aa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda84114d10 con 0x7fda8410f4f0 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda8aa00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fda84115010 con 0x7fda84102d90 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 0x7fda8419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 0x7fda8419b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51326/0 (socket says 172.21.15.86:51326) 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 -- 172.21.15.86:0/1455092394 learned_addr learned my addr 172.21.15.86:0/1455092394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda83e00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102470 0x7fda8419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda88a00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 0x7fda8419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 -- 172.21.15.86:0/1455092394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 msgr2=0x7fda8419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 0x7fda8419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 -- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102470 msgr2=0x7fda8419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.554+0000 7fda7be00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102470 0x7fda8419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda7be00700 1 -- 172.21.15.86:0/1455092394 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda841a02c0 con 0x7fda84102d90 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda88a00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 0x7fda8419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda7be00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 0x7fda8419b840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda81600700 1 -- 172.21.15.86:0/1455092394 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c027070 con 0x7fda84102d90 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda81600700 1 -- 172.21.15.86:0/1455092394 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda6c022470 con 0x7fda84102d90 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda81600700 1 -- 172.21.15.86:0/1455092394 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c017650 con 0x7fda84102d90 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda841a0550 con 0x7fda84102d90 2024-04-25T19:20:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.555+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841a7e30 con 0x7fda84102d90 2024-04-25T19:20:24.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.559+0000 7fda81600700 1 -- 172.21.15.86:0/1455092394 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fda6c02a070 con 0x7fda84102d90 2024-04-25T19:20:24.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 Processor -- start 2024-04-25T19:20:24.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 -- start start 2024-04-25T19:20:24.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda81600700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fda680a3ed0 0x7fda680a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x5576678581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x557667863020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x557667865950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576677ab020 con 0x5576678586f0 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5576677518c0 con 0x557667857dd0 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda95466b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5576677a6fe0 con 0x557667863560 2024-04-25T19:20:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.560+0000 7fda81600700 1 -- 172.21.15.86:0/1455092394 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fda6c0d1590 con 0x7fda84102d90 2024-04-25T19:20:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda88a00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fda680a3ed0 0x7fda680a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x5576678581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x5576678581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:46704/0 (socket says 172.21.15.86:46704) 2024-04-25T19:20:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x557667863020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 -- 172.21.15.86:0/243677463 learned_addr learned my addr 172.21.15.86:0/243677463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 -- 172.21.15.86:0/243677463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 msgr2=0x557667865950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x557667865950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 -- 172.21.15.86:0/243677463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 msgr2=0x557667863020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x557667863020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda83e00700 1 -- 172.21.15.86:0/243677463 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557667747810 con 0x557667857dd0 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.561+0000 7fda7be00700 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x557667863020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.562+0000 7fda83e00700 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x5576678581b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d20 tx=0x7fda740116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a046cc428f10f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.562+0000 7fda88a00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fda680a3ed0 0x7fda680a6390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5576678597a0 tx=0x7fda7c009920 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.562+0000 7fda79600700 1 -- 172.21.15.86:0/243677463 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011e20 con 0x557667857dd0 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.562+0000 7fda79600700 1 -- 172.21.15.86:0/243677463 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda74012900 con 0x557667857dd0 2024-04-25T19:20:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.562+0000 7fda79600700 1 -- 172.21.15.86:0/243677463 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400f840 con 0x557667857dd0 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 -- 172.21.15.86:0/243677463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 msgr2=0x5576678581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x5576678581b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d20 tx=0x7fda740116c0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 -- 172.21.15.86:0/243677463 shutdown_connections 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x557667865950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x5576678581b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 --2- 172.21.15.86:0/243677463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x557667863020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 -- 172.21.15.86:0/243677463 >> 172.21.15.86:0/243677463 conn(0x5576677870e0 msgr2=0x5576678542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 -- 172.21.15.86:0/243677463 shutdown_connections 2024-04-25T19:20:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.563+0000 7fda95466b80 1 -- 172.21.15.86:0/243677463 wait complete. 2024-04-25T19:20:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 -- start start 2024-04-25T19:20:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 Processor -- start 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 0x557667862c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5576678586f0 0x55766785ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667860490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557667865f30 con 0x557667857dd0 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5576677649f0 con 0x557667863560 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.564+0000 7fda95466b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5576678660b0 con 0x5576678586f0 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda95466b80 10 client.0 _ll_register_callbacks cb 0x5576677438b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 0x557667862c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 0x557667862c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60418/0 (socket says 172.21.15.86:60418) 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 -- 172.21.15.86:0/2450999088 learned_addr learned my addr 172.21.15.86:0/2450999088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda7be00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5576678586f0 0x55766785ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda88a00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667860490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 -- 172.21.15.86:0/2450999088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5576678586f0 msgr2=0x55766785ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5576678586f0 0x55766785ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 -- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 msgr2=0x557667860490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667860490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 -- 172.21.15.86:0/2450999088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576676f8a60 con 0x557667857dd0 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda7be00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5576678586f0 0x55766785ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda88a00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667860490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda83e00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 0x557667862c50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fda740096e0 tx=0x7fda74012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cee8530eb1bf916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.565+0000 7fda72000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda72000700 1 -- 172.21.15.86:0/2450999088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74013dd0 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda72000700 1 -- 172.21.15.86:0/2450999088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fda740050a0 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda74012900 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda72000700 1 -- 172.21.15.86:0/2450999088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda740052e0 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda95466b80 10 client.17271 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557667741c80 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.566+0000 7fda72000700 1 -- 172.21.15.86:0/2450999088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda74012900 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda72000700 1 -- 172.21.15.86:0/2450999088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fda74024450 con 0x557667857dd0 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda95466b80 10 client.17271 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda95466b80 10 client.17271 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda95466b80 20 client.17271 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda95466b80 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557667872bc0 0x557667874fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda95466b80 4 client.17271 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:20:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576677b8810 con 0x557667872bc0 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.567+0000 7fda7be00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557667872bc0 0x557667874fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.568+0000 7fda7be00700 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557667872bc0 0x557667874fb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258b53bf26219c05 server_cookie=ab5a958a62b7d67e in_seq=0 out_seq=0 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.568+0000 7fda72000700 10 client.17271 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.568+0000 7fda72000700 1 -- 172.21.15.86:0/2450999088 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576677b8810 con 0x557667872bc0 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.568+0000 7fda72000700 10 client.17271 handle_command_reply: tid=0 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 client.17271 shutdown 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 msgr2=0x557667862c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 0x557667862c50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fda740096e0 tx=0x7fda74012750 comp rx=0 tx=0).stop 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 shutdown_connections 2024-04-25T19:20:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x557667872bc0 0x557667874fb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667860490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557667857dd0 0x557667862c50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 --2- 172.21.15.86:0/2450999088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5576678586f0 0x55766785ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 >> 172.21.15.86:0/2450999088 conn(0x5576677870e0 msgr2=0x557667853ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda72000700 0 client.17271 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 shutdown_connections 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 1 -- 172.21.15.86:0/2450999088 wait complete. 2024-04-25T19:20:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.569+0000 7fda95466b80 20 client.17271 trim_cache size 0 max 16384 2024-04-25T19:20:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.590+0000 7fda95466b80 1 Processor -- start 2024-04-25T19:20:24.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.590+0000 7fda95466b80 1 -- start start 2024-04-25T19:20:24.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda95466b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x55766785f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda95466b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x55766786ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda95466b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x55766787a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda95466b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda841af5d0 con 0x5576678586f0 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda95466b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b1110 con 0x557667857dd0 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda95466b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fda841afca0 con 0x557667863560 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x55766786ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x55766786ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60426/0 (socket says 172.21.15.86:60426) 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda7be00700 1 -- 172.21.15.86:0/2264179338 learned_addr learned my addr 172.21.15.86:0/2264179338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda88a00700 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x55766787a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.591+0000 7fda83e00700 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x55766785f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda7be00700 1 -- 172.21.15.86:0/2264179338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 msgr2=0x55766787a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda7be00700 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x55766787a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda7be00700 1 -- 172.21.15.86:0/2264179338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 msgr2=0x55766785f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda7be00700 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x55766785f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda7be00700 1 -- 172.21.15.86:0/2264179338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557667859590 con 0x5576678586f0 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda7be00700 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x55766786ee90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00c010 tx=0x7fda6c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e36b045180fa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda78c00700 1 -- 172.21.15.86:0/2264179338 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c086030 con 0x5576678586f0 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda78c00700 1 -- 172.21.15.86:0/2264179338 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda6c0a18a0 con 0x5576678586f0 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.592+0000 7fda78c00700 1 -- 172.21.15.86:0/2264179338 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c09d750 con 0x5576678586f0 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 -- 172.21.15.86:0/2264179338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 msgr2=0x55766786ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x55766786ee90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00c010 tx=0x7fda6c01a040 comp rx=0 tx=0).stop 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 -- 172.21.15.86:0/2264179338 shutdown_connections 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667863560 0x55766787a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667857dd0 0x55766785f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 --2- 172.21.15.86:0/2264179338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x55766786ee90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 -- 172.21.15.86:0/2264179338 >> 172.21.15.86:0/2264179338 conn(0x5576677870e0 msgr2=0x557667862010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:24.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 -- 172.21.15.86:0/2264179338 shutdown_connections 2024-04-25T19:20:24.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.593+0000 7fda95466b80 1 -- 172.21.15.86:0/2264179338 wait complete. 2024-04-25T19:20:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 -- start start 2024-04-25T19:20:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 Processor -- start 2024-04-25T19:20:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 0x557667868370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x5576678688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667868df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b0a40 con 0x5576678586f0 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda841b0370 con 0x557667863560 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fda841ad590 con 0x557667857dd0 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.594+0000 7fda95466b80 10 client.0 _ll_register_callbacks cb 0x5576677438b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 0x557667868370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda88a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667868df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 0x557667868370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:51362/0 (socket says 172.21.15.86:51362) 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 -- 172.21.15.86:0/3619833658 learned_addr learned my addr 172.21.15.86:0/3619833658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda7be00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x5576678688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 -- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 msgr2=0x557667868df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667868df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 -- 172.21.15.86:0/3619833658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 msgr2=0x5576678688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x5576678688b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 -- 172.21.15.86:0/3619833658 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576678d8fa0 con 0x557667857dd0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda88a00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667868df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda83e00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 0x557667868370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda74024ab0 tx=0x7fda74011f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6b09f3670b39f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.595+0000 7fda72000700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda72000700 1 -- 172.21.15.86:0/3619833658 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74009350 con 0x557667857dd0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6c0a18a0 con 0x557667857dd0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda72000700 1 -- 172.21.15.86:0/3619833658 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fda74005480 con 0x557667857dd0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda72000700 1 -- 172.21.15.86:0/3619833658 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda74045050 con 0x557667857dd0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda95466b80 10 client.27361 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576678da1a0 con 0x557667857dd0 2024-04-25T19:20:24.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda72000700 1 -- 172.21.15.86:0/3619833658 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda6c0a18a0 con 0x557667857dd0 2024-04-25T19:20:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.596+0000 7fda72000700 1 -- 172.21.15.86:0/3619833658 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fda74003680 con 0x557667857dd0 2024-04-25T19:20:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda95466b80 10 client.27361 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda95466b80 10 client.27361 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:20:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda95466b80 20 client.27361 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda95466b80 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5576678da970 0x5576678dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:24.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda95466b80 4 client.27361 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.8, "await_for": 42.0}] 2024-04-25T19:20:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.8, "await_for": 42.0}) -- 0x5576677b28b0 con 0x5576678da970 2024-04-25T19:20:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda7be00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5576678da970 0x5576678dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda72000700 10 client.27361 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:24.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:24.597+0000 7fda7be00700 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5576678da970 0x5576678dcd90 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364169a327f9ba5 server_cookie=b69bd6d707fee7a6 in_seq=0 out_seq=0 2024-04-25T19:20:24.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:24 smithi086 ceph-mon[27783]: pgmap v550: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 KiB/s wr, 19 op/s 2024-04-25T19:20:24.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:24 smithi194 ceph-mon[32740]: pgmap v550: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 KiB/s wr, 19 op/s 2024-04-25T19:20:24.987 DEBUG:teuthology.parallel:result is None 2024-04-25T19:20:24.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda72000700 1 -- 172.21.15.86:0/3619833658 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5576677b28b0 con 0x5576678da970 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda72000700 10 client.27361 handle_command_reply: tid=0 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 client.27361 shutdown 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 msgr2=0x557667868370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 0x557667868370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda74024ab0 tx=0x7fda74011f30 comp rx=0 tx=0).stop 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 shutdown_connections 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5576678da970 0x5576678dcd90 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557667863560 0x557667868df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557667857dd0 0x557667868370 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 --2- 172.21.15.86:0/3619833658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576678586f0 0x5576678688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda72000700 0 client.27361 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 >> 172.21.15.86:0/3619833658 conn(0x5576677870e0 msgr2=0x557667859c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 shutdown_connections 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.076+0000 7fda95466b80 1 -- 172.21.15.86:0/3619833658 wait complete. 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.077+0000 7fda95466b80 20 client.27361 trim_cache size 0 max 16384 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:20:25.079 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 54, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "d843219e": { 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 54, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 89.8, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:20:25.080 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:20:25.081 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fda680a3ed0 msgr2=0x7fda680a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fda680a3ed0 0x7fda680a6390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5576678597a0 tx=0x7fda7c009920 comp rx=0 tx=0).stop 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 msgr2=0x7fda8419b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 0x7fda8419b840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c009510 comp rx=0 tx=0).stop 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 shutdown_connections 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda8410f4f0 0x7fda8419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda84102470 0x7fda8419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fda680a3ed0 0x7fda680a6390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 --2- 172.21.15.86:0/1455092394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda84102d90 0x7fda8419b840 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 >> 172.21.15.86:0/1455092394 conn(0x7fda840788e0 msgr2=0x7fda84110a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 shutdown_connections 2024-04-25T19:20:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:25.078+0000 7fda8aa00700 1 -- 172.21.15.86:0/1455092394 wait complete. 2024-04-25T19:20:25.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:24 smithi183 ceph-mon[32344]: pgmap v550: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 50 KiB/s wr, 19 op/s 2024-04-25T19:20:25.940 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd843219e', quiesce took 8.1 seconds. Will release after: 29.8 2024-04-25T19:20:26.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:26 smithi194 ceph-mon[32740]: pgmap v551: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T19:20:27.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:26 smithi086 ceph-mon[27783]: pgmap v551: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T19:20:27.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:26 smithi183 ceph-mon[32344]: pgmap v551: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-04-25T19:20:28.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:28 smithi194 ceph-mon[32740]: pgmap v552: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:20:29.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:28 smithi086 ceph-mon[27783]: pgmap v552: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:20:29.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:28 smithi183 ceph-mon[32344]: pgmap v552: 97 pgs: 97 active+clean; 1.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T19:20:30.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:30 smithi086 ceph-mon[27783]: pgmap v553: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-25T19:20:30.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:30 smithi194 ceph-mon[32740]: pgmap v553: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-25T19:20:31.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:30 smithi183 ceph-mon[32344]: pgmap v553: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-25T19:20:32.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:32 smithi194 ceph-mon[32740]: pgmap v554: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-25T19:20:33.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:32 smithi086 ceph-mon[27783]: pgmap v554: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-25T19:20:33.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:32 smithi183 ceph-mon[32344]: pgmap v554: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-25T19:20:34.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:34 smithi194 ceph-mon[32740]: pgmap v555: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-04-25T19:20:35.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:34 smithi086 ceph-mon[27783]: pgmap v555: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-04-25T19:20:35.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:34 smithi183 ceph-mon[32344]: pgmap v555: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-04-25T19:20:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 -- 172.21.15.86:0/299050996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1070a0 msgr2=0x7fe4fc10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 --2- 172.21.15.86:0/299050996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1070a0 0x7fe4fc10ec70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009ca0 tx=0x7fe4f402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 -- 172.21.15.86:0/299050996 shutdown_connections 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 --2- 172.21.15.86:0/299050996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc1070a0 0x7fe4fc10ec70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 --2- 172.21.15.86:0/299050996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4fc102980 0x7fe4fc1069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 --2- 172.21.15.86:0/299050996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 -- 172.21.15.86:0/299050996 >> 172.21.15.86:0/299050996 conn(0x7fe4fc0fd890 msgr2=0x7fe4fc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 -- 172.21.15.86:0/299050996 shutdown_connections 2024-04-25T19:20:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.814+0000 7fe500c00640 1 -- 172.21.15.86:0/299050996 wait complete. 2024-04-25T19:20:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.815+0000 7fe500c00640 1 Processor -- start 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.815+0000 7fe500c00640 1 -- start start 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe500c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe500c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4fc1070a0 0x7fe4fc1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe500c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc19a670 0x7fe4fc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe500c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc112010 con 0x7fe4fc101fd0 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe500c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc111e90 con 0x7fe4fc19a670 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe500c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc112190 con 0x7fe4fc1070a0 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47598/0 (socket says 172.21.15.86:47598) 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 -- 172.21.15.86:0/2272689646 learned_addr learned my addr 172.21.15.86:0/2272689646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4faa00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4fc1070a0 0x7fe4fc1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc19a670 0x7fe4fc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 -- 172.21.15.86:0/2272689646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4fc1070a0 msgr2=0x7fe4fc1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4fc1070a0 0x7fe4fc1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 -- 172.21.15.86:0/2272689646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc19a670 msgr2=0x7fe4fc19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc19a670 0x7fe4fc19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fb400640 1 -- 172.21.15.86:0/2272689646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc19b190 con 0x7fe4fc101fd0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.816+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc19a670 0x7fe4fc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.817+0000 7fe4fb400640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1a0470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9f0 tx=0x7fe4ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.817+0000 7fe4f3e00640 1 -- 172.21.15.86:0/2272689646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00cd80 con 0x7fe4fc101fd0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.817+0000 7fe4f3e00640 1 -- 172.21.15.86:0/2272689646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ec0108e0 con 0x7fe4fc101fd0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.817+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc19b480 con 0x7fe4fc101fd0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.817+0000 7fe4f3e00640 1 -- 172.21.15.86:0/2272689646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec018980 con 0x7fe4fc101fd0 2024-04-25T19:20:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.817+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc1a7680 con 0x7fe4fc101fd0 2024-04-25T19:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe4f3e00640 1 -- 172.21.15.86:0/2272689646 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4ec010430 con 0x7fe4fc101fd0 2024-04-25T19:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 Processor -- start 2024-04-25T19:20:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 -- start start 2024-04-25T19:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed6c208900 con 0x55ed6c2a29b0 2024-04-25T19:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed6c1aeef0 con 0x55ed6c2a32d0 2024-04-25T19:20:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe50507f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ed6c2048c0 con 0x55ed6c2adb90 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.821+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47606/0 (socket says 172.21.15.86:47606) 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 -- 172.21.15.86:0/2591662896 learned_addr learned my addr 172.21.15.86:0/2591662896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4faa00640 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 -- 172.21.15.86:0/2591662896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 msgr2=0x55ed6c2aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 -- 172.21.15.86:0/2591662896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 msgr2=0x55ed6c2ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 -- 172.21.15.86:0/2591662896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed6bf1b210 con 0x55ed6c2a29b0 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fb400640 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4f3e00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe4e00a3d10 0x7fe4e00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.822+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2a2d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009390 tx=0x7fe4f40093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3c3c2b6662818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.823+0000 7fe4fb400640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe4e00a3d10 0x7fe4e00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.823+0000 7fe4f3e00640 1 -- 172.21.15.86:0/2272689646 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fe4ec0d0c60 con 0x7fe4fc101fd0 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.823+0000 7fe4e7e00640 1 -- 172.21.15.86:0/2591662896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4031cf0 con 0x55ed6c2a29b0 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.823+0000 7fe4e7e00640 1 -- 172.21.15.86:0/2591662896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4036040 con 0x55ed6c2a29b0 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.823+0000 7fe4e7e00640 1 -- 172.21.15.86:0/2591662896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40042a0 con 0x55ed6c2a29b0 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 -- 172.21.15.86:0/2591662896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 msgr2=0x55ed6c2a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2a2d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009390 tx=0x7fe4f40093c0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe4fb400640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe4e00a3d10 0x7fe4e00a61d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55ed6c2a3f50 tx=0x7fe4ec0d4710 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 -- 172.21.15.86:0/2591662896 shutdown_connections 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 --2- 172.21.15.86:0/2591662896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2a2d90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 -- 172.21.15.86:0/2591662896 >> 172.21.15.86:0/2591662896 conn(0x55ed6c29eec0 msgr2=0x55ed6c29f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 -- 172.21.15.86:0/2591662896 shutdown_connections 2024-04-25T19:20:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 -- 172.21.15.86:0/2591662896 wait complete. 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 -- start start 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.824+0000 7fe50507f740 1 Processor -- start 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed6c2b0760 con 0x55ed6c2a29b0 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed6c1c2310 con 0x55ed6c2a32d0 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ed6c2b08e0 con 0x55ed6c2adb90 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe50507f740 10 client.0 _ll_register_callbacks cb 0x55ed6c190cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47608/0 (socket says 172.21.15.86:47608) 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe4fbe00640 1 -- 172.21.15.86:0/1043095150 learned_addr learned my addr 172.21.15.86:0/1043095150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.825+0000 7fe4faa00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 -- 172.21.15.86:0/1043095150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 msgr2=0x55ed6c2ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2ab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 -- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 msgr2=0x55ed6c2ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 -- 172.21.15.86:0/1043095150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed6bfe7620 con 0x55ed6c2a29b0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4faa00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2aab70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4003eb0 tx=0x7fe4f402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecda86b04a1ac3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4e4c00640 1 -- 172.21.15.86:0/1043095150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4058060 con 0x55ed6c2a29b0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe4e4c00640 1 -- 172.21.15.86:0/1043095150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fe4f4031450 con 0x55ed6c2a29b0 2024-04-25T19:20:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.826+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4036040 con 0x55ed6c2a29b0 2024-04-25T19:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe4e4c00640 1 -- 172.21.15.86:0/1043095150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4f4031650 con 0x55ed6c2a29b0 2024-04-25T19:20:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 10 client.17289 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed6c2b0760 con 0x55ed6c2a29b0 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe4e4c00640 1 -- 172.21.15.86:0/1043095150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4036040 con 0x55ed6c2a29b0 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe4e4c00640 1 -- 172.21.15.86:0/1043095150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe4f4003f50 con 0x55ed6c2a29b0 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 10 client.17289 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 20 client.17289 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c2bdb90 0x55ed6c2bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 4 client.17289 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.827+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed6c21c8b0 con 0x55ed6c2bdb90 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.828+0000 7fe4faa00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c2bdb90 0x55ed6c2bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.828+0000 7fe4faa00640 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c2bdb90 0x55ed6c2bff80 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2a1c1a3c902940 server_cookie=befc2e26c0861d5e in_seq=0 out_seq=0 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.828+0000 7fe4e4c00640 10 client.17289 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.829+0000 7fe4e4c00640 1 -- 172.21.15.86:0/1043095150 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed6c21c8b0 con 0x55ed6c2bdb90 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.829+0000 7fe4e4c00640 10 client.17289 handle_command_reply: tid=0 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.829+0000 7fe50507f740 1 client.17289 shutdown 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 msgr2=0x55ed6c2aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2aab70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4003eb0 tx=0x7fe4f402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 shutdown_connections 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c2bdb90 0x55ed6c2bff80 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2adb90 0x55ed6c2ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a32d0 0x55ed6c2ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe4e4c00640 0 client.17289 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 --2- 172.21.15.86:0/1043095150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2aab70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 >> 172.21.15.86:0/1043095150 conn(0x55ed6c29eec0 msgr2=0x55ed6c2a4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 shutdown_connections 2024-04-25T19:20:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 1 -- 172.21.15.86:0/1043095150 wait complete. 2024-04-25T19:20:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.830+0000 7fe50507f740 20 client.17289 trim_cache size 0 max 16384 2024-04-25T19:20:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.846+0000 7fe50507f740 1 Processor -- start 2024-04-25T19:20:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.846+0000 7fe50507f740 1 -- start start 2024-04-25T19:20:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe50507f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe50507f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a32d0 0x55ed6c29e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe50507f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe50507f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1afeb0 con 0x55ed6c2adb90 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe50507f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b19f0 con 0x55ed6c2a29b0 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe50507f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b0580 con 0x55ed6c2a32d0 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47620/0 (socket says 172.21.15.86:47620) 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4faa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a32d0 0x55ed6c29e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4fbe00640 1 -- 172.21.15.86:0/681508237 learned_addr learned my addr 172.21.15.86:0/681508237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4faa00640 1 -- 172.21.15.86:0/681508237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a29b0 msgr2=0x55ed6c2b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4faa00640 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4faa00640 1 -- 172.21.15.86:0/681508237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 msgr2=0x55ed6c2aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4faa00640 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4faa00640 1 -- 172.21.15.86:0/681508237 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed6c2a3c60 con 0x55ed6c2a32d0 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.847+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.848+0000 7fe4faa00640 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a32d0 0x55ed6c29e260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55ed6c2a02e0 tx=0x7fe4e8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc6d56230ac8c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.848+0000 7fe4e7400640 1 -- 172.21.15.86:0/681508237 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8005020 con 0x55ed6c2a32d0 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.848+0000 7fe4e7400640 1 -- 172.21.15.86:0/681508237 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4e80047a0 con 0x55ed6c2a32d0 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.848+0000 7fe4e7400640 1 -- 172.21.15.86:0/681508237 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80223d0 con 0x55ed6c2a32d0 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 -- 172.21.15.86:0/681508237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a32d0 msgr2=0x55ed6c29e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a32d0 0x55ed6c29e260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55ed6c2a02e0 tx=0x7fe4e8005f90 comp rx=0 tx=0).stop 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 -- 172.21.15.86:0/681508237 shutdown_connections 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2adb90 0x55ed6c2aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a32d0 0x55ed6c29e260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 --2- 172.21.15.86:0/681508237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2a29b0 0x55ed6c2b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 -- 172.21.15.86:0/681508237 >> 172.21.15.86:0/681508237 conn(0x55ed6c29eec0 msgr2=0x55ed6c2a4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 -- 172.21.15.86:0/681508237 shutdown_connections 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 -- 172.21.15.86:0/681508237 wait complete. 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 -- start start 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.849+0000 7fe50507f740 1 Processor -- start 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a29b0 0x55ed6c31c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 0x55ed6c31c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2adb90 0x55ed6c2b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b1320 con 0x55ed6c2a32d0 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b0c50 con 0x55ed6c2adb90 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1adeb0 con 0x55ed6c2a29b0 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe50507f740 10 client.0 _ll_register_callbacks cb 0x7fe4e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe4faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 0x55ed6c31c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe4faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 0x55ed6c31c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47626/0 (socket says 172.21.15.86:47626) 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe4faa00640 1 -- 172.21.15.86:0/518292487 learned_addr learned my addr 172.21.15.86:0/518292487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a29b0 0x55ed6c31c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.850+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2adb90 0x55ed6c2b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4faa00640 1 -- 172.21.15.86:0/518292487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a29b0 msgr2=0x55ed6c31c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4faa00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a29b0 0x55ed6c31c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4faa00640 1 -- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2adb90 msgr2=0x55ed6c2b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4faa00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2adb90 0x55ed6c2b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4faa00640 1 -- 172.21.15.86:0/518292487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed6c2b3600 con 0x55ed6c2a32d0 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2adb90 0x55ed6c2b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4faa00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 0x55ed6c31c750 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8014010 tx=0x7fe4e8009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33a9d6bad823832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4e4c00640 1 -- 172.21.15.86:0/518292487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8004450 con 0x55ed6c2a32d0 2024-04-25T19:20:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4e4c00640 1 -- 172.21.15.86:0/518292487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fe4e8022170 con 0x55ed6c2a32d0 2024-04-25T19:20:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e80047a0 con 0x55ed6c2a32d0 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.851+0000 7fe4e4c00640 1 -- 172.21.15.86:0/518292487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4e8022370 con 0x55ed6c2a32d0 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 10 client.17295 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed6c323a20 con 0x55ed6c2a32d0 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe4e4c00640 1 -- 172.21.15.86:0/518292487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4e80047a0 con 0x55ed6c2a32d0 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe4e4c00640 1 -- 172.21.15.86:0/518292487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe4e800f500 con 0x55ed6c2a32d0 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 10 client.17295 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 20 client.17295 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c324120 0x55ed6c326540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 4 client.17295 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:20:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.852+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed6c216320 con 0x55ed6c324120 2024-04-25T19:20:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.853+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c324120 0x55ed6c326540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.853+0000 7fe4fbe00640 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c324120 0x55ed6c326540 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f99f49bec77c server_cookie=43d43782b3c11f65 in_seq=0 out_seq=0 2024-04-25T19:20:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.853+0000 7fe4e4c00640 10 client.17295 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe4e4c00640 1 -- 172.21.15.86:0/518292487 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed6c216320 con 0x55ed6c324120 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe4e4c00640 10 client.17295 handle_command_reply: tid=0 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 client.17295 shutdown 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 msgr2=0x55ed6c31c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 0x55ed6c31c750 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8014010 tx=0x7fe4e8009e40 comp rx=0 tx=0).stop 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 shutdown_connections 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed6c324120 0x55ed6c326540 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed6c2adb90 0x55ed6c2b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.869+0000 7fe4e4c00640 0 client.17295 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.868+0000 7fe50507f740 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed6c2a32d0 0x55ed6c31c750 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.869+0000 7fe50507f740 1 --2- 172.21.15.86:0/518292487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed6c2a29b0 0x55ed6c31c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.869+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 >> 172.21.15.86:0/518292487 conn(0x55ed6c29eec0 msgr2=0x55ed6c2a4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.869+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 shutdown_connections 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.869+0000 7fe50507f740 1 -- 172.21.15.86:0/518292487 wait complete. 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.869+0000 7fe50507f740 20 client.17295 trim_cache size 0 max 16384 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:20:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:20:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe4e00a3d10 msgr2=0x7fe4e00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe4e00a3d10 0x7fe4e00a61d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55ed6c2a3f50 tx=0x7fe4ec0d4710 comp rx=0 tx=0).stop 2024-04-25T19:20:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 msgr2=0x7fe4fc1a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1a0470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9f0 tx=0x7fe4ec00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 shutdown_connections 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe4e00a3d10 0x7fe4e00a61d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4fc19a670 0x7fe4fc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4fc1070a0 0x7fe4fc1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 --2- 172.21.15.86:0/2272689646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4fc101fd0 0x7fe4fc1a0470 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 >> 172.21.15.86:0/2272689646 conn(0x7fe4fc0fd890 msgr2=0x7fe4fc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 shutdown_connections 2024-04-25T19:20:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:35.870+0000 7fe500c00640 1 -- 172.21.15.86:0/2272689646 wait complete. 2024-04-25T19:20:36.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:36 smithi194 ceph-mon[32740]: pgmap v556: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-04-25T19:20:37.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:36 smithi086 ceph-mon[27783]: pgmap v556: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-04-25T19:20:37.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:36 smithi183 ceph-mon[32344]: pgmap v556: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-04-25T19:20:37.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:37 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:20:38.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:37 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:20:38.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:37 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:20:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:38 smithi086 ceph-mon[27783]: pgmap v557: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-04-25T19:20:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:38 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:20:39.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:38 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:20:39.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:38 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:20:39.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:38 smithi183 ceph-mon[32344]: pgmap v557: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-04-25T19:20:39.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:38 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:20:39.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:38 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:20:39.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:38 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:20:39.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:38 smithi194 ceph-mon[32740]: pgmap v557: 97 pgs: 97 active+clean; 1.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-04-25T19:20:39.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:38 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:20:39.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:38 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:20:39.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:38 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:20:41.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:40 smithi086 ceph-mon[27783]: pgmap v558: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T19:20:41.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:40 smithi183 ceph-mon[32344]: pgmap v558: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T19:20:41.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:40 smithi194 ceph-mon[32740]: pgmap v558: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T19:20:43.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:42 smithi086 ceph-mon[27783]: pgmap v559: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2024-04-25T19:20:43.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:42 smithi183 ceph-mon[32344]: pgmap v559: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2024-04-25T19:20:43.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:42 smithi194 ceph-mon[32740]: pgmap v559: 97 pgs: 97 active+clean; 1.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2024-04-25T19:20:45.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:44 smithi086 ceph-mon[27783]: pgmap v560: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 17 op/s 2024-04-25T19:20:45.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:44 smithi183 ceph-mon[32344]: pgmap v560: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 17 op/s 2024-04-25T19:20:45.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:44 smithi194 ceph-mon[32740]: pgmap v560: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 17 op/s 2024-04-25T19:20:47.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:46 smithi086 ceph-mon[27783]: pgmap v561: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-04-25T19:20:47.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:46 smithi183 ceph-mon[32344]: pgmap v561: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-04-25T19:20:47.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:46 smithi194 ceph-mon[32740]: pgmap v561: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 -- 172.21.15.86:0/3953443686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298112df0 msgr2=0x7fb2981151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 --2- 172.21.15.86:0/3953443686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298112df0 0x7fb2981151e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ca0 tx=0x7fb29002a8c0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 -- 172.21.15.86:0/3953443686 shutdown_connections 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 --2- 172.21.15.86:0/3953443686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298112df0 0x7fb2981151e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 --2- 172.21.15.86:0/3953443686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076da0 0x7fb2981128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 --2- 172.21.15.86:0/3953443686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298076480 0x7fb298076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 -- 172.21.15.86:0/3953443686 >> 172.21.15.86:0/3953443686 conn(0x7fb2980ffc80 msgr2=0x7fb2981020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 -- 172.21.15.86:0/3953443686 shutdown_connections 2024-04-25T19:20:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.014+0000 7fb29e200640 1 -- 172.21.15.86:0/3953443686 wait complete. 2024-04-25T19:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 Processor -- start 2024-04-25T19:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 -- start start 2024-04-25T19:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076480 0x7fb2981a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 0x7fb2981a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298112df0 0x7fb29819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb298118830 con 0x7fb298076da0 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981186b0 con 0x7fb298076480 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.015+0000 7fb29e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981189b0 con 0x7fb298112df0 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 0x7fb2981a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 0x7fb2981a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40180/0 (socket says 172.21.15.86:40180) 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 -- 172.21.15.86:0/1826891234 learned_addr learned my addr 172.21.15.86:0/1826891234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb297e00640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298112df0 0x7fb29819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb297400640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076480 0x7fb2981a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 -- 172.21.15.86:0/1826891234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298112df0 msgr2=0x7fb29819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298112df0 0x7fb29819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 -- 172.21.15.86:0/1826891234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076480 msgr2=0x7fb2981a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076480 0x7fb2981a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 -- 172.21.15.86:0/1826891234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29819f3d0 con 0x7fb298076da0 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb297e00640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298112df0 0x7fb29819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb297400640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076480 0x7fb2981a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.016+0000 7fb296a00640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 0x7fb2981a4f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb28400ea60 tx=0x7fb28400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.017+0000 7fb28fe00640 1 -- 172.21.15.86:0/1826891234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28400cde0 con 0x7fb298076da0 2024-04-25T19:20:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.017+0000 7fb28fe00640 1 -- 172.21.15.86:0/1826891234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840035f0 con 0x7fb298076da0 2024-04-25T19:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.017+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29819f6c0 con 0x7fb298076da0 2024-04-25T19:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.017+0000 7fb28fe00640 1 -- 172.21.15.86:0/1826891234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284010690 con 0x7fb298076da0 2024-04-25T19:20:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.017+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981ab9d0 con 0x7fb298076da0 2024-04-25T19:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.020+0000 7fb2a0fc3740 1 Processor -- start 2024-04-25T19:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.020+0000 7fb2a0fc3740 1 -- start start 2024-04-25T19:20:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b502c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb28fe00640 1 -- 172.21.15.86:0/1826891234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2840037a0 con 0x7fb298076da0 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55704b468900 con 0x55704b502820 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55704b40eef0 con 0x55704b503140 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55704b4648c0 con 0x55704b50da00 2024-04-25T19:20:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b502c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb296a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b502c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40188/0 (socket says 172.21.15.86:40188) 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 -- 172.21.15.86:0/1395819913 learned_addr learned my addr 172.21.15.86:0/1395819913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 -- 172.21.15.86:0/1395819913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 msgr2=0x55704b50fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 -- 172.21.15.86:0/1395819913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 msgr2=0x55704b50d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.021+0000 7fb297e00640 1 -- 172.21.15.86:0/1395819913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704b17b210 con 0x55704b502820 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.022+0000 7fb297e00640 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b502c00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ca0 tx=0x7fb290009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed2a7adb4b3ea29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.022+0000 7fb283e00640 1 -- 172.21.15.86:0/1395819913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290031ce0 con 0x55704b502820 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.022+0000 7fb283e00640 1 -- 172.21.15.86:0/1395819913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb290036040 con 0x55704b502820 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.022+0000 7fb283e00640 1 -- 172.21.15.86:0/1395819913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900042a0 con 0x55704b502820 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.022+0000 7fb28fe00640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb27c0a3e40 0x7fb27c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb28fe00640 1 -- 172.21.15.86:0/1826891234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fb2840d0bd0 con 0x7fb298076da0 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb297400640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb27c0a3e40 0x7fb27c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1395819913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 msgr2=0x55704b502c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b502c00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ca0 tx=0x7fb290009800 comp rx=0 tx=0).stop 2024-04-25T19:20:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1395819913 shutdown_connections 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1395819913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b502c00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1395819913 >> 172.21.15.86:0/1395819913 conn(0x55704b4fed30 msgr2=0x55704b4ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1395819913 shutdown_connections 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1395819913 wait complete. 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.023+0000 7fb297400640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb27c0a3e40 0x7fb27c0a6300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55704b503dc0 tx=0x7fb288005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 -- start start 2024-04-25T19:20:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 Processor -- start 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b50a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55704b5105d0 con 0x55704b502820 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55704b422310 con 0x55704b503140 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55704b510750 con 0x55704b50da00 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.024+0000 7fb2a0fc3740 10 client.0 _ll_register_callbacks cb 0x55704b3f0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb297400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b50a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb297400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b50a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40204/0 (socket says 172.21.15.86:40204) 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb297400640 1 -- 172.21.15.86:0/1191802563 learned_addr learned my addr 172.21.15.86:0/1191802563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb297e00640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 -- 172.21.15.86:0/1191802563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 msgr2=0x55704b50b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 -- 172.21.15.86:0/1191802563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 msgr2=0x55704b50a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b50a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 -- 172.21.15.86:0/1191802563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704b247620 con 0x55704b503140 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb297400640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b50a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb296a00640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50aea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2840ce250 tx=0x7fb284098620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668ca58c1d602fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.025+0000 7fb280c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb280c00640 1 -- 172.21.15.86:0/1191802563 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28409c5e0 con 0x55704b503140 2024-04-25T19:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb290036040 con 0x55704b503140 2024-04-25T19:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb280c00640 1 -- 172.21.15.86:0/1191802563 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fb284099110 con 0x55704b503140 2024-04-25T19:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb280c00640 1 -- 172.21.15.86:0/1191802563 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb28409c400 con 0x55704b503140 2024-04-25T19:20:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 10 client.27398 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55704b51c8e0 con 0x55704b503140 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb280c00640 1 -- 172.21.15.86:0/1191802563 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb290036040 con 0x55704b503140 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb280c00640 1 -- 172.21.15.86:0/1191802563 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb2840ce4d0 con 0x55704b503140 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 10 client.27398 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 10 client.27398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 20 client.27398 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b51dd50 0x55704b520180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 4 client.27398 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.026+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55704b47c8b0 con 0x55704b51dd50 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.027+0000 7fb297400640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b51dd50 0x55704b520180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.027+0000 7fb297400640 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b51dd50 0x55704b520180 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7db5189c05a53f server_cookie=e72813ecd3cb871b in_seq=0 out_seq=0 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.027+0000 7fb280c00640 10 client.27398 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.028+0000 7fb280c00640 1 -- 172.21.15.86:0/1191802563 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55704b47c8b0 con 0x55704b51dd50 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.028+0000 7fb280c00640 10 client.27398 handle_command_reply: tid=0 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.028+0000 7fb2a0fc3740 1 client.27398 shutdown 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 msgr2=0x55704b50aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50aea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2840ce250 tx=0x7fb284098620 comp rx=0 tx=0).stop 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 shutdown_connections 2024-04-25T19:20:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b51dd50 0x55704b520180 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b50b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb280c00640 0 client.27398 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b503140 0x55704b50aea0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/1191802563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b502820 0x55704b50a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 >> 172.21.15.86:0/1191802563 conn(0x55704b4fed30 msgr2=0x55704b504ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 shutdown_connections 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/1191802563 wait complete. 2024-04-25T19:20:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.029+0000 7fb2a0fc3740 20 client.27398 trim_cache size 0 max 16384 2024-04-25T19:20:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.045+0000 7fb2a0fc3740 1 Processor -- start 2024-04-25T19:20:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.045+0000 7fb2a0fc3740 1 -- start start 2024-04-25T19:20:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b505230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b503140 0x55704b50b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b5259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b4410 con 0x55704b503140 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b5f50 con 0x55704b502820 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b4ae0 con 0x55704b50da00 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb297400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b505230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb296a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b503140 0x55704b50b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb297400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b505230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:43648/0 (socket says 172.21.15.86:43648) 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb297400640 1 -- 172.21.15.86:0/2356370980 learned_addr learned my addr 172.21.15.86:0/2356370980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb297e00640 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b5259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb296a00640 1 -- 172.21.15.86:0/2356370980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 msgr2=0x55704b5259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb296a00640 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b5259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb296a00640 1 -- 172.21.15.86:0/2356370980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 msgr2=0x55704b505230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb296a00640 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b505230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb296a00640 1 -- 172.21.15.86:0/2356370980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704b502f70 con 0x55704b503140 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.046+0000 7fb297400640 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b505230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.047+0000 7fb296a00640 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b503140 0x55704b50b030 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55704b50bf30 tx=0x7fb2840a54f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a7f2658198c43c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.047+0000 7fb283400640 1 -- 172.21.15.86:0/2356370980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28409c5e0 con 0x55704b503140 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.047+0000 7fb283400640 1 -- 172.21.15.86:0/2356370980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28409d1a0 con 0x55704b503140 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.047+0000 7fb283400640 1 -- 172.21.15.86:0/2356370980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284098d40 con 0x55704b503140 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2356370980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b503140 msgr2=0x55704b50b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b503140 0x55704b50b030 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55704b50bf30 tx=0x7fb2840a54f0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2356370980 shutdown_connections 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b50da00 0x55704b5259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b503140 0x55704b50b030 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2356370980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b505230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2356370980 >> 172.21.15.86:0/2356370980 conn(0x55704b4fed30 msgr2=0x55704b50a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2356370980 shutdown_connections 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2356370980 wait complete. 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 -- start start 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.048+0000 7fb2a0fc3740 1 Processor -- start 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b513610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b503140 0x55704b513b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 0x55704b514090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b5880 con 0x55704b50da00 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b51b0 con 0x55704b502820 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981b2410 con 0x55704b503140 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb2a0fc3740 10 client.0 _ll_register_callbacks cb 0x7fb284004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb297e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 0x55704b514090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb297e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 0x55704b514090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40220/0 (socket says 172.21.15.86:40220) 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb297e00640 1 -- 172.21.15.86:0/2191065248 learned_addr learned my addr 172.21.15.86:0/2191065248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb297400640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b513610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb296a00640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b503140 0x55704b513b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.049+0000 7fb297e00640 1 -- 172.21.15.86:0/2191065248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b503140 msgr2=0x55704b513b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb297e00640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b503140 0x55704b513b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb297e00640 1 -- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 msgr2=0x55704b513610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb297e00640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b513610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb297e00640 1 -- 172.21.15.86:0/2191065248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704b583480 con 0x55704b50da00 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb297400640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b513610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb296a00640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b503140 0x55704b513b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb297e00640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 0x55704b514090 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ca0 tx=0x7fb29002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf2d33dc04a39ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb280c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb280c00640 1 -- 172.21.15.86:0/2191065248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290053020 con 0x55704b50da00 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb280c00640 1 -- 172.21.15.86:0/2191065248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fb290031ae0 con 0x55704b50da00 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28409d1a0 con 0x55704b50da00 2024-04-25T19:20:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb280c00640 1 -- 172.21.15.86:0/2191065248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb290031ce0 con 0x55704b50da00 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb2a0fc3740 10 client.17313 fetch_fsmap learned FSMap version 29 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.050+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55704b584770 con 0x55704b50da00 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb280c00640 1 -- 172.21.15.86:0/2191065248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28409d1a0 con 0x55704b50da00 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb280c00640 1 -- 172.21.15.86:0/2191065248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb290004040 con 0x55704b50da00 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb2a0fc3740 10 client.17313 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb2a0fc3740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb2a0fc3740 20 client.17313 populate_metadata read hostname 'smithi086' 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b584ac0 0x55704b586eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb2a0fc3740 4 client.17313 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55704b476320 con 0x55704b584ac0 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.051+0000 7fb297400640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b584ac0 0x55704b586eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.052+0000 7fb297400640 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b584ac0 0x55704b586eb0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11eadfc55e67139f server_cookie=8f0742868f266ca2 in_seq=0 out_seq=0 2024-04-25T19:20:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.052+0000 7fb280c00640 10 client.17313 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.062+0000 7fb280c00640 1 -- 172.21.15.86:0/2191065248 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55704b476320 con 0x55704b584ac0 2024-04-25T19:20:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.062+0000 7fb280c00640 10 client.17313 handle_command_reply: tid=0 2024-04-25T19:20:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 client.17313 shutdown 2024-04-25T19:20:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 msgr2=0x55704b514090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 0x55704b514090 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ca0 tx=0x7fb29002f310 comp rx=0 tx=0).stop 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 shutdown_connections 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55704b584ac0 0x55704b586eb0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704b50da00 0x55704b514090 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55704b503140 0x55704b513b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 --2- 172.21.15.86:0/2191065248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55704b502820 0x55704b513610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 >> 172.21.15.86:0/2191065248 conn(0x55704b4fed30 msgr2=0x55704b50b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb280c00640 0 client.17313 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 shutdown_connections 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 1 -- 172.21.15.86:0/2191065248 wait complete. 2024-04-25T19:20:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.063+0000 7fb2a0fc3740 20 client.17313 trim_cache size 0 max 16384 2024-04-25T19:20:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:20:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:20:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:20:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:20:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb27c0a3e40 msgr2=0x7fb27c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb27c0a3e40 0x7fb27c0a6300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55704b503dc0 tx=0x7fb288005f90 comp rx=0 tx=0).stop 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 msgr2=0x7fb2981a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 0x7fb2981a4f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb28400ea60 tx=0x7fb28400ef20 comp rx=0 tx=0).stop 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 shutdown_connections 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb27c0a3e40 0x7fb27c0a6300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb298112df0 0x7fb29819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb298076da0 0x7fb2981a4f90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 --2- 172.21.15.86:0/1826891234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb298076480 0x7fb2981a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 >> 172.21.15.86:0/1826891234 conn(0x7fb2980ffc80 msgr2=0x7fb298113a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 shutdown_connections 2024-04-25T19:20:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:20:49.064+0000 7fb29e200640 1 -- 172.21.15.86:0/1826891234 wait complete. 2024-04-25T19:20:49.069 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:48 smithi086 ceph-mon[27783]: pgmap v562: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-04-25T19:20:49.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:48 smithi183 ceph-mon[32344]: pgmap v562: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-04-25T19:20:49.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:48 smithi194 ceph-mon[32740]: pgmap v562: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-04-25T19:20:51.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:50 smithi086 ceph-mon[27783]: pgmap v563: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-25T19:20:51.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:50 smithi183 ceph-mon[32344]: pgmap v563: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-25T19:20:51.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:50 smithi194 ceph-mon[32740]: pgmap v563: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-25T19:20:53.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:52 smithi086 ceph-mon[27783]: pgmap v564: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T19:20:53.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:52 smithi183 ceph-mon[32344]: pgmap v564: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T19:20:53.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:52 smithi194 ceph-mon[32740]: pgmap v564: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T19:20:55.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:54 smithi086 ceph-mon[27783]: pgmap v565: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-25T19:20:55.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:54 smithi183 ceph-mon[32344]: pgmap v565: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-25T19:20:55.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:54 smithi194 ceph-mon[32740]: pgmap v565: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-04-25T19:20:55.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:20:56.200 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:20:57.033 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:56 smithi194 ceph-mon[32740]: pgmap v566: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:20:57.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:56 smithi086 ceph-mon[27783]: pgmap v566: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:20:57.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:56 smithi183 ceph-mon[32344]: pgmap v566: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2588390832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 msgr2=0x7f3d08111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2588390832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 0x7f3d08111b70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b0d0 tx=0x7f3d0000b3e0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2588390832 shutdown_connections 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2588390832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 0x7f3d08111b70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2588390832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08104ec0 0x7f3d081052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2588390832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08105870 0x7f3d081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2588390832 >> 172.21.15.86:0/2588390832 conn(0x7f3d08100750 msgr2=0x7f3d08102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2588390832 shutdown_connections 2024-04-25T19:20:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.554+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2588390832 wait complete. 2024-04-25T19:20:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.555+0000 7f3d0ca00700 1 Processor -- start 2024-04-25T19:20:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 -- start start 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08104ec0 0x7f3d0819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 0x7f3d0819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 0x7f3d0819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08114e40 con 0x7f3d08105870 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08114cc0 con 0x7f3d08104ec0 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d0ca00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08114fc0 con 0x7f3d08109fa0 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 0x7f3d0819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d05e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08104ec0 0x7f3d0819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 0x7f3d0819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37944/0 (socket says 172.21.15.86:37944) 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d05400700 1 -- 172.21.15.86:0/2247122999 learned_addr learned my addr 172.21.15.86:0/2247122999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:20:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.556+0000 7f3d06800700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 0x7f3d0819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05e00700 1 -- 172.21.15.86:0/2247122999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 msgr2=0x7f3d0819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05e00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 0x7f3d0819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05e00700 1 -- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 msgr2=0x7f3d0819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05e00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 0x7f3d0819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05e00700 1 -- 172.21.15.86:0/2247122999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d081a0320 con 0x7f3d08104ec0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05400700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 0x7f3d0819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.557+0000 7f3d05e00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08104ec0 0x7f3d0819b310 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e9f0 tx=0x7f3cf800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.558+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800c9c0 con 0x7f3d08104ec0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.558+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf800cb60 con 0x7f3d08104ec0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.558+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d081a0610 con 0x7f3d08104ec0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.558+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8018680 con 0x7f3d08104ec0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.558+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a7e90 con 0x7f3d08104ec0 2024-04-25T19:20:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.559+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d08069730 con 0x7f3d08104ec0 2024-04-25T19:20:58.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.561+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3cf8010430 con 0x7f3d08104ec0 2024-04-25T19:20:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.562+0000 7f3cfea00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3cec0a3e30 0x7f3cec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.563+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7f3cf80d0d40 con 0x7f3d08104ec0 2024-04-25T19:20:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.563+0000 7f3d05400700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3cec0a3e30 0x7f3cec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.563+0000 7f3d05400700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3cec0a3e30 0x7f3cec0a62f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009680 tx=0x7f3cf4009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.565+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cf809d310 con 0x7f3d08104ec0 2024-04-25T19:20:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.841+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d0802df80 con 0x7f3d08104ec0 2024-04-25T19:20:58.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.842+0000 7f3cfea00700 1 -- 172.21.15.86:0/2247122999 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f3cf809cc10 con 0x7f3d08104ec0 2024-04-25T19:20:58.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:20:58.845 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3cec0a3e30 msgr2=0x7f3cec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3cec0a3e30 0x7f3cec0a62f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009680 tx=0x7f3cf4009260 comp rx=0 tx=0).stop 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08104ec0 msgr2=0x7f3d0819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08104ec0 0x7f3d0819b310 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e9f0 tx=0x7f3cf800ed00 comp rx=0 tx=0).stop 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 shutdown_connections 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3cec0a3e30 0x7f3cec0a62f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d08109fa0 0x7f3d0819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d08104ec0 0x7f3d0819b310 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 --2- 172.21.15.86:0/2247122999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d08105870 0x7f3d0819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.845+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 >> 172.21.15.86:0/2247122999 conn(0x7f3d08100750 msgr2=0x7f3d081012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.846+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 shutdown_connections 2024-04-25T19:20:58.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:20:58.846+0000 7f3d0ca00700 1 -- 172.21.15.86:0/2247122999 wait complete. 2024-04-25T19:20:58.849 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:20:59.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:58 smithi086 ceph-mon[27783]: pgmap v567: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:20:59.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:58 smithi183 ceph-mon[32344]: pgmap v567: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:20:59.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:58 smithi194 ceph-mon[32740]: pgmap v567: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:20:59.631 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --set-id d843219e --release --await' 2024-04-25T19:20:59.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --set-id d843219e --release --await 2024-04-25T19:20:59.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:20:59 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2247122999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:21:00.082 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:00.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:20:59 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2247122999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:21:00.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:20:59 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2247122999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:21:00.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:00 smithi086 ceph-mon[27783]: pgmap v568: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-25T19:21:01.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:00 smithi183 ceph-mon[32344]: pgmap v568: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-25T19:21:01.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:00 smithi194 ceph-mon[32740]: pgmap v568: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.204+0000 7f1e27e00640 1 -- 172.21.15.86:0/1732740608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28103890 msgr2=0x7f1e2810bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.204+0000 7f1e27e00640 1 --2- 172.21.15.86:0/1732740608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28103890 0x7f1e2810bd70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e1002a490 comp rx=0 tx=0).stop 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 -- 172.21.15.86:0/1732740608 shutdown_connections 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 --2- 172.21.15.86:0/1732740608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 0x7f1e2810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 --2- 172.21.15.86:0/1732740608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28103890 0x7f1e2810bd70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 --2- 172.21.15.86:0/1732740608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28102f70 0x7f1e28103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 -- 172.21.15.86:0/1732740608 >> 172.21.15.86:0/1732740608 conn(0x7f1e280fc770 msgr2=0x7f1e280feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 -- 172.21.15.86:0/1732740608 shutdown_connections 2024-04-25T19:21:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.205+0000 7f1e27e00640 1 -- 172.21.15.86:0/1732740608 wait complete. 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 Processor -- start 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 -- start start 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 0x7f1e28198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28103890 0x7f1e28198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 0x7f1e2819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28111fc0 con 0x7f1e28102f70 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28111e40 con 0x7f1e28103890 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e27e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28112140 con 0x7f1e2810c340 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 0x7f1e28198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.206+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 0x7f1e28198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40142/0 (socket says 172.21.15.86:40142) 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 -- 172.21.15.86:0/3886479967 learned_addr learned my addr 172.21.15.86:0/3886479967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28103890 0x7f1e28198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e27400640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 0x7f1e2819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 -- 172.21.15.86:0/3886479967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 msgr2=0x7f1e2819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 0x7f1e2819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 -- 172.21.15.86:0/3886479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28103890 msgr2=0x7f1e28198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28103890 0x7f1e28198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 -- 172.21.15.86:0/3886479967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2819d550 con 0x7f1e28102f70 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e27400640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 0x7f1e2819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28103890 0x7f1e28198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e26a00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 0x7f1e28198560 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800e9f0 tx=0x7f1e1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e2ca00640 1 -- 172.21.15.86:0/3886479967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800cd80 con 0x7f1e28102f70 2024-04-25T19:21:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.207+0000 7f1e2ca00640 1 -- 172.21.15.86:0/3886479967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e180108e0 con 0x7f1e28102f70 2024-04-25T19:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.208+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2819d840 con 0x7f1e28102f70 2024-04-25T19:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.208+0000 7f1e2ca00640 1 -- 172.21.15.86:0/3886479967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18018980 con 0x7f1e28102f70 2024-04-25T19:21:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.208+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e281a50d0 con 0x7f1e28102f70 2024-04-25T19:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.211+0000 7f1e2ca00640 1 -- 172.21.15.86:0/3886479967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e18010430 con 0x7f1e28102f70 2024-04-25T19:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.211+0000 7f1e30c1e740 1 Processor -- start 2024-04-25T19:21:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.211+0000 7f1e30c1e740 1 -- start start 2024-04-25T19:21:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e197d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed8e0fd900 con 0x55ed8e1982d0 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed8e0a3ef0 con 0x55ed8e1979b0 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ed8e0f98c0 con 0x55ed8e1a2b90 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40144/0 (socket says 172.21.15.86:40144) 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e26a00640 1 -- 172.21.15.86:0/2563065703 learned_addr learned my addr 172.21.15.86:0/2563065703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e27400640 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.212+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e26a00640 1 -- 172.21.15.86:0/2563065703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 msgr2=0x55ed8e1a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e26a00640 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e26a00640 1 -- 172.21.15.86:0/2563065703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 msgr2=0x55ed8e197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e26a00640 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e26a00640 1 -- 172.21.15.86:0/2563065703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8de10210 con 0x55ed8e1982d0 2024-04-25T19:21:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e26a00640 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a2650 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e180ce2b0 tx=0x7f1e180d97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7187ae0dbf4b36bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.213+0000 7f1e17e00640 1 -- 172.21.15.86:0/2563065703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180d7870 con 0x55ed8e1982d0 2024-04-25T19:21:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.214+0000 7f1e17e00640 1 -- 172.21.15.86:0/2563065703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e180de070 con 0x55ed8e1982d0 2024-04-25T19:21:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.214+0000 7f1e17e00640 1 -- 172.21.15.86:0/2563065703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180e3d80 con 0x55ed8e1982d0 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.214+0000 7f1e2ca00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e0c0a3d10 0x7f1e0c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.214+0000 7f1e2ca00640 1 -- 172.21.15.86:0/3886479967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7f1e180d0cf0 con 0x7f1e28102f70 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.214+0000 7f1e27400640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e0c0a3d10 0x7f1e0c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 -- 172.21.15.86:0/2563065703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 msgr2=0x55ed8e1a2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a2650 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e180ce2b0 tx=0x7f1e180d97b0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 -- 172.21.15.86:0/2563065703 shutdown_connections 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a2650 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/2563065703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 -- 172.21.15.86:0/2563065703 >> 172.21.15.86:0/2563065703 conn(0x55ed8e193ec0 msgr2=0x55ed8e1942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e27400640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e0c0a3d10 0x7f1e0c0a61d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005fd0 tx=0x7f1e20009500 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 -- 172.21.15.86:0/2563065703 shutdown_connections 2024-04-25T19:21:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 -- 172.21.15.86:0/2563065703 wait complete. 2024-04-25T19:21:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.215+0000 7f1e30c1e740 1 -- start start 2024-04-25T19:21:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 Processor -- start 2024-04-25T19:21:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed8e1a57c0 con 0x55ed8e1982d0 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed8e0b7310 con 0x55ed8e1979b0 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ed8e1a5940 con 0x55ed8e1a2b90 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e30c1e740 10 client.0 _ll_register_callbacks cb 0x55ed8e085cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.216+0000 7f1e1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40154/0 (socket says 172.21.15.86:40154) 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 -- 172.21.15.86:0/1645619130 learned_addr learned my addr 172.21.15.86:0/1645619130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e26a00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e27400640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 -- 172.21.15.86:0/1645619130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 msgr2=0x55ed8e1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 -- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 msgr2=0x55ed8e19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 -- 172.21.15.86:0/1645619130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8dedc620 con 0x55ed8e1982d0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e26a00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a0050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e10031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93388da27ed57b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.217+0000 7f1e14c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e14c00640 1 -- 172.21.15.86:0/1645619130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10035040 con 0x55ed8e1982d0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e14c00640 1 -- 172.21.15.86:0/1645619130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7f1e10004240 con 0x55ed8e1982d0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e180de070 con 0x55ed8e1982d0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e14c00640 1 -- 172.21.15.86:0/1645619130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e10004440 con 0x55ed8e1982d0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e30c1e740 10 client.17331 fetch_fsmap learned FSMap version 29 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed8e1b1a00 con 0x55ed8e1982d0 2024-04-25T19:21:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e14c00640 1 -- 172.21.15.86:0/1645619130 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e180de070 con 0x55ed8e1982d0 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.218+0000 7f1e14c00640 1 -- 172.21.15.86:0/1645619130 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1e10004aa0 con 0x55ed8e1982d0 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e30c1e740 10 client.17331 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e30c1e740 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e30c1e740 20 client.17331 populate_metadata read hostname 'smithi086' 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e1b2ec0 0x55ed8e1b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e30c1e740 4 client.17331 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8e1118b0 con 0x55ed8e1b2ec0 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e26a00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e1b2ec0 0x55ed8e1b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e26a00640 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e1b2ec0 0x55ed8e1b52d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40929300ab99087a server_cookie=9a94ce62d4a2bbb9 in_seq=0 out_seq=0 2024-04-25T19:21:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.219+0000 7f1e14c00640 10 client.17331 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.220+0000 7f1e14c00640 1 -- 172.21.15.86:0/1645619130 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed8e1118b0 con 0x55ed8e1b2ec0 2024-04-25T19:21:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.220+0000 7f1e14c00640 10 client.17331 handle_command_reply: tid=0 2024-04-25T19:21:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.220+0000 7f1e30c1e740 1 client.17331 shutdown 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 msgr2=0x55ed8e1a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a0050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e10031750 comp rx=0 tx=0).stop 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 shutdown_connections 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e1b2ec0 0x55ed8e1b52d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a0050 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/1645619130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e14c00640 0 client.17331 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 >> 172.21.15.86:0/1645619130 conn(0x55ed8e193ec0 msgr2=0x55ed8e199c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 shutdown_connections 2024-04-25T19:21:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 1 -- 172.21.15.86:0/1645619130 wait complete. 2024-04-25T19:21:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.221+0000 7f1e30c1e740 20 client.17331 trim_cache size 0 max 16384 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.237+0000 7f1e30c1e740 1 Processor -- start 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.237+0000 7f1e30c1e740 1 -- start start 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e19a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281adc60 con 0x55ed8e1a2b90 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281af7a0 con 0x55ed8e1979b0 2024-04-25T19:21:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281ae330 con 0x55ed8e1982d0 2024-04-25T19:21:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e1fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e1fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56844/0 (socket says 172.21.15.86:56844) 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.238+0000 7f1e1fe00640 1 -- 172.21.15.86:0/3532015791 learned_addr learned my addr 172.21.15.86:0/3532015791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e27400640 1 -- 172.21.15.86:0/3532015791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 msgr2=0x55ed8e19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e27400640 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e27400640 1 -- 172.21.15.86:0/3532015791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 msgr2=0x55ed8e1ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e27400640 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e27400640 1 -- 172.21.15.86:0/3532015791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8e19e060 con 0x55ed8e1a2b90 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e27400640 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a1a50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009fd0 tx=0x7f1e20019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d08afc2569d81e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e17400640 1 -- 172.21.15.86:0/3532015791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2001e970 con 0x55ed8e1a2b90 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.239+0000 7f1e17400640 1 -- 172.21.15.86:0/3532015791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e20029480 con 0x55ed8e1a2b90 2024-04-25T19:21:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e17400640 1 -- 172.21.15.86:0/3532015791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2001c650 con 0x55ed8e1a2b90 2024-04-25T19:21:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 -- 172.21.15.86:0/3532015791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 msgr2=0x55ed8e1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a1a50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009fd0 tx=0x7f1e20019000 comp rx=0 tx=0).stop 2024-04-25T19:21:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 -- 172.21.15.86:0/3532015791 shutdown_connections 2024-04-25T19:21:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a1a50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/3532015791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 -- 172.21.15.86:0/3532015791 >> 172.21.15.86:0/3532015791 conn(0x55ed8e193ec0 msgr2=0x55ed8e1a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 -- 172.21.15.86:0/3532015791 shutdown_connections 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.240+0000 7f1e30c1e740 1 -- 172.21.15.86:0/3532015791 wait complete. 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.241+0000 7f1e30c1e740 1 -- start start 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.241+0000 7f1e30c1e740 1 Processor -- start 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.241+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.241+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.241+0000 7f1e30c1e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281af0d0 con 0x55ed8e1a2b90 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281aea00 con 0x55ed8e1979b0 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e30c1e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281abc60 con 0x55ed8e1982d0 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e30c1e740 10 client.0 _ll_register_callbacks cb 0x7f1e10032ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:21:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40182/0 (socket says 172.21.15.86:40182) 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e27400640 1 -- 172.21.15.86:0/382307141 learned_addr learned my addr 172.21.15.86:0/382307141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59644/0 (socket says 172.21.15.86:59644) 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 -- 172.21.15.86:0/382307141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 msgr2=0x55ed8e1a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 -- 172.21.15.86:0/382307141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 msgr2=0x55ed8e1a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e26a00640 1 -- 172.21.15.86:0/382307141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8e1a8d80 con 0x55ed8e1979b0 2024-04-25T19:21:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.242+0000 7f1e27400640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e26a00640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1aae10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1e1809c8c0 tx=0x7f1e18000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cb47ee3e922c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e14c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e14c00640 1 -- 172.21.15.86:0/382307141 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18098e60 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e20029480 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e14c00640 1 -- 172.21.15.86:0/382307141 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7f1e18010a90 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e14c00640 1 -- 172.21.15.86:0/382307141 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e1809c520 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e30c1e740 10 client.27428 fetch_fsmap learned FSMap version 29 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed8e21d730 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.243+0000 7f1e14c00640 1 -- 172.21.15.86:0/382307141 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e20029480 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e14c00640 1 -- 172.21.15.86:0/382307141 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1e180bd0b0 con 0x55ed8e1979b0 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e30c1e740 10 client.27428 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:21:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e30c1e740 10 client.27428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e30c1e740 20 client.27428 populate_metadata read hostname 'smithi086' 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e21da20 0x55ed8e21fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e30c1e740 4 client.27428 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed8e10b320 con 0x55ed8e21da20 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.244+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e21da20 0x55ed8e21fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.245+0000 7f1e1fe00640 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e21da20 0x55ed8e21fe10 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6623f509ecfdac0 server_cookie=32e2bf2bbd5f7e37 in_seq=0 out_seq=0 2024-04-25T19:21:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.245+0000 7f1e14c00640 10 client.27428 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.253+0000 7f1e14c00640 1 -- 172.21.15.86:0/382307141 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed8e10b320 con 0x55ed8e21da20 2024-04-25T19:21:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.253+0000 7f1e14c00640 10 client.27428 handle_command_reply: tid=0 2024-04-25T19:21:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.253+0000 7f1e30c1e740 1 client.27428 shutdown 2024-04-25T19:21:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 msgr2=0x55ed8e1aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1aae10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1e1809c8c0 tx=0x7f1e18000bc0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 shutdown_connections 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55ed8e21da20 0x55ed8e21fe10 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8e1a2b90 0x55ed8e1a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed8e1982d0 0x55ed8e1a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 --2- 172.21.15.86:0/382307141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8e1979b0 0x55ed8e1aae10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e14c00640 0 client.27428 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 >> 172.21.15.86:0/382307141 conn(0x55ed8e193ec0 msgr2=0x55ed8e1a0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 shutdown_connections 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 1 -- 172.21.15.86:0/382307141 wait complete. 2024-04-25T19:21:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.254+0000 7f1e30c1e740 20 client.27428 trim_cache size 0 max 16384 2024-04-25T19:21:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-25T19:21:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-25T19:21:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-25T19:21:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-25T19:21:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e0c0a3d10 msgr2=0x7f1e0c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e0c0a3d10 0x7f1e0c0a61d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005fd0 tx=0x7f1e20009500 comp rx=0 tx=0).stop 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 msgr2=0x7f1e28198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 0x7f1e28198560 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800e9f0 tx=0x7f1e1800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 shutdown_connections 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f1e0c0a3d10 0x7f1e0c0a61d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e2810c340 0x7f1e2819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e28103890 0x7f1e28198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 --2- 172.21.15.86:0/3886479967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28102f70 0x7f1e28198560 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 >> 172.21.15.86:0/3886479967 conn(0x7f1e280fc770 msgr2=0x7f1e280fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 shutdown_connections 2024-04-25T19:21:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-25T19:21:02.255+0000 7f1e27e00640 1 -- 172.21.15.86:0/3886479967 wait complete. 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 -- 172.21.15.86:0/3509867609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 msgr2=0x7fe09810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 --2- 172.21.15.86:0/3509867609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 0x7fe09810dcd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe0840099c0 tx=0x7fe084009cd0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 -- 172.21.15.86:0/3509867609 shutdown_connections 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 --2- 172.21.15.86:0/3509867609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09810e3b0 0x7fe098115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 --2- 172.21.15.86:0/3509867609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0981092d0 0x7fe0981096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 --2- 172.21.15.86:0/3509867609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 0x7fe09810dcd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 -- 172.21.15.86:0/3509867609 >> 172.21.15.86:0/3509867609 conn(0x7fe098076210 msgr2=0x7fe098107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 -- 172.21.15.86:0/3509867609 shutdown_connections 2024-04-25T19:21:02.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.300+0000 7fe09f800700 1 -- 172.21.15.86:0/3509867609 wait complete. 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.301+0000 7fe09f800700 1 Processor -- start 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 -- start start 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 0x7fe0981a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 0x7fe0981a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe09810e3b0 0x7fe09819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe098119270 con 0x7fe098109c80 2024-04-25T19:21:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981190f0 con 0x7fe0981092d0 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09f800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981193f0 con 0x7fe09810e3b0 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 0x7fe0981a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe097e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 0x7fe0981a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 0x7fe0981a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59662/0 (socket says 172.21.15.86:59662) 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.302+0000 7fe09ce00700 1 -- 172.21.15.86:0/2777855867 learned_addr learned my addr 172.21.15.86:0/2777855867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09d800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe09810e3b0 0x7fe09819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09ce00700 1 -- 172.21.15.86:0/2777855867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe09810e3b0 msgr2=0x7fe09819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09ce00700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe09810e3b0 0x7fe09819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09ce00700 1 -- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 msgr2=0x7fe0981a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09ce00700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 0x7fe0981a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09ce00700 1 -- 172.21.15.86:0/2777855867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09819fd70 con 0x7fe0981092d0 2024-04-25T19:21:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe09ce00700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 0x7fe0981a5400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00d950 tx=0x7fe08c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.303+0000 7fe095600700 1 -- 172.21.15.86:0/2777855867 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c016070 con 0x7fe0981092d0 2024-04-25T19:21:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.304+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0981a0060 con 0x7fe0981092d0 2024-04-25T19:21:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.304+0000 7fe095600700 1 -- 172.21.15.86:0/2777855867 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe08c00fd40 con 0x7fe0981092d0 2024-04-25T19:21:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.304+0000 7fe095600700 1 -- 172.21.15.86:0/2777855867 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00e3b0 con 0x7fe0981092d0 2024-04-25T19:21:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.304+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981ac2d0 con 0x7fe0981092d0 2024-04-25T19:21:02.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.307+0000 7fe095600700 1 -- 172.21.15.86:0/2777855867 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe08c019070 con 0x7fe0981092d0 2024-04-25T19:21:02.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.308+0000 7fe0aa223b80 1 Processor -- start 2024-04-25T19:21:02.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.308+0000 7fe0aa223b80 1 -- start start 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe095600700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0800a3db0 0x7fe0800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 0x5649ecf22d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649ece75f90 con 0x5649ecf22920 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649ece1c830 con 0x5649ecf23240 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5649ece71f50 con 0x5649ecf2e0f0 2024-04-25T19:21:02.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe095600700 1 -- 172.21.15.86:0/2777855867 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fe08c0d0390 con 0x7fe0981092d0 2024-04-25T19:21:02.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe09d800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0800a3db0 0x7fe0800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 0x5649ecf22d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 0x5649ecf22d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40206/0 (socket says 172.21.15.86:40206) 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe09ce00700 1 -- 172.21.15.86:0/1529196369 learned_addr learned my addr 172.21.15.86:0/1529196369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe097e00700 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.309+0000 7fe09d800700 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09ce00700 1 -- 172.21.15.86:0/1529196369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 msgr2=0x5649ecf304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09ce00700 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09ce00700 1 -- 172.21.15.86:0/1529196369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 msgr2=0x5649ecf2dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09ce00700 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09ce00700 1 -- 172.21.15.86:0/1529196369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ece12780 con 0x5649ecf22920 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe097e00700 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09d800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0800a3db0 0x7fe0800a6270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe090004150 tx=0x7fe090003e20 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe09ce00700 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 0x5649ecf22d00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0a49a0 tx=0x7fe08c09bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a18cd86dc89691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe08a000700 1 -- 172.21.15.86:0/1529196369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c020040 con 0x5649ecf22920 2024-04-25T19:21:02.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe08a000700 1 -- 172.21.15.86:0/1529196369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe08c09c960 con 0x5649ecf22920 2024-04-25T19:21:02.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.310+0000 7fe08a000700 1 -- 172.21.15.86:0/1529196369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0a4ed0 con 0x5649ecf22920 2024-04-25T19:21:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 -- 172.21.15.86:0/1529196369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 msgr2=0x5649ecf22d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 0x5649ecf22d00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0a49a0 tx=0x7fe08c09bc40 comp rx=0 tx=0).stop 2024-04-25T19:21:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 -- 172.21.15.86:0/1529196369 shutdown_connections 2024-04-25T19:21:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf22920 0x5649ecf22d00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/1529196369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 -- 172.21.15.86:0/1529196369 >> 172.21.15.86:0/1529196369 conn(0x5649ece516a0 msgr2=0x5649ecf1edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 -- 172.21.15.86:0/1529196369 shutdown_connections 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.311+0000 7fe0aa223b80 1 -- 172.21.15.86:0/1529196369 wait complete. 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.312+0000 7fe0aa223b80 1 -- start start 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.312+0000 7fe0aa223b80 1 Processor -- start 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 0x5649ecf2d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf2b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649ecf30b50 con 0x5649ecf2e0f0 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649ece2f960 con 0x5649ecf23240 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5649ecf30cd0 con 0x5649ecf22920 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe0aa223b80 10 client.0 _ll_register_callbacks cb 0x5649ece0e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 0x5649ecf2d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 0x5649ecf2d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56888/0 (socket says 172.21.15.86:56888) 2024-04-25T19:21:02.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 -- 172.21.15.86:0/3835537606 learned_addr learned my addr 172.21.15.86:0/3835537606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe097e00700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09d800700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf2b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 -- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 msgr2=0x5649ecf2ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 -- 172.21.15.86:0/3835537606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 msgr2=0x5649ecf2b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf2b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09ce00700 1 -- 172.21.15.86:0/3835537606 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ecdc39d0 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.313+0000 7fe09d800700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf2b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe09ce00700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 0x5649ecf2d820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe08c012040 tx=0x7fe08c09a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f57226a06278c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe07ea00700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe07ea00700 1 -- 172.21.15.86:0/3835537606 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0ba060 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe07ea00700 1 -- 172.21.15.86:0/3835537606 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fe08c01c040 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08c09c960 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe07ea00700 1 -- 172.21.15.86:0/3835537606 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe08c0b4030 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe0aa223b80 10 client.27433 fetch_fsmap learned FSMap version 29 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649ece0cbf0 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.314+0000 7fe07ea00700 1 -- 172.21.15.86:0/3835537606 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe08c09c960 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe07ea00700 1 -- 172.21.15.86:0/3835537606 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe08c09c4c0 con 0x5649ecf22920 2024-04-25T19:21:02.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe0aa223b80 10 client.27433 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe0aa223b80 10 client.27433 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe0aa223b80 20 client.27433 populate_metadata read hostname 'smithi086' 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecf3d6b0 0x5649ecf3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe0aa223b80 4 client.27433 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649ece83780 con 0x5649ecf3d6b0 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.315+0000 7fe097e00700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecf3d6b0 0x5649ecf3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.316+0000 7fe07ea00700 10 client.27433 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.316+0000 7fe097e00700 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecf3d6b0 0x5649ecf3faa0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08fcaf49e6e8a79 server_cookie=ea445d643390ed57 in_seq=0 out_seq=0 2024-04-25T19:21:02.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe07ea00700 1 -- 172.21.15.86:0/3835537606 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649ece83780 con 0x5649ecf3d6b0 2024-04-25T19:21:02.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe07ea00700 10 client.27433 handle_command_reply: tid=0 2024-04-25T19:21:02.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 client.27433 shutdown 2024-04-25T19:21:02.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 msgr2=0x5649ecf2d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 0x5649ecf2d820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe08c012040 tx=0x7fe08c09a770 comp rx=0 tx=0).stop 2024-04-25T19:21:02.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 shutdown_connections 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecf3d6b0 0x5649ecf3faa0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf2b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf22920 0x5649ecf2d820 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/3835537606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf23240 0x5649ecf2ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe07ea00700 0 client.27433 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 >> 172.21.15.86:0/3835537606 conn(0x5649ece516a0 msgr2=0x5649ecf1e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 shutdown_connections 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.317+0000 7fe0aa223b80 1 -- 172.21.15.86:0/3835537606 wait complete. 2024-04-25T19:21:02.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.318+0000 7fe0aa223b80 20 client.27433 trim_cache size 0 max 16384 2024-04-25T19:21:02.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.338+0000 7fe0aa223b80 1 Processor -- start 2024-04-25T19:21:02.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.338+0000 7fe0aa223b80 1 -- start start 2024-04-25T19:21:02.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf42ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf23240 0x5649ecf43010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf27e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b3b90 con 0x5649ecf2e0f0 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b56d0 con 0x5649ecf22920 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b4260 con 0x5649ecf23240 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe09d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf27e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe097e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf23240 0x5649ecf43010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe09d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf27e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40238/0 (socket says 172.21.15.86:40238) 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf42ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.339+0000 7fe09d800700 1 -- 172.21.15.86:0/608938653 learned_addr learned my addr 172.21.15.86:0/608938653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe097e00700 1 -- 172.21.15.86:0/608938653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 msgr2=0x5649ecf42ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe097e00700 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf42ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe097e00700 1 -- 172.21.15.86:0/608938653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 msgr2=0x5649ecf27e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe097e00700 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf27e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe097e00700 1 -- 172.21.15.86:0/608938653 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ecf23870 con 0x5649ecf23240 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe097e00700 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf23240 0x5649ecf43010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe08400b580 tx=0x7fe0840058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9aa1809357ba17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe089600700 1 -- 172.21.15.86:0/608938653 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08400bd50 con 0x5649ecf23240 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe089600700 1 -- 172.21.15.86:0/608938653 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0840055c0 con 0x5649ecf23240 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.340+0000 7fe089600700 1 -- 172.21.15.86:0/608938653 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840291d0 con 0x5649ecf23240 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 -- 172.21.15.86:0/608938653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf23240 msgr2=0x5649ecf43010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf23240 0x5649ecf43010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe08400b580 tx=0x7fe0840058e0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 -- 172.21.15.86:0/608938653 shutdown_connections 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf2e0f0 0x5649ecf27e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf42ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/608938653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf23240 0x5649ecf43010 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 -- 172.21.15.86:0/608938653 >> 172.21.15.86:0/608938653 conn(0x5649ece516a0 msgr2=0x5649ecf397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 -- 172.21.15.86:0/608938653 shutdown_connections 2024-04-25T19:21:02.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 -- 172.21.15.86:0/608938653 wait complete. 2024-04-25T19:21:02.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.341+0000 7fe0aa223b80 1 -- start start 2024-04-25T19:21:02.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 Processor -- start 2024-04-25T19:21:02.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf23240 0x5649ecf9d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b5000 con 0x5649ecf23240 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b4930 con 0x5649ecf22920 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b1b50 con 0x5649ecf2e0f0 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.342+0000 7fe0aa223b80 10 client.0 _ll_register_callbacks cb 0x5649ece0e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe097e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf23240 0x5649ecf9d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf9d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59706/0 (socket says 172.21.15.86:59706) 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 -- 172.21.15.86:0/2540316122 learned_addr learned my addr 172.21.15.86:0/2540316122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09d800700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 -- 172.21.15.86:0/2540316122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 msgr2=0x5649ecf978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 -- 172.21.15.86:0/2540316122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf23240 msgr2=0x5649ecf9d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf23240 0x5649ecf9d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 -- 172.21.15.86:0/2540316122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ecf97fc0 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe097e00700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf23240 0x5649ecf9d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.343+0000 7fe09ce00700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf9d4a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe08c09c1f0 tx=0x7fe08c0141e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa3edf83e28fe3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe07ea00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe07ea00700 1 -- 172.21.15.86:0/2540316122 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0ba040 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe07ea00700 1 -- 172.21.15.86:0/2540316122 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7fe08c0a0120 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0840055c0 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe07ea00700 1 -- 172.21.15.86:0/2540316122 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe08c014040 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe0aa223b80 10 client.27446 fetch_fsmap learned FSMap version 29 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649ecfa4d10 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe07ea00700 1 -- 172.21.15.86:0/2540316122 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0840055c0 con 0x5649ecf22920 2024-04-25T19:21:02.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.344+0000 7fe07ea00700 1 -- 172.21.15.86:0/2540316122 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe08c09b6b0 con 0x5649ecf22920 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe0aa223b80 10 client.27446 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe0aa223b80 10 client.27446 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe0aa223b80 20 client.27446 populate_metadata read hostname 'smithi086' 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecfa5170 0x5649ecfa7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe0aa223b80 4 client.27446 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d843219e", "release": true, "await": true}] 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d843219e", "release": true, "await": true}) -- 0x5649ece7d820 con 0x5649ecfa5170 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe097e00700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecfa5170 0x5649ecfa7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe097e00700 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecfa5170 0x5649ecfa7560 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa846bd48d77a506 server_cookie=4a240091dab432da in_seq=0 out_seq=0 2024-04-25T19:21:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.345+0000 7fe07ea00700 10 client.27446 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.348+0000 7fe07ea00700 1 -- 172.21.15.86:0/2540316122 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5649ece7d820 con 0x5649ecfa5170 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.348+0000 7fe07ea00700 10 client.27446 handle_command_reply: tid=0 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.348+0000 7fe0aa223b80 1 client.27446 shutdown 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.348+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 msgr2=0x5649ecf9d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.348+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf9d4a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe08c09c1f0 tx=0x7fe08c0141e0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 shutdown_connections 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x5649ecfa5170 0x5649ecfa7560 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5649ecf2e0f0 0x5649ecf978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649ecf22920 0x5649ecf9d4a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 --2- 172.21.15.86:0/2540316122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ecf23240 0x5649ecf9d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 >> 172.21.15.86:0/2540316122 conn(0x5649ece516a0 msgr2=0x5649ecf29340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe07ea00700 0 client.27446 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 shutdown_connections 2024-04-25T19:21:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 1 -- 172.21.15.86:0/2540316122 wait complete. 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.349+0000 7fe0aa223b80 20 client.27446 trim_cache size 0 max 16384 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 56, 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "d843219e": { 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 56, 2024-04-25T19:21:02.353 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 60.0, 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 89.8, 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-25T19:21:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0800a3db0 msgr2=0x7fe0800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0800a3db0 0x7fe0800a6270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe090004150 tx=0x7fe090003e20 comp rx=0 tx=0).stop 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 msgr2=0x7fe0981a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 0x7fe0981a5400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00d950 tx=0x7fe08c00dc60 comp rx=0 tx=0).stop 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 shutdown_connections 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe0800a3db0 0x7fe0800a6270 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe09810e3b0 0x7fe09819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0981092d0 0x7fe0981a5400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 --2- 172.21.15.86:0/2777855867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe098109c80 0x7fe0981a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 >> 172.21.15.86:0/2777855867 conn(0x7fe098076210 msgr2=0x7fe09810f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 shutdown_connections 2024-04-25T19:21:02.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:02.350+0000 7fe09f800700 1 -- 172.21.15.86:0/2777855867 wait complete. 2024-04-25T19:21:02.364 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T19:21:02.364 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T19:21:02.374 INFO:tasks.quiescer:joining Quiescers 2024-04-25T19:21:02.374 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-25T19:21:02.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:21:03.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:02 smithi086 ceph-mon[27783]: pgmap v569: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:21:03.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:02 smithi183 ceph-mon[32344]: pgmap v569: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:21:03.209 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd843219e', seconds elapsed: 45.4 2024-04-25T19:21:03.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:02 smithi194 ceph-mon[32740]: pgmap v569: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-04-25T19:21:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:04.919 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:04 smithi086 ceph-mon[27783]: pgmap v570: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 32 op/s 2024-04-25T19:21:05.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:04 smithi183 ceph-mon[32344]: pgmap v570: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 32 op/s 2024-04-25T19:21:05.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:04 smithi194 ceph-mon[32740]: pgmap v570: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 32 op/s 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 -- 172.21.15.86:0/69365243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75901092d0 msgr2=0x7f75901096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 --2- 172.21.15.86:0/69365243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75901092d0 0x7f75901096b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f75800099c0 tx=0x7f7580009cd0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 -- 172.21.15.86:0/69365243 shutdown_connections 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 --2- 172.21.15.86:0/69365243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f759010e3b0 0x7f7590115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 --2- 172.21.15.86:0/69365243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75901092d0 0x7f75901096b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 --2- 172.21.15.86:0/69365243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7590109c80 0x7f759010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 -- 172.21.15.86:0/69365243 >> 172.21.15.86:0/69365243 conn(0x7f7590076210 msgr2=0x7f7590107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 -- 172.21.15.86:0/69365243 shutdown_connections 2024-04-25T19:21:05.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.477+0000 7f7596e00700 1 -- 172.21.15.86:0/69365243 wait complete. 2024-04-25T19:21:05.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.478+0000 7f7596e00700 1 Processor -- start 2024-04-25T19:21:05.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.478+0000 7f7596e00700 1 -- start start 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7596e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75901092d0 0x7f759019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7596e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 0x7f759019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7596e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 0x7f75901a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7596e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7590119240 con 0x7f759010e3b0 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7596e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75901190c0 con 0x7f7590109c80 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7596e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75901193c0 con 0x7f75901092d0 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7587400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 0x7f759019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7594e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 0x7f75901a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f758fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75901092d0 0x7f759019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7594e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 0x7f75901a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40264/0 (socket says 172.21.15.86:40264) 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7587400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 0x7f759019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59724/0 (socket says 172.21.15.86:59724) 2024-04-25T19:21:05.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.479+0000 7f7594e00700 1 -- 172.21.15.86:0/3908418815 learned_addr learned my addr 172.21.15.86:0/3908418815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7587400700 1 -- 172.21.15.86:0/3908418815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75901092d0 msgr2=0x7f759019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7587400700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75901092d0 0x7f759019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7587400700 1 -- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 msgr2=0x7f75901a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7587400700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 0x7f75901a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7587400700 1 -- 172.21.15.86:0/3908418815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75901a4740 con 0x7f7590109c80 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f758fe00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75901092d0 0x7f759019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7594e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 0x7f75901a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7587400700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 0x7f759019fc70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f759006c260 tx=0x7f757800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578016070 con 0x7f7590109c80 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.480+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75901a4a30 con 0x7f7590109c80 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.481+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757800fd40 con 0x7f7590109c80 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.481+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800e3b0 con 0x7f7590109c80 2024-04-25T19:21:05.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.481+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901ac2b0 con 0x7f7590109c80 2024-04-25T19:21:05.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.482+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7590069730 con 0x7f7590109c80 2024-04-25T19:21:05.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.484+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7578019070 con 0x7f7590109c80 2024-04-25T19:21:05.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.485+0000 7f758d600700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f75740a3db0 0x7f75740a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.486+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8637+0+0 (secure 0 0 0) 0x7f75780d0620 con 0x7f7590109c80 2024-04-25T19:21:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.486+0000 7f758fe00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f75740a3db0 0x7f75740a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.486+0000 7f758fe00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f75740a3db0 0x7f75740a6270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7580009990 tx=0x7f7580000bc0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.488+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f757809cbf0 con 0x7f7590109c80 2024-04-25T19:21:05.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.760+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f759002df80 con 0x7f7590109c80 2024-04-25T19:21:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.761+0000 7f758d600700 1 -- 172.21.15.86:0/3908418815 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f757809c4f0 con 0x7f7590109c80 2024-04-25T19:21:05.763 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:21:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f75740a3db0 msgr2=0x7f75740a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f75740a3db0 0x7f75740a6270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7580009990 tx=0x7f7580000bc0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 msgr2=0x7f759019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 0x7f759019fc70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f759006c260 tx=0x7f757800dc30 comp rx=0 tx=0).stop 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 shutdown_connections 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759010e3b0 0x7f75901a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f75740a3db0 0x7f75740a6270 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75901092d0 0x7f759019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 --2- 172.21.15.86:0/3908418815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7590109c80 0x7f759019fc70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 >> 172.21.15.86:0/3908418815 conn(0x7f7590076210 msgr2=0x7f7590077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.764+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 shutdown_connections 2024-04-25T19:21:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:05.765+0000 7f7596e00700 1 -- 172.21.15.86:0/3908418815 wait complete. 2024-04-25T19:21:05.767 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:21:06.068 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:05 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3908418815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:21:06.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:05 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3908418815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:21:06.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:05 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3908418815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:21:06.618 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24527 quiesce db --cancel --all' 2024-04-25T19:21:06.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24527 quiesce db --cancel --all 2024-04-25T19:21:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:07.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:06 smithi086 ceph-mon[27783]: pgmap v571: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-04-25T19:21:07.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:06 smithi183 ceph-mon[32344]: pgmap v571: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-04-25T19:21:07.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:06 smithi194 ceph-mon[32740]: pgmap v571: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-04-25T19:21:08.107 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:07 smithi086 ceph-mon[27783]: osdmap e119: 12 total, 12 up, 12 in 2024-04-25T19:21:08.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:07 smithi183 ceph-mon[32344]: osdmap e119: 12 total, 12 up, 12 in 2024-04-25T19:21:08.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:07 smithi194 ceph-mon[32740]: osdmap e119: 12 total, 12 up, 12 in 2024-04-25T19:21:09.041 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:08 smithi086 ceph-mon[27783]: pgmap v573: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-25T19:21:09.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:08 smithi183 ceph-mon[32344]: pgmap v573: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-25T19:21:09.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:08 smithi194 ceph-mon[32740]: pgmap v573: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-04-25T19:21:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.334+0000 7fd10c400700 1 -- 172.21.15.86:0/381011138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104074c70 msgr2=0x7fd104075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.334+0000 7fd10c400700 1 --2- 172.21.15.86:0/381011138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104074c70 0x7fd104075050 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b0a0 tx=0x7fd0f800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 -- 172.21.15.86:0/381011138 shutdown_connections 2024-04-25T19:21:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 --2- 172.21.15.86:0/381011138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd10410f440 0x7fd104111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 --2- 172.21.15.86:0/381011138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104074c70 0x7fd104075050 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 --2- 172.21.15.86:0/381011138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104075590 0x7fd10410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 -- 172.21.15.86:0/381011138 >> 172.21.15.86:0/381011138 conn(0x7fd104079ba0 msgr2=0x7fd104079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 -- 172.21.15.86:0/381011138 shutdown_connections 2024-04-25T19:21:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.335+0000 7fd10c400700 1 -- 172.21.15.86:0/381011138 wait complete. 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.336+0000 7fd10c400700 1 Processor -- start 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.336+0000 7fd10c400700 1 -- start start 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.336+0000 7fd10c400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 0x7fd10419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.336+0000 7fd10c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104075590 0x7fd10419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd10c400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10410f440 0x7fd10419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd10c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114e40 con 0x7fd104075590 2024-04-25T19:21:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd10c400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114cc0 con 0x7fd10410f440 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd10c400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114fc0 con 0x7fd104074c70 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 0x7fd10419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 0x7fd10419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56922/0 (socket says 172.21.15.86:56922) 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 -- 172.21.15.86:0/2527965403 learned_addr learned my addr 172.21.15.86:0/2527965403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109000700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104075590 0x7fd10419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd10a400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10410f440 0x7fd10419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 -- 172.21.15.86:0/2527965403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10410f440 msgr2=0x7fd10419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10410f440 0x7fd10419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 -- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104075590 msgr2=0x7fd10419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104075590 0x7fd10419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd109a00700 1 -- 172.21.15.86:0/2527965403 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041a02d0 con 0x7fd104074c70 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.337+0000 7fd10a400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10410f440 0x7fd10419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.338+0000 7fd109a00700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 0x7fd10419b310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800d010 tx=0x7fd0f8004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.338+0000 7fd102000700 1 -- 172.21.15.86:0/2527965403 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8021070 con 0x7fd104074c70 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.338+0000 7fd102000700 1 -- 172.21.15.86:0/2527965403 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f8007cc0 con 0x7fd104074c70 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.338+0000 7fd102000700 1 -- 172.21.15.86:0/2527965403 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004710 con 0x7fd104074c70 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.338+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041a0560 con 0x7fd104074c70 2024-04-25T19:21:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.338+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a7e40 con 0x7fd104074c70 2024-04-25T19:21:09.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.343+0000 7fd102000700 1 -- 172.21.15.86:0/2527965403 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0f8003a00 con 0x7fd104074c70 2024-04-25T19:21:09.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd116d33b80 1 Processor -- start 2024-04-25T19:21:09.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd116d33b80 1 -- start start 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd102000700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0ec0a3db0 0x7fd0ec0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec6890 0x55e96fec6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 0x55e96fed1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fed4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.344+0000 7fd116d33b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e96fe19f00 con 0x55e96fed2060 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd116d33b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e96fdc07a0 con 0x55e96fec71b0 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd116d33b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e96fe15ec0 con 0x55e96fec6890 2024-04-25T19:21:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd102000700 1 -- 172.21.15.86:0/2527965403 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7fd0f800f020 con 0x7fd104074c70 2024-04-25T19:21:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109a00700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0ec0a3db0 0x7fd0ec0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 0x55e96fed1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 0x55e96fed1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59754/0 (socket says 172.21.15.86:59754) 2024-04-25T19:21:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 -- 172.21.15.86:0/4041814926 learned_addr learned my addr 172.21.15.86:0/4041814926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd10a400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec6890 0x55e96fec6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd10a400700 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fed4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 -- 172.21.15.86:0/4041814926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec6890 msgr2=0x55e96fec6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec6890 0x55e96fec6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 -- 172.21.15.86:0/4041814926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 msgr2=0x55e96fed4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fed4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.345+0000 7fd109000700 1 -- 172.21.15.86:0/4041814926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96fdb66f0 con 0x55e96fec71b0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.346+0000 7fd109a00700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0ec0a3db0 0x7fd0ec0a6270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd0f809dc80 tx=0x7fd0f8009040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.346+0000 7fd109000700 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 0x55e96fed1b20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009c80 tx=0x7fd0f0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa0fe11668fcb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.346+0000 7fd0f6a00700 1 -- 172.21.15.86:0/4041814926 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0011bf0 con 0x55e96fec71b0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.346+0000 7fd0f6a00700 1 -- 172.21.15.86:0/4041814926 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0011d90 con 0x55e96fec71b0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.346+0000 7fd0f6a00700 1 -- 172.21.15.86:0/4041814926 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00056e0 con 0x55e96fec71b0 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 -- 172.21.15.86:0/4041814926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 msgr2=0x55e96fed1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 0x55e96fed1b20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009c80 tx=0x7fd0f0009f90 comp rx=0 tx=0).stop 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 -- 172.21.15.86:0/4041814926 shutdown_connections 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fed4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec6890 0x55e96fec6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4041814926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec71b0 0x55e96fed1b20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 -- 172.21.15.86:0/4041814926 >> 172.21.15.86:0/4041814926 conn(0x55e96fdf5610 msgr2=0x55e96fec2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 -- 172.21.15.86:0/4041814926 shutdown_connections 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.347+0000 7fd116d33b80 1 -- 172.21.15.86:0/4041814926 wait complete. 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 -- start start 2024-04-25T19:21:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 Processor -- start 2024-04-25T19:21:09.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fecea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96fecef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e96fed4b80 con 0x55e96fec71b0 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e96fdd38d0 con 0x55e96fec6890 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.348+0000 7fd116d33b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e96fed4d00 con 0x55e96fed2060 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd116d33b80 10 client.0 _ll_register_callbacks cb 0x55e96fdb2790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fecea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fecea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40292/0 (socket says 172.21.15.86:40292) 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 -- 172.21.15.86:0/2413099729 learned_addr learned my addr 172.21.15.86:0/2413099729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd10a400700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96fecef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd10a400700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 -- 172.21.15.86:0/2413099729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 msgr2=0x55e96fecef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96fecef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 -- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 msgr2=0x55e96fed1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd109000700 1 -- 172.21.15.86:0/2413099729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96fd67940 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.349+0000 7fd10a400700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd109000700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fecea50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00051e0 tx=0x7fd0f0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662300946f6bd970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd0eb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd0eb400700 1 -- 172.21.15.86:0/2413099729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0027020 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd0eb400700 1 -- 172.21.15.86:0/2413099729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7fd0f00055f0 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f0011d90 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd0eb400700 1 -- 172.21.15.86:0/2413099729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0032050 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd116d33b80 10 client.17373 fetch_fsmap learned FSMap version 29 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.350+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e96fdb0b60 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd0eb400700 1 -- 172.21.15.86:0/2413099729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0011d90 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd0eb400700 1 -- 172.21.15.86:0/2413099729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0f0012a00 con 0x55e96fec71b0 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd116d33b80 10 client.17373 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:21:09.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd116d33b80 10 client.17373 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd116d33b80 20 client.17373 populate_metadata read hostname 'smithi086' 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd116d33b80 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96fee17b0 0x55e96fee3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd116d33b80 4 client.17373 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.351+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e96fe276f0 con 0x55e96fee17b0 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.352+0000 7fd10a400700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96fee17b0 0x55e96fee3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.352+0000 7fd10a400700 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96fee17b0 0x55e96fee3ba0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f76aead53d3c3e server_cookie=f6860382568c9a3b in_seq=0 out_seq=0 2024-04-25T19:21:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.352+0000 7fd0eb400700 10 client.17373 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd0eb400700 1 -- 172.21.15.86:0/2413099729 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e96fe276f0 con 0x55e96fee17b0 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd0eb400700 10 client.17373 handle_command_reply: tid=0 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 client.17373 shutdown 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 msgr2=0x55e96fecea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fecea50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00051e0 tx=0x7fd0f0009480 comp rx=0 tx=0).stop 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 shutdown_connections 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96fee17b0 0x55e96fee3ba0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96fecef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 --2- 172.21.15.86:0/2413099729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fecea50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.353+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 >> 172.21.15.86:0/2413099729 conn(0x55e96fdf5610 msgr2=0x55e96fec2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.354+0000 7fd0eb400700 0 client.17373 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.354+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 shutdown_connections 2024-04-25T19:21:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.354+0000 7fd116d33b80 1 -- 172.21.15.86:0/2413099729 wait complete. 2024-04-25T19:21:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.354+0000 7fd116d33b80 20 client.17373 trim_cache size 0 max 16384 2024-04-25T19:21:09.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 Processor -- start 2024-04-25T19:21:09.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 -- start start 2024-04-25T19:21:09.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec71b0 0x55e96fee9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fee96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041af5b0 con 0x55e96fed2060 2024-04-25T19:21:09.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041b10f0 con 0x55e96fec6890 2024-04-25T19:21:09.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.375+0000 7fd116d33b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041afc80 con 0x55e96fec71b0 2024-04-25T19:21:09.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd109000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec71b0 0x55e96fee9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd109000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec71b0 0x55e96fee9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56942/0 (socket says 172.21.15.86:56942) 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59780/0 (socket says 172.21.15.86:59780) 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd109000700 1 -- 172.21.15.86:0/4193984887 learned_addr learned my addr 172.21.15.86:0/4193984887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fee96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 -- 172.21.15.86:0/4193984887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec71b0 msgr2=0x55e96fee9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec71b0 0x55e96fee9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 -- 172.21.15.86:0/4193984887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 msgr2=0x55e96fee96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fee96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.376+0000 7fd10a400700 1 -- 172.21.15.86:0/4193984887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96feccfc0 con 0x55e96fec6890 2024-04-25T19:21:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.377+0000 7fd10a400700 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed0990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00a010 tx=0x7fd0fc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b4d6f4be45763cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.377+0000 7fd0f6000700 1 -- 172.21.15.86:0/4193984887 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc013c80 con 0x55e96fec6890 2024-04-25T19:21:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.377+0000 7fd0f6000700 1 -- 172.21.15.86:0/4193984887 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc013e20 con 0x55e96fec6890 2024-04-25T19:21:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.377+0000 7fd0f6000700 1 -- 172.21.15.86:0/4193984887 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc005040 con 0x55e96fec6890 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 -- 172.21.15.86:0/4193984887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 msgr2=0x55e96fed0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed0990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00a010 tx=0x7fd0fc009260 comp rx=0 tx=0).stop 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 -- 172.21.15.86:0/4193984887 shutdown_connections 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fed2060 0x55e96fee96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed0990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 --2- 172.21.15.86:0/4193984887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fec71b0 0x55e96fee9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 -- 172.21.15.86:0/4193984887 >> 172.21.15.86:0/4193984887 conn(0x55e96fdf5610 msgr2=0x55e96fed0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 -- 172.21.15.86:0/4193984887 shutdown_connections 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 -- 172.21.15.86:0/4193984887 wait complete. 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 -- start start 2024-04-25T19:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.378+0000 7fd116d33b80 1 Processor -- start 2024-04-25T19:21:09.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fed6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96ff3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041b0a20 con 0x55e96fec71b0 2024-04-25T19:21:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041b0350 con 0x55e96fec6890 2024-04-25T19:21:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041ad570 con 0x55e96fed2060 2024-04-25T19:21:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd116d33b80 10 client.0 _ll_register_callbacks cb 0x55e96fdb2790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:21:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd10a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd109000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fed6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd10a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59782/0 (socket says 172.21.15.86:59782) 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd10a400700 1 -- 172.21.15.86:0/3293161386 learned_addr learned my addr 172.21.15.86:0/3293161386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd109000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fed6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40314/0 (socket says 172.21.15.86:40314) 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.379+0000 7fd10a400700 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96ff3f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd109000700 1 -- 172.21.15.86:0/3293161386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 msgr2=0x55e96ff3f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd109000700 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96ff3f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd109000700 1 -- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 msgr2=0x55e96fed65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:21:09.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd109000700 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd109000700 1 -- 172.21.15.86:0/3293161386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96ff401c0 con 0x55e96fec71b0 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd109000700 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fed6ae0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0000c00 tx=0x7fd0f0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fb71110f8c2434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd0eb400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd0eb400700 1 -- 172.21.15.86:0/3293161386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f003b060 con 0x55e96fec71b0 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd0eb400700 1 -- 172.21.15.86:0/3293161386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7fd0f0011d00 con 0x55e96fec71b0 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.380+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0fc013e20 con 0x55e96fec71b0 2024-04-25T19:21:09.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd0eb400700 1 -- 172.21.15.86:0/3293161386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0042050 con 0x55e96fec71b0 2024-04-25T19:21:09.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 10 client.17379 fetch_fsmap learned FSMap version 29 2024-04-25T19:21:09.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e96ff48500 con 0x55e96fec71b0 2024-04-25T19:21:09.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd0eb400700 1 -- 172.21.15.86:0/3293161386 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc013e20 con 0x55e96fec71b0 2024-04-25T19:21:09.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd0eb400700 1 -- 172.21.15.86:0/3293161386 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0f0012570 con 0x55e96fec71b0 2024-04-25T19:21:09.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 10 client.17379 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:21:09.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 10 client.17379 resolve_mds: validated gid 24527 aka daemon mds.b 2024-04-25T19:21:09.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 20 client.17379 populate_metadata read hostname 'smithi086' 2024-04-25T19:21:09.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96ff48bc0 0x55e96ff4afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:09.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 4 client.17379 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-25T19:21:09.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.381+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 --> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55e96fe21790 con 0x55e96ff48bc0 2024-04-25T19:21:09.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.382+0000 7fd10a400700 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96ff48bc0 0x55e96ff4afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.382+0000 7fd10a400700 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96ff48bc0 0x55e96ff4afe0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae48afa79322f66 server_cookie=276a16e6ae637f9 in_seq=0 out_seq=0 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.382+0000 7fd0eb400700 10 client.17379 ms_handle_connect on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd0eb400700 1 -- 172.21.15.86:0/3293161386 <== mds.0 v2:172.21.15.183:6838/703487979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55e96fe21790 con 0x55e96ff48bc0 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd0eb400700 10 client.17379 handle_command_reply: tid=0 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 client.17379 shutdown 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 msgr2=0x55e96fed6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fed6ae0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0000c00 tx=0x7fd0f0009390 comp rx=0 tx=0).stop 2024-04-25T19:21:09.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 shutdown_connections 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:6838/703487979,v1:172.21.15.183:6839/703487979] conn(0x55e96ff48bc0 0x55e96ff4afe0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96fed2060 0x55e96ff3f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e96fec6890 0x55e96fed65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 --2- 172.21.15.86:0/3293161386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e96fec71b0 0x55e96fed6ae0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd0eb400700 0 client.17379 ms_handle_reset on v2:172.21.15.183:6838/703487979 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 >> 172.21.15.86:0/3293161386 conn(0x55e96fdf5610 msgr2=0x55e96fed0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:09.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 shutdown_connections 2024-04-25T19:21:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 1 -- 172.21.15.86:0/3293161386 wait complete. 2024-04-25T19:21:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.383+0000 7fd116d33b80 20 client.17379 trim_cache size 0 max 16384 2024-04-25T19:21:09.393 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:09.393 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-25T19:21:09.393 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24527, 2024-04-25T19:21:09.394 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 56, 2024-04-25T19:21:09.394 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": {} 2024-04-25T19:21:09.394 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:09.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0ec0a3db0 msgr2=0x7fd0ec0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0ec0a3db0 0x7fd0ec0a6270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd0f809dc80 tx=0x7fd0f8009040 comp rx=0 tx=0).stop 2024-04-25T19:21:09.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 msgr2=0x7fd10419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:09.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 0x7fd10419b310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800d010 tx=0x7fd0f8004db0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 shutdown_connections 2024-04-25T19:21:09.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10410f440 0x7fd10419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fd0ec0a3db0 0x7fd0ec0a6270 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd104074c70 0x7fd10419b310 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 --2- 172.21.15.86:0/2527965403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd104075590 0x7fd10419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:09.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 >> 172.21.15.86:0/2527965403 conn(0x7fd104079ba0 msgr2=0x7fd1040ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:09.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 shutdown_connections 2024-04-25T19:21:09.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:09.384+0000 7fd10c400700 1 -- 172.21.15.86:0/2527965403 wait complete. 2024-04-25T19:21:10.209 INFO:tasks.quiescer:done joining Quiescers 2024-04-25T19:21:10.209 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T19:21:10.220 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:21:10.246 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T19:21:10.255 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T19:21:10.256 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T19:21:10.256 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:21:10.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T19:21:10.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T19:21:10.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T19:21:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.669+0000 7f5b15600640 1 -- 172.21.15.86:0/2840140214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108560 msgr2=0x7f5b101089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.669+0000 7f5b15600640 1 --2- 172.21.15.86:0/2840140214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108560 0x7f5b101089e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009890 tx=0x7f5af802a370 comp rx=0 tx=0).stop 2024-04-25T19:21:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 -- 172.21.15.86:0/2840140214 shutdown_connections 2024-04-25T19:21:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 --2- 172.21.15.86:0/2840140214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108f20 0x7f5b101153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 --2- 172.21.15.86:0/2840140214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108560 0x7f5b101089e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 --2- 172.21.15.86:0/2840140214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1010e620 0x7f5b1010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 -- 172.21.15.86:0/2840140214 >> 172.21.15.86:0/2840140214 conn(0x7f5b100764e0 msgr2=0x7f5b10078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 -- 172.21.15.86:0/2840140214 shutdown_connections 2024-04-25T19:21:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.670+0000 7f5b15600640 1 -- 172.21.15.86:0/2840140214 wait complete. 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 Processor -- start 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 -- start start 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 0x7f5b101a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108f20 0x7f5b101a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1010e620 0x7f5b1019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b101187b0 con 0x7f5b10108560 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10118630 con 0x7f5b1010e620 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b15600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10118930 con 0x7f5b10108f20 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b0ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 0x7f5b101a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b0f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1010e620 0x7f5b1019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b0ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 0x7f5b101a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54476/0 (socket says 172.21.15.86:54476) 2024-04-25T19:21:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b0ea00640 1 -- 172.21.15.86:0/1981026616 learned_addr learned my addr 172.21.15.86:0/1981026616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.671+0000 7f5b07e00640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108f20 0x7f5b101a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b0ea00640 1 -- 172.21.15.86:0/1981026616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108f20 msgr2=0x7f5b101a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b0ea00640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108f20 0x7f5b101a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b0ea00640 1 -- 172.21.15.86:0/1981026616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1010e620 msgr2=0x7f5b1019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b0ea00640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1010e620 0x7f5b1019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b0ea00640 1 -- 172.21.15.86:0/1981026616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1019f1f0 con 0x7f5b10108560 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b0ea00640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 0x7f5b101a48c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000e9f0 tx=0x7f5b0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.672+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000cd80 con 0x7f5b10108560 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.673+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b000054d0 con 0x7f5b10108560 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.673+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1019f4e0 con 0x7f5b10108560 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.673+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00018630 con 0x7f5b10108560 2024-04-25T19:21:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.673+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b101ab780 con 0x7f5b10108560 2024-04-25T19:21:10.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.674+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1004fce0 con 0x7f5b10108560 2024-04-25T19:21:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.676+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b00004e70 con 0x7f5b10108560 2024-04-25T19:21:10.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.678+0000 7f5b07400640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5af40a3dc0 0x7f5af40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:10.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.678+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7f5b000d0e60 con 0x7f5b10108560 2024-04-25T19:21:10.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.678+0000 7f5b07e00640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5af40a3dc0 0x7f5af40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:10.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.679+0000 7f5b07e00640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5af40a3dc0 0x7f5af40a6280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5af8035040 tx=0x7f5af8030340 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:10.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.683+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b0009d400 con 0x7f5b10108560 2024-04-25T19:21:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.912+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5b10062390 con 0x7f5af40a3dc0 2024-04-25T19:21:10.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.929+0000 7f5b07400640 1 -- 172.21.15.86:0/1981026616 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f5b10062390 con 0x7f5af40a3dc0 2024-04-25T19:21:10.930 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T19:02:58", "first": "2024-04-25T19:03:01", "last": "2024-04-25T19:21:00", "last_pruned": "2024-04-25T19:21:00", "created_count": 19, "pruned_count": 13, "active": true} 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.931+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5af40a3dc0 msgr2=0x7f5af40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.931+0000 7f5b15600640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5af40a3dc0 0x7f5af40a6280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5af8035040 tx=0x7f5af8030340 comp rx=0 tx=0).stop 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.931+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 msgr2=0x7f5b101a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.931+0000 7f5b15600640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 0x7f5b101a48c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000e9f0 tx=0x7f5b0000edf0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 shutdown_connections 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f5af40a3dc0 0x7f5af40a6280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b1010e620 0x7f5b1019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b10108f20 0x7f5b101a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 --2- 172.21.15.86:0/1981026616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b10108560 0x7f5b101a48c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:10.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 >> 172.21.15.86:0/1981026616 conn(0x7f5b100764e0 msgr2=0x7f5b10077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 shutdown_connections 2024-04-25T19:21:10.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:10.932+0000 7f5b15600640 1 -- 172.21.15.86:0/1981026616 wait complete. 2024-04-25T19:21:10.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T19:21:11.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 -- 172.21.15.86:0/1896525577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b0104bb0 msgr2=0x7f89b0108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 --2- 172.21.15.86:0/1896525577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b0104bb0 0x7f89b0108bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f899c009900 tx=0x7f899c02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 -- 172.21.15.86:0/1896525577 shutdown_connections 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 --2- 172.21.15.86:0/1896525577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b01092d0 0x7f89b0110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 --2- 172.21.15.86:0/1896525577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b0104bb0 0x7f89b0108bf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 --2- 172.21.15.86:0/1896525577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b0104200 0x7f89b01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 -- 172.21.15.86:0/1896525577 >> 172.21.15.86:0/1896525577 conn(0x7f89b00ffa60 msgr2=0x7f89b0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 -- 172.21.15.86:0/1896525577 shutdown_connections 2024-04-25T19:21:11.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.152+0000 7f89af200640 1 -- 172.21.15.86:0/1896525577 wait complete. 2024-04-25T19:21:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.153+0000 7f89af200640 1 Processor -- start 2024-04-25T19:21:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.153+0000 7f89af200640 1 -- start start 2024-04-25T19:21:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89af200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b0104200 0x7f89b01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89af200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b01092d0 0x7f89b01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89af200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b019a5d0 0x7f89b019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89af200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01141f0 con 0x7f89b0104200 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89af200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0114070 con 0x7f89b019a5d0 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89af200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0114370 con 0x7f89b01092d0 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b0104200 0x7f89b01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ae800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b019a5d0 0x7f89b019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b0104200 0x7f89b01a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54506/0 (socket says 172.21.15.86:54506) 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ade00640 1 -- 172.21.15.86:0/319965735 learned_addr learned my addr 172.21.15.86:0/319965735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ad400640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b01092d0 0x7f89b01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ae800640 1 -- 172.21.15.86:0/319965735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b01092d0 msgr2=0x7f89b01a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ae800640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b01092d0 0x7f89b01a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ae800640 1 -- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b0104200 msgr2=0x7f89b01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ae800640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b0104200 0x7f89b01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.154+0000 7f89ae800640 1 -- 172.21.15.86:0/319965735 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b019b0f0 con 0x7f89b019a5d0 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.155+0000 7f89ae800640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b019a5d0 0x7f89b019aa80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f89a800ec50 tx=0x7f89a800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.155+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8003b70 con 0x7f89b019a5d0 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.155+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a8003d10 con 0x7f89b019a5d0 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.155+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b019b3e0 con 0x7f89b019a5d0 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.155+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8010830 con 0x7f89b019a5d0 2024-04-25T19:21:11.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.155+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b01a75e0 con 0x7f89b019a5d0 2024-04-25T19:21:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.156+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b0050530 con 0x7f89b019a5d0 2024-04-25T19:21:11.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.158+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f89a80109d0 con 0x7f89b019a5d0 2024-04-25T19:21:11.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.160+0000 7f89a6a00640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89940a3d10 0x7f89940a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.160+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7f89a80d0e60 con 0x7f89b019a5d0 2024-04-25T19:21:11.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.160+0000 7f89ade00640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89940a3d10 0x7f89940a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.161+0000 7f89ade00640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89940a3d10 0x7f89940a61d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009c90 tx=0x7f89a0005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:11.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.162+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89a809d490 con 0x7f89b019a5d0 2024-04-25T19:21:11.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:10 smithi183 ceph-mon[32344]: pgmap v574: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:21:11.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:10 smithi194 ceph-mon[32740]: pgmap v574: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:21:11.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:10 smithi086 ceph-mon[27783]: pgmap v574: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:21:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.388+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 --> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f89b0104bb0 con 0x7f89940a3d10 2024-04-25T19:21:11.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.402+0000 7f89a6a00640 1 -- 172.21.15.86:0/319965735 <== mgr.14991 v2:172.21.15.86:6800/421083475 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f89b0104bb0 con 0x7f89940a3d10 2024-04-25T19:21:11.403 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-04-25T19:21:11.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89940a3d10 msgr2=0x7f89940a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89940a3d10 0x7f89940a61d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009c90 tx=0x7f89a0005f90 comp rx=0 tx=0).stop 2024-04-25T19:21:11.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b019a5d0 msgr2=0x7f89b019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b019a5d0 0x7f89b019aa80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f89a800ec50 tx=0x7f89a800c6a0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 shutdown_connections 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f89940a3d10 0x7f89940a61d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89b019a5d0 0x7f89b019aa80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b01092d0 0x7f89b01a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 --2- 172.21.15.86:0/319965735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b0104200 0x7f89b01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 >> 172.21.15.86:0/319965735 conn(0x7f89b00ffa60 msgr2=0x7f89b0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 shutdown_connections 2024-04-25T19:21:11.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.405+0000 7f89af200640 1 -- 172.21.15.86:0/319965735 wait complete. 2024-04-25T19:21:11.425 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T19:21:11.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:21:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 -- 172.21.15.86:0/520430728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 msgr2=0x7f9bb0110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/520430728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 0x7f9bb0110e50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009be0 tx=0x7f9ba802a6c0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 -- 172.21.15.86:0/520430728 shutdown_connections 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/520430728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 0x7f9bb0110e50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/520430728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bb0104b60 0x7f9bb0108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/520430728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb0104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 -- 172.21.15.86:0/520430728 >> 172.21.15.86:0/520430728 conn(0x7f9bb00ff9f0 msgr2=0x7f9bb0101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 -- 172.21.15.86:0/520430728 shutdown_connections 2024-04-25T19:21:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.649+0000 7f9bb5e00640 1 -- 172.21.15.86:0/520430728 wait complete. 2024-04-25T19:21:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.650+0000 7f9bb5e00640 1 Processor -- start 2024-04-25T19:21:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.650+0000 7f9bb5e00640 1 -- start start 2024-04-25T19:21:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.650+0000 7f9bb5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.650+0000 7f9bb5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 0x7f9bb019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.650+0000 7f9bb5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bb019efe0 0x7f9bb019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb01141f0 con 0x7f9bb01041b0 2024-04-25T19:21:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb5e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb0114070 con 0x7f9bb0109280 2024-04-25T19:21:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb5e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb0114370 con 0x7f9bb019efe0 2024-04-25T19:21:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54526/0 (socket says 172.21.15.86:54526) 2024-04-25T19:21:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bb019efe0 0x7f9bb019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 -- 172.21.15.86:0/2908943727 learned_addr learned my addr 172.21.15.86:0/2908943727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9ba7e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 0x7f9bb019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 -- 172.21.15.86:0/2908943727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bb019efe0 msgr2=0x7f9bb019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bb019efe0 0x7f9bb019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 -- 172.21.15.86:0/2908943727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 msgr2=0x7f9bb019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 0x7f9bb019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.651+0000 7f9bb4a00640 1 -- 172.21.15.86:0/2908943727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb019fad0 con 0x7f9bb01041b0 2024-04-25T19:21:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.652+0000 7f9bb4a00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb019a770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e9c0 tx=0x7f9ba000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.652+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000ccc0 con 0x7f9bb01041b0 2024-04-25T19:21:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.652+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba00054d0 con 0x7f9bb01041b0 2024-04-25T19:21:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.652+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb019fdc0 con 0x7f9bb01041b0 2024-04-25T19:21:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.652+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0010430 con 0x7f9bb01041b0 2024-04-25T19:21:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.652+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb01a7600 con 0x7f9bb01041b0 2024-04-25T19:21:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.653+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb00690e0 con 0x7f9bb01041b0 2024-04-25T19:21:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.655+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9ba0004f80 con 0x7f9bb01041b0 2024-04-25T19:21:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.657+0000 7f9bae000640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9b940a3da0 0x7f9b940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.657+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7f9ba00d0c90 con 0x7f9bb01041b0 2024-04-25T19:21:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.657+0000 7f9ba7e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9b940a3da0 0x7f9b940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.658+0000 7f9ba7e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9b940a3da0 0x7f9b940a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bb010a860 tx=0x7f9b98005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.661+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ba009d230 con 0x7f9bb01041b0 2024-04-25T19:21:11.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.867+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bb010c510 con 0x7f9bb01041b0 2024-04-25T19:21:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.870+0000 7f9bae000640 1 -- 172.21.15.86:0/2908943727 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+23404 (secure 0 0 0) 0x7f9ba009cb30 con 0x7f9bb01041b0 2024-04-25T19:21:11.872 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:21:11.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":119,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T19:21:06.809916+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2093097457":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1310908896":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1288701391":"2024-04-26T19:02:56.815518+0000","172.21.15.86:6800/237697797":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/324236586":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/237697797":"2024-04-26T19:02:56.815518+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9b940a3da0 msgr2=0x7f9b940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9b940a3da0 0x7f9b940a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bb010a860 tx=0x7f9b98005f90 comp rx=0 tx=0).stop 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 msgr2=0x7f9bb019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb019a770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e9c0 tx=0x7f9ba000ee80 comp rx=0 tx=0).stop 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 shutdown_connections 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f9b940a3da0 0x7f9b940a6260 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bb019efe0 0x7f9bb019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bb0109280 0x7f9bb019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 --2- 172.21.15.86:0/2908943727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bb01041b0 0x7f9bb019a770 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 >> 172.21.15.86:0/2908943727 conn(0x7f9bb00ff9f0 msgr2=0x7f9bb0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 shutdown_connections 2024-04-25T19:21:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:11.873+0000 7f9bb5e00640 1 -- 172.21.15.86:0/2908943727 wait complete. 2024-04-25T19:21:11.894 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T18:53:19.337576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T18:58:42.545870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T18:58:47.510628+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:21:11.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:21:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 -- 172.21.15.86:0/3315054962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac810e810 msgr2=0x7feac8110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 --2- 172.21.15.86:0/3315054962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac810e810 0x7feac8110c00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feac4009ca0 tx=0x7feac402a8b0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 -- 172.21.15.86:0/3315054962 shutdown_connections 2024-04-25T19:21:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 --2- 172.21.15.86:0/3315054962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac810e810 0x7feac8110c00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 --2- 172.21.15.86:0/3315054962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac806aa90 0x7feac806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 --2- 172.21.15.86:0/3315054962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac8102e20 0x7feac806a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 -- 172.21.15.86:0/3315054962 >> 172.21.15.86:0/3315054962 conn(0x7feac8076c50 msgr2=0x7feac8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 -- 172.21.15.86:0/3315054962 shutdown_connections 2024-04-25T19:21:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.102+0000 7fead0800640 1 -- 172.21.15.86:0/3315054962 wait complete. 2024-04-25T19:21:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.103+0000 7fead0800640 1 Processor -- start 2024-04-25T19:21:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.103+0000 7fead0800640 1 -- start start 2024-04-25T19:21:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.103+0000 7fead0800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 0x7feac81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7fead0800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac8102e20 0x7feac81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7fead0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac810e810 0x7feac819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7fead0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feac81143e0 con 0x7feac810e810 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7fead0800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feac8114260 con 0x7feac806aa90 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7fead0800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feac8114560 con 0x7feac8102e20 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 0x7feac81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 0x7feac81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:59258/0 (socket says 172.21.15.86:59258) 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 -- 172.21.15.86:0/215178750 learned_addr learned my addr 172.21.15.86:0/215178750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feace800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac810e810 0x7feac819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacd400640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac8102e20 0x7feac81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 -- 172.21.15.86:0/215178750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac8102e20 msgr2=0x7feac81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac8102e20 0x7feac81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 -- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac810e810 msgr2=0x7feac819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac810e810 0x7feac819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.104+0000 7feacde00640 1 -- 172.21.15.86:0/215178750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac819afc0 con 0x7feac806aa90 2024-04-25T19:21:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.105+0000 7feacde00640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 0x7feac81a0640 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feabc00ea60 tx=0x7feabc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.105+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc00cde0 con 0x7feac806aa90 2024-04-25T19:21:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.105+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac819b2b0 con 0x7feac806aa90 2024-04-25T19:21:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.105+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc0035f0 con 0x7feac806aa90 2024-04-25T19:21:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.105+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc010690 con 0x7feac806aa90 2024-04-25T19:21:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.105+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81a75c0 con 0x7feac806aa90 2024-04-25T19:21:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.106+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac8069da0 con 0x7feac806aa90 2024-04-25T19:21:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.109+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feabc0037a0 con 0x7feac806aa90 2024-04-25T19:21:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.110+0000 7feac2a00640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feab00a3d20 0x7feab00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.110+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7feabc0d0b00 con 0x7feac806aa90 2024-04-25T19:21:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.110+0000 7feacd400640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feab00a3d20 0x7feab00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.111+0000 7feacd400640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feab00a3d20 0x7feab00a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feac819bf50 tx=0x7feab8005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.114+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feabc09d100 con 0x7feac806aa90 2024-04-25T19:21:12.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.318+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feac81a7790 con 0x7feac806aa90 2024-04-25T19:21:12.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.319+0000 7feac2a00640 1 -- 172.21.15.86:0/215178750 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7feabc09ca00 con 0x7feac806aa90 2024-04-25T19:21:12.320 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.321+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feab00a3d20 msgr2=0x7feab00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.321+0000 7fead0800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feab00a3d20 0x7feab00a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feac819bf50 tx=0x7feab8005f90 comp rx=0 tx=0).stop 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.321+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 msgr2=0x7feac81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.321+0000 7fead0800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 0x7feac81a0640 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feabc00ea60 tx=0x7feabc00ef20 comp rx=0 tx=0).stop 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 shutdown_connections 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7feab00a3d20 0x7feab00a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac810e810 0x7feac819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac8102e20 0x7feac81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 --2- 172.21.15.86:0/215178750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac806aa90 0x7feac81a0640 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 >> 172.21.15.86:0/215178750 conn(0x7feac8076c50 msgr2=0x7feac810f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 shutdown_connections 2024-04-25T19:21:12.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.322+0000 7fead0800640 1 -- 172.21.15.86:0/215178750 wait complete. 2024-04-25T19:21:12.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 -- 172.21.15.86:0/1665663052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c10e830 msgr2=0x7fb49c110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 --2- 172.21.15.86:0/1665663052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c10e830 0x7fb49c110c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb494009be0 tx=0x7fb49402a750 comp rx=0 tx=0).stop 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 -- 172.21.15.86:0/1665663052 shutdown_connections 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 --2- 172.21.15.86:0/1665663052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c10e830 0x7fb49c110c20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 --2- 172.21.15.86:0/1665663052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074b40 0x7fb49c10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 --2- 172.21.15.86:0/1665663052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb49c074220 0x7fb49c074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 -- 172.21.15.86:0/1665663052 >> 172.21.15.86:0/1665663052 conn(0x7fb49c0793a0 msgr2=0x7fb49c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.549+0000 7fb4a1600640 1 -- 172.21.15.86:0/1665663052 shutdown_connections 2024-04-25T19:21:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.550+0000 7fb4a1600640 1 -- 172.21.15.86:0/1665663052 wait complete. 2024-04-25T19:21:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.550+0000 7fb4a1600640 1 Processor -- start 2024-04-25T19:21:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 -- start start 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 0x7fb49c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c074b40 0x7fb49c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb49c10e830 0x7fb49c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1143a0 con 0x7fb49c074220 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c114220 con 0x7fb49c074b40 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb4a1600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c114520 con 0x7fb49c10e830 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb49aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 0x7fb49c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb49b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb49c10e830 0x7fb49c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb49aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 0x7fb49c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54552/0 (socket says 172.21.15.86:54552) 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb49aa00640 1 -- 172.21.15.86:0/541065382 learned_addr learned my addr 172.21.15.86:0/541065382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.551+0000 7fb492000640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c074b40 0x7fb49c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb49aa00640 1 -- 172.21.15.86:0/541065382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb49c10e830 msgr2=0x7fb49c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb49aa00640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb49c10e830 0x7fb49c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb49aa00640 1 -- 172.21.15.86:0/541065382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c074b40 msgr2=0x7fb49c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb49aa00640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c074b40 0x7fb49c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb49aa00640 1 -- 172.21.15.86:0/541065382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c19af70 con 0x7fb49c074220 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb49aa00640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 0x7fb49c1a0630 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9c0 tx=0x7fb48c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00ccc0 con 0x7fb49c074220 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48c0054d0 con 0x7fb49c074220 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.552+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c19b260 con 0x7fb49c074220 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.553+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c010430 con 0x7fb49c074220 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.553+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c1a7480 con 0x7fb49c074220 2024-04-25T19:21:12.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.554+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49c075960 con 0x7fb49c074220 2024-04-25T19:21:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.556+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb48c004f80 con 0x7fb49c074220 2024-04-25T19:21:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.557+0000 7fb493e00640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb4800a3d20 0x7fb4800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.558+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7fb48c0d1070 con 0x7fb49c074220 2024-04-25T19:21:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.558+0000 7fb492000640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb4800a3d20 0x7fb4800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:12.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.558+0000 7fb492000640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb4800a3d20 0x7fb4800a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb49c110620 tx=0x7fb484005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.560+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb48c09d610 con 0x7fb49c074220 2024-04-25T19:21:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.768+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb49c02d5c0 con 0x7fb49c074220 2024-04-25T19:21:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.769+0000 7fb493e00640 1 -- 172.21.15.86:0/541065382 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7fb48c09cf10 con 0x7fb49c074220 2024-04-25T19:21:12.770 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T19:21:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb4800a3d20 msgr2=0x7fb4800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb4800a3d20 0x7fb4800a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb49c110620 tx=0x7fb484005f90 comp rx=0 tx=0).stop 2024-04-25T19:21:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 msgr2=0x7fb49c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 0x7fb49c1a0630 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9c0 tx=0x7fb48c00ee80 comp rx=0 tx=0).stop 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 shutdown_connections 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fb4800a3d20 0x7fb4800a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb49c10e830 0x7fb49c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb49c074b40 0x7fb49c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 --2- 172.21.15.86:0/541065382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb49c074220 0x7fb49c1a0630 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 >> 172.21.15.86:0/541065382 conn(0x7fb49c0793a0 msgr2=0x7fb49c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 shutdown_connections 2024-04-25T19:21:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:12.772+0000 7fb4a1600640 1 -- 172.21.15.86:0/541065382 wait complete. 2024-04-25T19:21:12.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 -- 172.21.15.86:0/965418086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c102a20 msgr2=0x7f663c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 --2- 172.21.15.86:0/965418086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c102a20 0x7f663c106a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6628009900 tx=0x7f662802a400 comp rx=0 tx=0).stop 2024-04-25T19:21:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 -- 172.21.15.86:0/965418086 shutdown_connections 2024-04-25T19:21:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 --2- 172.21.15.86:0/965418086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f663c107140 0x7f663c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 --2- 172.21.15.86:0/965418086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c102a20 0x7f663c106a60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 --2- 172.21.15.86:0/965418086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 0x7f663c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 -- 172.21.15.86:0/965418086 >> 172.21.15.86:0/965418086 conn(0x7f663c0fd8f0 msgr2=0x7f663c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 -- 172.21.15.86:0/965418086 shutdown_connections 2024-04-25T19:21:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.000+0000 7f6642600640 1 -- 172.21.15.86:0/965418086 wait complete. 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.001+0000 7f6642600640 1 Processor -- start 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.001+0000 7f6642600640 1 -- start start 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6642600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 0x7f663c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6642600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c107140 0x7f663c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6642600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f663c074cd0 0x7f663c1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6642600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1120b0 con 0x7f663c107140 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6642600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f663c111f30 con 0x7f663c074cd0 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6642600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f663c112230 con 0x7f663c102070 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6641200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 0x7f663c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f663be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c107140 0x7f663c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6641200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 0x7f663c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41384/0 (socket says 172.21.15.86:41384) 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6641200640 1 -- 172.21.15.86:0/315499042 learned_addr learned my addr 172.21.15.86:0/315499042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f6641c00640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f663c074cd0 0x7f663c1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f663be00640 1 -- 172.21.15.86:0/315499042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 msgr2=0x7f663c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f663be00640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 0x7f663c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f663be00640 1 -- 172.21.15.86:0/315499042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f663c074cd0 msgr2=0x7f663c1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f663be00640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f663c074cd0 0x7f663c1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.002+0000 7f663be00640 1 -- 172.21.15.86:0/315499042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c1a6f00 con 0x7f663c107140 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.003+0000 7f663be00640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c107140 0x7f663c074790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f662802a920 tx=0x7f6628003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.003+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662803f070 con 0x7f663c107140 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.003+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6628031b60 con 0x7f663c107140 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.003+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c1a7190 con 0x7f663c107140 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.003+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628035030 con 0x7f663c107140 2024-04-25T19:21:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.003+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c1a7650 con 0x7f663c107140 2024-04-25T19:21:13.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.004+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663c0690e0 con 0x7f663c107140 2024-04-25T19:21:13.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.006+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6628042070 con 0x7f663c107140 2024-04-25T19:21:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.008+0000 7f6639600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f66240a3d20 0x7f66240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.008+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7f66280f0d80 con 0x7f663c107140 2024-04-25T19:21:13.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.008+0000 7f6641200640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f66240a3d20 0x7f66240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.009+0000 7f6641200640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f66240a3d20 0x7f66240a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6630009c90 tx=0x7f6630005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:13.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.012+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66280bcd30 con 0x7f663c107140 2024-04-25T19:21:13.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:12 smithi086 ceph-mon[27783]: from='client.17385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:21:13.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:12 smithi086 ceph-mon[27783]: pgmap v575: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:21:13.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:12 smithi086 ceph-mon[27783]: from='client.27482 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:21:13.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:12 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2908943727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:21:13.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:12 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/215178750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T19:21:13.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:12 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/541065382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T19:21:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:12 smithi183 ceph-mon[32344]: from='client.17385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:21:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:12 smithi183 ceph-mon[32344]: pgmap v575: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:21:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:12 smithi183 ceph-mon[32344]: from='client.27482 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:21:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:12 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2908943727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:21:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:12 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/215178750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T19:21:13.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:12 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/541065382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T19:21:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.214+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f663c10a370 con 0x7f663c107140 2024-04-25T19:21:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.215+0000 7f6639600640 1 -- 172.21.15.86:0/315499042 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f66280c1030 con 0x7f663c107140 2024-04-25T19:21:13.216 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.217+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f66240a3d20 msgr2=0x7f66240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.217+0000 7f6642600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f66240a3d20 0x7f66240a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6630009c90 tx=0x7f6630005f90 comp rx=0 tx=0).stop 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.217+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c107140 msgr2=0x7f663c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.217+0000 7f6642600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c107140 0x7f663c074790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f662802a920 tx=0x7f6628003c10 comp rx=0 tx=0).stop 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.217+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 shutdown_connections 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.217+0000 7f6642600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f66240a3d20 0x7f66240a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.218+0000 7f6642600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f663c074cd0 0x7f663c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.218+0000 7f6642600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c107140 0x7f663c074790 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.218+0000 7f6642600640 1 --2- 172.21.15.86:0/315499042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f663c102070 0x7f663c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.218+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 >> 172.21.15.86:0/315499042 conn(0x7f663c0fd8f0 msgr2=0x7f663c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.218+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 shutdown_connections 2024-04-25T19:21:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.218+0000 7f6642600640 1 -- 172.21.15.86:0/315499042 wait complete. 2024-04-25T19:21:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:12 smithi194 ceph-mon[32740]: from='client.17385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:21:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:12 smithi194 ceph-mon[32740]: pgmap v575: 97 pgs: 97 active+clean; 1.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-25T19:21:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:12 smithi194 ceph-mon[32740]: from='client.27482 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:21:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:12 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2908943727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:21:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:12 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/215178750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T19:21:13.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:12 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/541065382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T19:21:13.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 -- 172.21.15.86:0/57158443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c074380 msgr2=0x7ff61c074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 --2- 172.21.15.86:0/57158443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c074380 0x7ff61c074760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009900 tx=0x7ff60c02a480 comp rx=0 tx=0).stop 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 -- 172.21.15.86:0/57158443 shutdown_connections 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 --2- 172.21.15.86:0/57158443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff61c10e570 0x7ff61c1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 --2- 172.21.15.86:0/57158443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074ca0 0x7ff61c10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 --2- 172.21.15.86:0/57158443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c074380 0x7ff61c074760 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 -- 172.21.15.86:0/57158443 >> 172.21.15.86:0/57158443 conn(0x7ff61c0fd880 msgr2=0x7ff61c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 -- 172.21.15.86:0/57158443 shutdown_connections 2024-04-25T19:21:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.442+0000 7ff620a00640 1 -- 172.21.15.86:0/57158443 wait complete. 2024-04-25T19:21:13.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 Processor -- start 2024-04-25T19:21:13.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 -- start start 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 0x7ff61c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff61c074ca0 0x7ff61c1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 0x7ff61c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c1141f0 con 0x7ff61c10e570 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c114070 con 0x7ff61c074ca0 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.444+0000 7ff620a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c114370 con 0x7ff61c074380 2024-04-25T19:21:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 0x7ff61c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 0x7ff61c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff61c074ca0 0x7ff61c1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 0x7ff61c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54580/0 (socket says 172.21.15.86:54580) 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 0x7ff61c1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:41406/0 (socket says 172.21.15.86:41406) 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61be00640 1 -- 172.21.15.86:0/672483698 learned_addr learned my addr 172.21.15.86:0/672483698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 -- 172.21.15.86:0/672483698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff61c074ca0 msgr2=0x7ff61c1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff61c074ca0 0x7ff61c1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 -- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 msgr2=0x7ff61c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 0x7ff61c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 -- 172.21.15.86:0/672483698 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c19ad30 con 0x7ff61c074380 2024-04-25T19:21:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61be00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 0x7ff61c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:21:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.445+0000 7ff61b400640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 0x7ff61c1a0400 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff60c02aa70 tx=0x7ff60c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.446+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c03f070 con 0x7ff61c074380 2024-04-25T19:21:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.446+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c19afc0 con 0x7ff61c074380 2024-04-25T19:21:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.446+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff60c031b50 con 0x7ff61c074380 2024-04-25T19:21:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.446+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c035030 con 0x7ff61c074380 2024-04-25T19:21:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.446+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a72c0 con 0x7ff61c074380 2024-04-25T19:21:13.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.447+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c0690e0 con 0x7ff61c074380 2024-04-25T19:21:13.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.449+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff60c042070 con 0x7ff61c074380 2024-04-25T19:21:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.451+0000 7ff613e00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff6000a3da0 0x7ff6000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.451+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8681+0+0 (secure 0 0 0) 0x7ff60c0f1390 con 0x7ff61c074380 2024-04-25T19:21:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.451+0000 7ff61aa00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff6000a3da0 0x7ff6000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.452+0000 7ff61aa00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff6000a3da0 0x7ff6000a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6080095f0 tx=0x7ff608005f90 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:13.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.453+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff60c031d00 con 0x7ff61c074380 2024-04-25T19:21:13.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.713+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ff61c109720 con 0x7ff61c074380 2024-04-25T19:21:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.726+0000 7ff613e00640 1 -- 172.21.15.86:0/672483698 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746060 (secure 0 0 0) 0x7ff60c0bd2e0 con 0x7ff61c074380 2024-04-25T19:21:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:21:13.739 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 26, 2024-04-25T19:21:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 14991, 2024-04-25T19:21:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 421083475 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 421083475 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/421083475", 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-04-25T19:02:56.815814+0000", 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 25105, 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-25T19:21:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-25T19:21:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-25T19:21:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-25T19:21:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-25T19:21:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T19:21:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-25T19:21:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-25T19:21:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-25T19:21:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-25T19:21:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-25T19:21:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-25T19:21:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-25T19:21:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-25T19:21:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T19:21:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T19:21:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T19:21:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T19:21:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-25T19:21:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T19:21:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T19:21:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-25T19:21:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-25T19:21:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T19:21:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-25T19:21:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-25T19:21:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T19:21:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T19:21:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T19:21:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T19:21:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T19:21:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-25T19:21:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-25T19:21:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T19:21:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-25T19:21:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T19:21:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-25T19:21:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T19:21:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-25T19:21:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-25T19:21:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-25T19:21:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T19:21:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-25T19:21:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-25T19:21:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-25T19:21:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-25T19:21:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-25T19:21:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-25T19:21:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T19:21:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-25T19:21:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 25115, 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-25T19:21:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-25T19:21:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-25T19:21:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T19:21:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-25T19:21:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-25T19:21:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-25T19:21:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-25T19:21:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-25T19:21:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-25T19:21:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-25T19:21:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-25T19:21:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-25T19:21:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T19:21:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T19:21:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T19:21:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-25T19:21:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T19:21:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-25T19:21:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T19:21:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T19:21:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T19:21:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-25T19:21:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-25T19:21:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-25T19:21:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-25T19:21:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-25T19:21:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-25T19:21:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-25T19:21:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-25T19:21:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-25T19:21:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-25T19:21:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-25T19:21:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-25T19:21:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-25T19:21:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-25T19:21:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-25T19:21:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T19:21:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-25T19:21:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T19:21:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-25T19:21:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-25T19:21:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T19:21:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-25T19:21:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-25T19:21:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-25T19:21:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-25T19:21:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-25T19:21:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-25T19:21:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-25T19:21:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-25T19:21:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "restful", 2024-04-25T19:21:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_schedule" 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-25T19:21:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-25T19:21:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T19:21:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:21:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-25T19:21:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-25T19:21:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T19:21:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T19:21:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-25T19:21:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-25T19:21:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-25T19:21:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-25T19:21:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-25T19:21:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T19:21:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T19:21:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T19:21:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T19:21:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T19:21:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T19:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T19:21:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T19:21:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T19:21:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T19:21:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T19:21:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T19:21:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T19:21:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T19:21:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-25T19:21:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T19:21:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-25T19:21:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-25T19:21:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-25T19:21:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-25T19:21:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-25T19:21:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.767 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:13 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/315499042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T19:21:14.767 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:13 smithi194 ceph-mon[32740]: osdmap e120: 12 total, 12 up, 12 in 2024-04-25T19:21:14.767 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:13 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/672483698' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T19:21:14.768 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:13 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/315499042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T19:21:14.768 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:13 smithi183 ceph-mon[32344]: osdmap e120: 12 total, 12 up, 12 in 2024-04-25T19:21:14.768 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:13 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/672483698' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-25T19:21:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-25T19:21:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T19:21:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-25T19:21:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T19:21:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-25T19:21:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-25T19:21:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-25T19:21:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-25T19:21:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-25T19:21:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-25T19:21:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-25T19:21:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-25T19:21:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-25T19:21:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-25T19:21:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-25T19:21:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-25T19:21:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-25T19:21:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-25T19:21:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-25T19:21:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-25T19:21:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-04-25T19:21:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-25T19:21:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "squid": [ 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-25T19:21:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 94, 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2259314965 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1373271292 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2718987035 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1749400536 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3265130553 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T19:21:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3522082212 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T19:21:14.979 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff6000a3da0 msgr2=0x7ff6000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff6000a3da0 0x7ff6000a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6080095f0 tx=0x7ff608005f90 comp rx=0 tx=0).stop 2024-04-25T19:21:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 msgr2=0x7ff61c1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 0x7ff61c1a0400 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff60c02aa70 tx=0x7ff60c003b90 comp rx=0 tx=0).stop 2024-04-25T19:21:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 shutdown_connections 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7ff6000a3da0 0x7ff6000a6260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff61c10e570 0x7ff61c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff61c074ca0 0x7ff61c1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 --2- 172.21.15.86:0/672483698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff61c074380 0x7ff61c1a0400 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 >> 172.21.15.86:0/672483698 conn(0x7ff61c0fd880 msgr2=0x7ff61c0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 shutdown_connections 2024-04-25T19:21:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:13.742+0000 7ff620a00640 1 -- 172.21.15.86:0/672483698 wait complete. 2024-04-25T19:21:14.987 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T19:21:14.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T19:21:14.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:13 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/315499042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T19:21:14.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:13 smithi086 ceph-mon[27783]: osdmap e120: 12 total, 12 up, 12 in 2024-04-25T19:21:14.987 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:13 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/672483698' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T19:21:15.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:14 smithi183 ceph-mon[32344]: pgmap v576: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-04-25T19:21:15.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:14 smithi194 ceph-mon[32740]: pgmap v576: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-04-25T19:21:15.319 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:14 smithi086 ceph-mon[27783]: pgmap v576: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-04-25T19:21:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:17.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:16 smithi183 ceph-mon[32344]: pgmap v578: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-25T19:21:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:16 smithi194 ceph-mon[32740]: pgmap v578: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-25T19:21:17.256 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:16 smithi086 ceph-mon[27783]: pgmap v578: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 16 op/s 2024-04-25T19:21:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:17.801 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:17.801 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 374, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 363, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1633360, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 224896, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.145829670, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.057856663, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.083505181, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.016344361, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 362, 2024-04-25T19:21:17.802 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.033923266, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000093710 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1633360, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 224896 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 6, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 279, 2024-04-25T19:21:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3360, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 205408, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.054252356, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.041811952, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.004538479, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 277, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.040038685, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000144543 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:17.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 205408 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-25T19:21:17.805 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.000849720, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000696793, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:17.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:17.807 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:17.808 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:17.809 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 149745, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 31, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-25T19:21:17.810 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 376, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1580505, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 376, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 376, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1580505, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:17.812 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:17.813 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:17.813 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:18.788 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T19:21:18.789 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T19:21:18.789 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T19:21:18.789 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T19:21:18.789 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T19:21:18.789 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T19:21:18.789 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T19:21:18.789 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T19:21:18.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T19:21:19.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:18 smithi086 ceph-mon[27783]: pgmap v579: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-04-25T19:21:19.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:18 smithi183 ceph-mon[32344]: pgmap v579: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-04-25T19:21:19.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:18 smithi194 ceph-mon[32740]: pgmap v579: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2024-04-25T19:21:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:21.027 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:20 smithi086 ceph-mon[27783]: pgmap v580: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T19:21:21.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:20 smithi183 ceph-mon[32344]: pgmap v580: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T19:21:21.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:20 smithi194 ceph-mon[32740]: pgmap v580: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T19:21:21.575 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1336, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1338, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 352537, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 424912, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.271287954, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.099644922, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.069803642, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.070925722, 2024-04-25T19:21:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1336, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.109211078, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000081744 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1662, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1649, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1989928, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 630943, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.444365229, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.174119525, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.174227249, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.097654407, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1648, 2024-04-25T19:21:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.134237500, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000081454 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1684128, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 235664 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1333, 2024-04-25T19:21:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1612, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 449232, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1073634, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.386557575, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.194083870, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.076120215, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.076837182, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1610, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.169397333, 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000105215 2024-04-25T19:21:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 651648 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-25T19:21:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 6421, 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.334602772, 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000052110 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-25T19:21:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-25T19:21:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-25T19:21:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 7, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-25T19:21:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4196266, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4196125, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 7 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 49664, 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-25T19:21:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-25T19:21:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-25T19:21:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 3, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 2, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-25T19:21:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 4062, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 191976, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 55, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 176, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-25T19:21:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 3906, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 3906, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3906, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.701921732, 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000947752 2024-04-25T19:21:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3906, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3907 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 3906, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3906, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 2568, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-25T19:21:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1336, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-25T19:21:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 120, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 31, 2024-04-25T19:21:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 5, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 6, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 1, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 4327, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2377767, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 4327, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4327, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2377767, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3906, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1967, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3906, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1967, 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3906, 2024-04-25T19:21:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3906, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3906, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3906, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5627ab294780": { 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:21.623 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:22.653 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:22.653 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T19:21:22.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T19:21:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:23.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:22 smithi183 ceph-mon[32344]: pgmap v581: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T19:21:23.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:22 smithi194 ceph-mon[32740]: pgmap v581: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T19:21:23.256 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:22 smithi086 ceph-mon[27783]: pgmap v581: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T19:21:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:24 smithi086 ceph-mon[27783]: pgmap v582: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T19:21:25.169 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:24 smithi183 ceph-mon[32344]: pgmap v582: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T19:21:25.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:24 smithi194 ceph-mon[32740]: pgmap v582: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2743, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2066, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2315241, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 549436, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.449949360, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.187194885, 2024-04-25T19:21:25.387 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.275230880, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.029486913, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2065, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.159635088, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000077305 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3749, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001378296, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000367 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1711344, 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 238544 2024-04-25T19:21:25.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 27, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 303, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 8102, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 819242, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 20, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.071437952, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.044692943, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.017063448, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000256137, 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:25.389 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 301, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.062776626, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000208560 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 13, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000003195, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000245 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 4448, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 742912 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:25.390 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 14, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 16, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3871, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 12055, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.006939753, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000355521, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.005370424, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000498059, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 14, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.030808314, 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002200593 2024-04-25T19:21:25.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-25T19:21:25.392 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 42, 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.002442492, 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000058154 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:25.393 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000009465, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000004732 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.394 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 11, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 17, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 1, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 16, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:25.395 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 13, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 4, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 3, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 39, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 34, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 5, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-25T19:21:25.396 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 4, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 234, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 6, 2024-04-25T19:21:25.397 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:25.398 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 17, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 11, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 12, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:25.399 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 14, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 86, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:25.400 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 4, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 4, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 2, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-25T19:21:25.401 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4195424, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.007906873, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002635624 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.402 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 21, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 23, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 2, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 20, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 24, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 4, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 17, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 21, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 4, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 51840, 2024-04-25T19:21:25.403 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 223608 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 4, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 86, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-25T19:21:25.404 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.405 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-25T19:21:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-25T19:21:25.408 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.409 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:25.411 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.412 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.414 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-25T19:21:25.415 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 73, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:25.416 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:25.417 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:25.418 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 239447, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 76, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-25T19:21:25.419 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 176, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 89293, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 353, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:25.420 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 38, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 11332, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 33, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 38, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 38, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.290993888, 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007657733 2024-04-25T19:21:25.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 38, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 60 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 38, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 0, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 38, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-04-25T19:21:25.422 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 3, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 6, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 6, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:25.423 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-25T19:21:25.424 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 120, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 31, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 9, 2024-04-25T19:21:25.425 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 14, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 5, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 14, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 1 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-25T19:21:25.426 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 99 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2780, 2024-04-25T19:21:25.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2050711, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2780, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2780, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2050711, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:25.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 38, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 11332, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 38, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 11332, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 38, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 38, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 38, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 38, 2024-04-25T19:21:25.430 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-25T19:21:25.431 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x564c578fa500": { 2024-04-25T19:21:25.432 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 4, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1120, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 4, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1120, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:25.433 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.434 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:25.434 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=17 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:26.410 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:26.410 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T19:21:26.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T19:21:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:27.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:26 smithi183 ceph-mon[32344]: pgmap v583: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T19:21:27.187 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:26 smithi086 ceph-mon[27783]: pgmap v583: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T19:21:27.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:26 smithi194 ceph-mon[32740]: pgmap v583: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T19:21:29.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:28 smithi183 ceph-mon[32344]: pgmap v584: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:29.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:28 smithi086 ceph-mon[27783]: pgmap v584: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:29.177 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T19:21:29.177 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:29.177 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.003810768, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000829899, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:29.178 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 7, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 289, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3520, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 285536, 2024-04-25T19:21:29.179 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.059990928, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.044565641, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.006897503, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 287, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.049912335, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000173910 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:29.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 285536 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 384, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 373, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1636080, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 231296, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.145943884, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.057705200, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.084580245, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.017533408, 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:29.181 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 372, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.035380532, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000095108 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1636080, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 231296 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-25T19:21:29.182 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:29.184 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148714, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 30, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-25T19:21:29.185 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:29.186 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 387, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1581832, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 387, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 387, 2024-04-25T19:21:29.187 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1581832, 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T19:21:29.188 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T19:21:29.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:28 smithi194 ceph-mon[32740]: pgmap v584: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T19:21:30.043 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T19:21:30.043 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T19:21:30.044 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T19:21:30.292 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:21:31.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:30 smithi183 ceph-mon[32344]: pgmap v585: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:31.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:30 smithi194 ceph-mon[32740]: pgmap v585: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:31.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:30 smithi086 ceph-mon[27783]: pgmap v585: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:32.200 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5852, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 7376, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1676262, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 16714936, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 99, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.571734254, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.202899001, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.207085605, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.123316103, 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7381, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.752954484, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000102012 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7471, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001797244, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000240 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 491472 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5541, 2024-04-25T19:21:32.202 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 7527, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1955415, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 12198399, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.694802125, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.272828587, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.285278138, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.142702949, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7528, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.431265262, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000057288 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.203 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8789, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002181381, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000248 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1096304, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 1738112 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4755, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4110, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3048174, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 7988566, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 156, 2024-04-25T19:21:32.204 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 10, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.571518419, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.166452738, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.263766589, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.142120884, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4099, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.175507010, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042817 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5901, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001650970, 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000279 2024-04-25T19:21:32.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2452976, 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 470832 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.206 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6758, 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.982029078, 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000293286 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000462356, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000028897 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 894, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 894, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 894, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 48.743044123, 2024-04-25T19:21:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.054522420 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 521, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 48, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 20, 2024-04-25T19:21:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 28, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 19, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 2, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 3, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-25T19:21:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 982, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 968, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T19:21:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 672, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 668, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 599, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 743, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 675, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 4, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 627 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 23, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-25T19:21:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 49, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 2, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 3, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 1, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 14, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 192, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 2670, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 2666, 2024-04-25T19:21:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 2666, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 11, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 90, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 89, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 89, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 1, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 1, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 1, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 29812385, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 29845447, 2024-04-25T19:21:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2576, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 7.356342057, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002855722 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 41, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 42, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 20, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 20, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-25T19:21:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 48, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 49, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 3, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 19, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 16, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 61392, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 231800 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 2183, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 345, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 894, 2024-04-25T19:21:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 47, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 690, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 17, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.011598054, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000682238 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 196, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.045078968, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000229994 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002059337, 2024-04-25T19:21:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000411867 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000233350, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000233350 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T19:21:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 19, 2024-04-25T19:21:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003974592, 2024-04-25T19:21:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000209189 2024-04-25T19:21:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T19:21:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 7.171242000, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.896405250 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 19, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.130849263, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006886803 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000245906, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000245906 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004565822, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002282911 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002328586, 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001164293 2024-04-25T19:21:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 37.292478146, 2024-04-25T19:21:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 18.646239073 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 13, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.085561656, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006581665 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 600, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.970562311, 2024-04-25T19:21:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006617603 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.019928800, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002491100 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-25T19:21:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002337332, 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002337332 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 71, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 35, 2024-04-25T19:21:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1112, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 313110, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 118, 2024-04-25T19:21:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 194706, 2024-04-25T19:21:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1460, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 4723, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 27539966, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 31, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 4723, 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T19:21:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4723, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 32.800094832, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006944758 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4723, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5804 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 4723, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 1, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 4722, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 501, 2024-04-25T19:21:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 28, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 2577, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 286, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 525, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 56, 2024-04-25T19:21:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 599, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T19:21:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T19:21:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 120, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 31, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 13, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 1214, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 17 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T19:21:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-04-25T19:21:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T19:21:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T19:21:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T19:21:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 547 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 1, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 1, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 1, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 1, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 16144, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5004650, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 16144, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 16144, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5004650, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4723, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 27540115, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4723, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 27540115, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4723, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4723, 2024-04-25T19:21:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4723, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4723, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 149, 2024-04-25T19:21:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 149, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55f31e914500": { 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2670, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 25651143, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2670, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2576, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 25651143, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:32.249 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T19:21:33.048 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T19:21:33.048 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T19:21:33.048 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=23 2024-04-25T19:21:33.048 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T19:21:33.048 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T19:21:33.048 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=19 2024-04-25T19:21:33.049 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=13 2024-04-25T19:21:33.049 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T19:21:33.049 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T19:21:33.070 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:32 smithi183 ceph-mon[32344]: pgmap v586: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:33.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:32 smithi194 ceph-mon[32740]: pgmap v586: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:33.301 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:21:33.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:32 smithi086 ceph-mon[27783]: pgmap v586: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:35.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:34 smithi183 ceph-mon[32344]: pgmap v587: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:35.078 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T19:21:35.078 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:35.078 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3201, 2024-04-25T19:21:35.078 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3188, 2024-04-25T19:21:35.078 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 11376303, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1121152, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.324627323, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.120000809, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.106077798, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.082002897, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3187, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.127022255, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000039856 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:35.079 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1685104, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 237744 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 777, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1061, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 8584932, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 990598, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:35.080 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.089750837, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.027342569, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.030851772, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.023168563, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1058, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.052455964, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000049580 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.081 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 744400 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1184, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1184, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 8686658, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 375695, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.114299333, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.028548843, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.038976935, 2024-04-25T19:21:35.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.037894640, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1183, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.036655019, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000030984 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.083 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8056, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.197875949, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000024562 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:35.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T19:21:35.085 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 38, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 20, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 18, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 11, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-25T19:21:35.086 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:35.087 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:35.088 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-25T19:21:35.089 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:21:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 2670, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T19:21:35.091 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 84, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 29812385, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 29845447, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 29845335, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 2670 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 33, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 34, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 12, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 12, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 38, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 38, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-25T19:21:35.093 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 51844, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215416 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.097 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.099 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T19:21:35.100 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.100 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.100 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.101 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T19:21:35.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.101 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T19:21:35.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.103 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T19:21:35.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.104 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T19:21:35.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.105 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:35.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.105 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.107 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T19:21:35.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.107 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.107 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T19:21:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T19:21:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T19:21:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T19:21:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T19:21:35.117 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-25T19:21:35.117 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-04-25T19:21:35.117 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-04-25T19:21:35.117 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 5, 2024-04-25T19:21:35.117 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T19:21:35.117 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-25T19:21:35.118 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-25T19:21:35.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 5571, 2024-04-25T19:21:35.118 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:35.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.119 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T19:21:35.119 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:35.119 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:35.119 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:35.119 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:35.120 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:35.120 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:35.120 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:35.120 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:35.120 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:35.120 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:35.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:35.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:35.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:35.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:35.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:35.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:35.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:35.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:35.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:35.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:35.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:35.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:35.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:35.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:35.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:35.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:35.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:35.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 208231, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 93, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T19:21:35.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 155722, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1174, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T19:21:35.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 4732, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 4732, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4732, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.548929550, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000538657 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4732, 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4733 2024-04-25T19:21:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 4732, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 4732, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2674, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 2055, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T19:21:35.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T19:21:35.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T19:21:35.130 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:35.130 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T19:21:35.131 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:35.131 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-25T19:21:35.131 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T19:21:35.131 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T19:21:35.131 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T19:21:35.132 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 120, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 31, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 7, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 8, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:35.133 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:35.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 5158, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 28158928, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 5158, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 5158, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 28158928, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.135 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4732, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 25651153, 2024-04-25T19:21:35.136 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4732, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 25651153, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:35.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4732, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4732, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4732, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4732, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:35.138 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x563b25e1a6e0": { 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:35.140 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:35.141 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:35.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:35.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:35.141 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T19:21:35.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:34 smithi194 ceph-mon[32740]: pgmap v587: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:35.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:34 smithi086 ceph-mon[27783]: pgmap v587: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:36.036 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T19:21:36.036 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T19:21:36.036 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T19:21:36.036 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T19:21:36.037 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-25T19:21:36.037 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:36.037 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:36.037 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T19:21:36.037 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T19:21:36.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:35 smithi194 ceph-mon[32740]: pgmap v588: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:36.314 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:21:36.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:35 smithi086 ceph-mon[27783]: pgmap v588: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:36.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:35 smithi183 ceph-mon[32344]: pgmap v588: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:38.104 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T19:21:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 500, 2024-04-25T19:21:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 513, 2024-04-25T19:21:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1721324, 2024-04-25T19:21:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 310023, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 13, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.097207060, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.032486901, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.059450067, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.010482787, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 512, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.043329950, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000084628 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 80, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000022222, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000277 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1716560, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 254016 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 25, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 309, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 8238, 2024-04-25T19:21:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 751944, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.019361485, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.012194025, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.003932913, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000294298, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 306, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.044803395, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000146416 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000001868, 2024-04-25T19:21:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000207 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 4448, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 739840 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2235, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1638, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 591432, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 538505, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 10, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.189068506, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.091721900, 2024-04-25T19:21:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.105279689, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.006812877, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1636, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.077849657, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000047585 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3668, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001062941, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000289 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 59, 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001630125, 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000027629 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000010787, 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000005393 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 4, 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 1, 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T19:21:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000222470, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000222470 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 3, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 17, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 1, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 16, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 13, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 4, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 3, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 44, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 35, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 1, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 6, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T19:21:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:38.114 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 2 2024-04-25T19:21:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 15, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 10, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 12, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 14, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 86, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T19:21:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 7, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 7, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-25T19:21:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4196266, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012270864, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002045144 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 21, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 23, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 2, 2024-04-25T19:21:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 20, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 23, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 3, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 17, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 21, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 4, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 1, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 1, 2024-04-25T19:21:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 53292, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223608 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 5, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 4, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 23, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 86, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000222470, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000222470 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T19:21:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 1, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 1, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1119, 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 247831, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 81, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 89341, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 359, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 43, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 12354, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-04-25T19:21:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 43, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 43, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.340990586, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.007930013 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 43, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 65 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 43, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 43, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T19:21:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 6, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 8, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-25T19:21:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T19:21:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 120, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 31, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 8, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 11, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 3, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 14, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 99 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-25T19:21:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2756, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2045422, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2756, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2756, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2045422, 2024-04-25T19:21:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 43, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 12354, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 43, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12354, 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T19:21:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 43, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 43, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 43, 2024-04-25T19:21:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 43, 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55ab4e814500": { 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 7, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1962, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 7, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 6, 2024-04-25T19:21:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1962, 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:38.157 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T19:21:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:38 smithi183 ceph-mon[32344]: pgmap v589: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:38.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:38 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:21:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:38 smithi194 ceph-mon[32740]: pgmap v589: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:38.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:38 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:21:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:38 smithi086 ceph-mon[27783]: pgmap v589: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T19:21:38.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:38 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=15 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:38.932 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:38.932 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T19:21:38.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T19:21:39.174 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:21:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:40 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:40 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:40 smithi183 ceph-mon[32344]: pgmap v590: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:40 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:40 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:40 smithi194 ceph-mon[32740]: pgmap v590: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:40 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:40 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:40.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:40 smithi086 ceph-mon[27783]: pgmap v590: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1774, 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1761, 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2138960, 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 668808, 2024-04-25T19:21:40.929 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.228794266, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.089106254, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.078767932, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.051961011, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1760, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.098936682, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000056214 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.930 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1686032, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 240144 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1127, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1414, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 277722, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1016538, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.119468667, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.037885710, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.027731507, 2024-04-25T19:21:40.931 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.042923634, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1411, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.055596091, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000039401 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 658720 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.932 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1358, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1358, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 359933, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 431021, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.105783309, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.033611124, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.029124545, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.033065472, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1357, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.037301191, 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000027487 2024-04-25T19:21:40.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:40.934 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6258, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.169207593, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000027038 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-25T19:21:40.935 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 0, 2024-04-25T19:21:40.936 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:40.937 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:40.938 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:40.939 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.941 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 8, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-25T19:21:40.942 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4196488, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4196347, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 8 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-25T19:21:40.943 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 3, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 3, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 1, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 1, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 51140, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215416 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.944 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.949 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.949 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-25T19:21:40.949 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.949 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.949 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.950 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.951 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-25T19:21:40.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.955 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-25T19:21:40.957 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 3, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 3, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1868, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:40.958 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:40.959 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:40.960 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 190923, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 54, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:40.961 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 176, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-25T19:21:40.962 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 8512, 2024-04-25T19:21:40.963 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 13, 2024-04-25T19:21:40.963 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3827, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3827, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3827, 2024-04-25T19:21:40.964 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.021944140, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000528336 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3827, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3828 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3827, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3827, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-25T19:21:40.965 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2470, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1355, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-04-25T19:21:40.966 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-25T19:21:40.967 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 120, 2024-04-25T19:21:40.968 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 31, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 7, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 2, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.969 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-25T19:21:40.970 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4255, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2368801, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4255, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:40.971 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4255, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2368801, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:40.972 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3827, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2189, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3827, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2189, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:40.973 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3827, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3827, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3827, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3827, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:40.975 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55ff434766e0": { 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:40.976 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-25T19:21:40.977 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-25T19:21:41.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:41 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:21:41.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:41 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:21:41.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:41 smithi194 ceph-mon[32740]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:41 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:21:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:41 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:21:41.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:41 smithi086 ceph-mon[27783]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:41.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:41 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:21:41.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:41 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:21:41.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:41 smithi183 ceph-mon[32344]: from='mgr.14991 ' entity='mgr.x' 2024-04-25T19:21:41.667 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T19:21:41.668 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T19:21:41.668 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T19:21:41.668 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-25T19:21:41.668 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-25T19:21:41.668 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:41.668 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:41.668 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T19:21:41.668 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T19:21:41.918 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:21:42.302 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:42 smithi194 ceph-mon[32740]: pgmap v591: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:42.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:42 smithi086 ceph-mon[27783]: pgmap v591: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:42.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:42 smithi183 ceph-mon[32344]: pgmap v591: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2314, 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1953, 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 615950, 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1250784, 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 10, 2024-04-25T19:21:43.922 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.191010515, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.081404651, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.110686572, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.008468511, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1952, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.095998806, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000049179 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3828, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001041985, 2024-04-25T19:21:43.923 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000272 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 9424, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 512448 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 484, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 475, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1705047, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 329738, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 18, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.082732226, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.027503734, 2024-04-25T19:21:43.924 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.053948452, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.010592258, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 474, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.061726049, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000130223 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 41, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000006452, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000157 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1701680, 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 248640 2024-04-25T19:21:43.925 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 50, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 177, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 12941, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 204380, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 12, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.012976137, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.005351346, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.005803748, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000549352, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 174, 2024-04-25T19:21:43.926 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.016805588, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000096583 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 105, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000022525, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000214 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 7984, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 195584 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-25T19:21:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 74, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001676060, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000022649 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:43.928 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000016015, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000008007 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 14, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 9, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 9, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.004386016, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000487335 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-25T19:21:43.929 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 5, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 11, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 17, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 1, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 16, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 13, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 5, 2024-04-25T19:21:43.930 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 2, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 2, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 3, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 59, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 48, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 2, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 6, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-25T19:21:43.931 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 234, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:43.932 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 6, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 6, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 17, 2024-04-25T19:21:43.933 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 11, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 13, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:43.934 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 14, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 86, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.935 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 12, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 12, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 2, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-25T19:21:43.936 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4197721, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 11, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.067128260, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006102569 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-25T19:21:43.937 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 21, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 23, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 2, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 20, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 24, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 4, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 17, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 22, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 5, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 2, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 6, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 4, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 56004, 2024-04-25T19:21:43.938 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223608 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 18, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 14, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 45, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 87, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.939 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.003280270, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000656054 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.940 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000980307, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000326769 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1, 2024-04-25T19:21:43.941 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000125439, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000125439 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.943 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.944 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.945 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:43.946 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.947 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.948 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-25T19:21:43.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 6, 2024-04-25T19:21:43.951 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 4, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 6, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 1, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1107, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:43.952 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:43.953 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:43.954 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 279911, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 93, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:43.955 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 176, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 90085, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 364, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-25T19:21:43.956 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 52, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 14250, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 30, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 52, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 52, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.448990885, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.008634440 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 52, 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 77 2024-04-25T19:21:43.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 52, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 0, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 52, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 0, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 11, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 9, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-25T19:21:43.958 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 6, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-25T19:21:43.959 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-25T19:21:43.960 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 120, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 31, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 10, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 15, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 5, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 20, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2024-04-25T19:21:43.961 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 1 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 99 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-25T19:21:43.962 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 2844, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2049021, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 2844, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:43.963 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2844, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2049021, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:43.964 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 52, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 14250, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 52, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 14250, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.965 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 52, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 52, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 52, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 52, 2024-04-25T19:21:43.966 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:43.967 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x5645b1356500": { 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 12, 2024-04-25T19:21:43.968 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 3417, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 12, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 11, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3417, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:43.969 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-25T19:21:44.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:44 smithi183 ceph-mon[32344]: pgmap v592: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:44.710 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:44 smithi194 ceph-mon[32740]: pgmap v592: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:44.737 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T19:21:44.737 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T19:21:44.737 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T19:21:44.737 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=17 2024-04-25T19:21:44.737 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-25T19:21:44.737 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:44.738 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:44.738 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T19:21:44.738 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T19:21:44.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:44 smithi086 ceph-mon[27783]: pgmap v592: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:44.988 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:21:46.432 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:46 smithi194 ceph-mon[32740]: pgmap v593: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:46.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:46 smithi183 ceph-mon[32344]: pgmap v593: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:46.699 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 3981, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 3983, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1124593, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1265965, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.358624774, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.121384824, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.084888234, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.124440074, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3981, 2024-04-25T19:21:46.700 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.165448811, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000041559 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 9, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 292, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 4076, 2024-04-25T19:21:46.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 748954, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 8, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.017107826, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.011631577, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.002452489, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000053951, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 290, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.031128876, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000107340 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:46.702 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 748256 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 425, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 412, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1685756, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 239182, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.075053279, 2024-04-25T19:21:46.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.026545051, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.045275874, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.009780144, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 411, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.023617123, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000057462 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1685488, 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 238864 2024-04-25T19:21:46.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6690, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.202003717, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000030194 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:46.705 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:46.707 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 1, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-25T19:21:46.708 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:46.709 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-04-25T19:21:46.710 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-25T19:21:46.711 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-25T19:21:46.712 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 4, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4195424, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 4195283, 2024-04-25T19:21:46.713 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 4 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 3, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 3, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 0, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 1, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 1, 2024-04-25T19:21:46.714 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 49544, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215416 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-04-25T19:21:46.715 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.716 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.717 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:46.718 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.719 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.720 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.721 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.722 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.723 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:46.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.726 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 0, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 1, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 1, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2024-04-25T19:21:46.727 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1586, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:46.728 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:46.729 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 182865, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 50, 2024-04-25T19:21:46.730 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 176, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 8512, 2024-04-25T19:21:46.731 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 13, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 3984, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 3984, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3984, 2024-04-25T19:21:46.732 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.836962673, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000461085 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3984, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 3985 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 3984, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 3984, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 2622, 2024-04-25T19:21:46.733 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1360, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2024-04-25T19:21:46.734 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-25T19:21:46.735 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 120, 2024-04-25T19:21:46.736 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 31, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 4, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 6, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 2, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-25T19:21:46.737 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.738 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 4411, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2392632, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 4411, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 4411, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2392632, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.739 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3984, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1125, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3984, 2024-04-25T19:21:46.740 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1125, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:46.741 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3984, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 3984, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3984, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3984, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:46.742 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:46.743 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55f40fe5a6e0": { 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:46.744 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:46.745 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:46.745 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.745 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:46.745 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-25T19:21:46.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:46 smithi086 ceph-mon[27783]: pgmap v593: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:47.564 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T19:21:47.564 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T19:21:47.564 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T19:21:47.564 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T19:21:47.564 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T19:21:47.564 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:47.565 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:47.565 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T19:21:47.565 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T19:21:47.804 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:21:48.620 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:48 smithi194 ceph-mon[32740]: pgmap v594: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:48.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:48 smithi183 ceph-mon[32344]: pgmap v594: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:48.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:48 smithi086 ceph-mon[27783]: pgmap v594: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T19:21:49.502 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2288, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1652, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 604922, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 378545, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 10, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.165215834, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.066931234, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.102370397, 2024-04-25T19:21:49.503 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.008450288, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1650, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.085815169, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000052009 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3750, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000851664, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000227 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.504 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 23, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 308, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 7739, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 747303, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 12, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.020263619, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.012201757, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.004465459, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000289765, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 305, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.048443092, 2024-04-25T19:21:49.505 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000158829 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 9, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000001678, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000186 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 4448, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 741632 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 476, 2024-04-25T19:21:49.506 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 465, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1709492, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 304377, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 12, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.081484776, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.026618534, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.116362208, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.010236970, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 464, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.046134024, 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000099426 2024-04-25T19:21:49.507 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 36, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000005274, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000146 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1707312, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 248288 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-25T19:21:49.508 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 63, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001809448, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000028721 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000013434, 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000006717 2024-04-25T19:21:49.509 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 4, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 3, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000971263, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000323754 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 1, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:49.510 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 11, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 17, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 1, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 16, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 13, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 4, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 3, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 45, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 40, 2024-04-25T19:21:49.511 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 5, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:49.512 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 234, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 3, 2024-04-25T19:21:49.513 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 3, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 17, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 11, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 12, 2024-04-25T19:21:49.514 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 14, 2024-04-25T19:21:49.515 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 86, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 8, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 8, 2024-04-25T19:21:49.516 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 2, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4196488, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 7, 2024-04-25T19:21:49.517 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.020003477, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002857639 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 21, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 23, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 2, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 20, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 24, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 4, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 17, 2024-04-25T19:21:49.518 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 21, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 4, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 3, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 3, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 55744, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223608 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 4, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 4, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 12, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 86, 2024-04-25T19:21:49.519 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000351524, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000351524 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.520 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000619739, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000309869 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.521 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.522 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-25T19:21:49.523 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.524 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.525 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.528 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.529 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.530 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 0, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 3, 2024-04-25T19:21:49.531 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 3, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 2, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 85, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:49.532 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:49.533 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:49.534 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 247879, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 80, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:49.535 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 176, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 89349, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 360, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.536 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 45, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 12711, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 28, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 45, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 45, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.436991116, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.009710913 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-25T19:21:49.537 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 45, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 69 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 45, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 0, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 45, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 0, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 7, 2024-04-25T19:21:49.538 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 7, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 6, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-25T19:21:49.539 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-25T19:21:49.540 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 120, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 31, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 8, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 11, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 3, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-25T19:21:49.541 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 17, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 2 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 99 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.542 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 2783, 2024-04-25T19:21:49.543 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2044805, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 2783, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 2783, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2044805, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:49.544 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 45, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 12711, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 45, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 12711, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.545 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 45, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 45, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 45, 2024-04-25T19:21:49.546 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 45, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:49.547 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55a962f8e500": { 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:49.548 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 8, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2184, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 8, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 7, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2184, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=17 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:50.460 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:50.460 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T19:21:50.460 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T19:21:50.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:50 smithi183 ceph-mon[32344]: pgmap v595: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:50.696 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:21:50.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:50 smithi194 ceph-mon[32740]: pgmap v595: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:50.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:50 smithi086 ceph-mon[27783]: pgmap v595: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:52.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:52 smithi194 ceph-mon[32740]: pgmap v596: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1363, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1363, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 459609, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 432611, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.176851600, 2024-04-25T19:21:52.405 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.054552747, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.046042592, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.057771909, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1362, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.088489201, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000064970 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.406 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2640, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2930, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 674477, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1501421, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.210975706, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.078455744, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.047315552, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.067184469, 2024-04-25T19:21:52.407 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2927, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.142528529, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000048694 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 662480 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 434, 2024-04-25T19:21:52.408 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 421, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1689364, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 244005, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 5, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.076339695, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.026934730, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.045373380, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.010058903, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 420, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.023820953, 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000056716 2024-04-25T19:21:52.409 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1688016, 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 243040 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.410 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6720, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.195291273, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000029061 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-04-25T19:21:52.411 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.412 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-04-25T19:21:52.413 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 1, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-04-25T19:21:52.414 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-04-25T19:21:52.415 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-04-25T19:21:52.415 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:21:52.415 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-04-25T19:21:52.415 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2024-04-25T19:21:52.415 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2024-04-25T19:21:52.415 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-04-25T19:21:52.417 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-04-25T19:21:52.418 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-04-25T19:21:52.419 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-04-25T19:21:52.420 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-04-25T19:21:52.421 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 12, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-04-25T19:21:52.422 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4197721, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 4197580, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 12 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 3, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 3, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 0, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 1, 2024-04-25T19:21:52.423 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 1, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 53192, 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215416 2024-04-25T19:21:52.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.425 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-04-25T19:21:52.426 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.427 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-04-25T19:21:52.428 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-04-25T19:21:52.429 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-04-25T19:21:52.430 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.431 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.432 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.433 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.434 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.434 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.434 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-04-25T19:21:52.434 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.434 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.435 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.436 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.438 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 6, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 4, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 5, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 5458, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-04-25T19:21:52.439 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:21:52.440 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-04-25T19:21:52.441 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 199025, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 58, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-04-25T19:21:52.442 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8424, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 176, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 8512, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 13, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-04-25T19:21:52.443 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 4000, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 4000, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4000, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2.626946558, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000656736 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4000, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 4001 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.444 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 4000, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 4000, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 2630, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1368, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-04-25T19:21:52.445 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:21:52.446 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-04-25T19:21:52.447 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 120, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 31, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 6, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 8, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 2, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.448 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-04-25T19:21:52.449 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-04-25T19:21:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-04-25T19:21:52.451 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-04-25T19:21:52.451 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-04-25T19:21:52.451 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.451 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:21:52.451 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:52.451 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:52.452 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:52.452 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 4433, 2024-04-25T19:21:52.452 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2399369, 2024-04-25T19:21:52.452 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:52.452 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 4433, 2024-04-25T19:21:52.452 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 4433, 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2399369, 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.453 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:52.454 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 4000, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 3422, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 4000, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3422, 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:52.455 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.456 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.456 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.456 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.456 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.456 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:52.457 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 4000, 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 4000, 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 4000, 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 4000, 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.458 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:52.459 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x5562379086e0": { 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-04-25T19:21:52.460 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-25T19:21:52.461 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-25T19:21:52.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:52 smithi183 ceph-mon[32344]: pgmap v596: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:52.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:52 smithi086 ceph-mon[27783]: pgmap v596: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:21:53.213 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:21:53.214 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T19:21:53.225 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T19:21:53.225 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T19:21:53.225 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:21:53.286 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T19:21:53.286 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:21:53.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-25T19:21:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:54.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:54 smithi086 ceph-mon[27783]: pgmap v597: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:54.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:54 smithi183 ceph-mon[32344]: pgmap v597: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:54.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:54 smithi194 ceph-mon[32740]: pgmap v597: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T19:21:55.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 -- 172.21.15.86:0/910085894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694109fc0 msgr2=0x7f4694111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 --2- 172.21.15.86:0/910085894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694109fc0 0x7f4694111b90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b0d0 tx=0x7f468c00b3e0 comp rx=0 tx=0).stop 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 -- 172.21.15.86:0/910085894 shutdown_connections 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 --2- 172.21.15.86:0/910085894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694109fc0 0x7f4694111b90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 --2- 172.21.15.86:0/910085894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4694104ee0 0x7f46941052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 --2- 172.21.15.86:0/910085894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694105890 0x7f46941098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 -- 172.21.15.86:0/910085894 >> 172.21.15.86:0/910085894 conn(0x7f4694100770 msgr2=0x7f4694102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 -- 172.21.15.86:0/910085894 shutdown_connections 2024-04-25T19:21:55.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.993+0000 7f469c000700 1 -- 172.21.15.86:0/910085894 wait complete. 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.994+0000 7f469c000700 1 Processor -- start 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 -- start start 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 0x7f469419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694105890 0x7f469419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4694109fc0 0x7f469419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4694114e60 con 0x7f4694104ee0 2024-04-25T19:21:55.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4694114ce0 con 0x7f4694105890 2024-04-25T19:21:55.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f469c000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4694114fe0 con 0x7f4694109fc0 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.995+0000 7f4699600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 0x7f469419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 0x7f469419b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53138/0 (socket says 172.21.15.86:53138) 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 -- 172.21.15.86:0/726183782 learned_addr learned my addr 172.21.15.86:0/726183782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4698c00700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694105890 0x7f469419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f469a000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4694109fc0 0x7f469419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 -- 172.21.15.86:0/726183782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4694109fc0 msgr2=0x7f469419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4694109fc0 0x7f469419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 -- 172.21.15.86:0/726183782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694105890 msgr2=0x7f469419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:55.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694105890 0x7f469419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 -- 172.21.15.86:0/726183782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46941a02e0 con 0x7f4694104ee0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.996+0000 7f4699600700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 0x7f469419b2d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f468800e9a0 tx=0x7f468800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.997+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468800ca10 con 0x7f4694104ee0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.997+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46941a05d0 con 0x7f4694104ee0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.997+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f468800cbb0 con 0x7f4694104ee0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.997+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688018680 con 0x7f4694104ee0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.997+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a7e50 con 0x7f4694104ee0 2024-04-25T19:21:55.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:55.998+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4694069730 con 0x7f4694104ee0 2024-04-25T19:21:56.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.000+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4688010430 con 0x7f4694104ee0 2024-04-25T19:21:56.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.002+0000 7f4692000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f467c0a3db0 0x7f467c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:21:56.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.003+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8637+0+0 (secure 0 0 0) 0x7f46880d0a60 con 0x7f4694104ee0 2024-04-25T19:21:56.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.003+0000 7f4698c00700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f467c0a3db0 0x7f467c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:21:56.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.004+0000 7f4698c00700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f467c0a3db0 0x7f467c0a6270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4680009680 tx=0x7f4680009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:21:56.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.007+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f468809d030 con 0x7f4694104ee0 2024-04-25T19:21:56.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.252+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f469402df80 con 0x7f4694104ee0 2024-04-25T19:21:56.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.253+0000 7f4692000700 1 -- 172.21.15.86:0/726183782 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v120) ==== 81+0+938 (secure 0 0 0) 0x7f468809c930 con 0x7f4694104ee0 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2093097457 2024-04-26T19:02:56.815518+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1310908896 2024-04-26T19:02:56.815518+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1288701391 2024-04-26T19:02:56.815518+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/237697797 2024-04-26T19:02:56.815518+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1094517747 2024-04-26T18:50:37.175300+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/324236586 2024-04-26T19:02:56.815518+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/378785417 2024-04-26T18:50:37.175300+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/4264123395 2024-04-26T18:50:37.175300+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3778539307 2024-04-26T18:50:04.542688+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/73406505 2024-04-26T18:50:04.542688+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/237697797 2024-04-26T19:02:56.815518+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/777604946 2024-04-26T18:50:04.542688+0000 2024-04-25T19:21:56.255 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/777604946 2024-04-26T18:50:04.542688+0000 2024-04-25T19:21:56.256 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2055127838 2024-04-26T18:50:37.175300+0000 2024-04-25T19:21:56.256 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3781450984 2024-04-26T18:50:04.542688+0000 2024-04-25T19:21:56.256 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/4264123395 2024-04-26T18:50:37.175300+0000 2024-04-25T19:21:56.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f467c0a3db0 msgr2=0x7f467c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:56.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f467c0a3db0 0x7f467c0a6270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4680009680 tx=0x7f4680009260 comp rx=0 tx=0).stop 2024-04-25T19:21:56.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 msgr2=0x7f469419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 0x7f469419b2d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f468800e9a0 tx=0x7f468800ecb0 comp rx=0 tx=0).stop 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 shutdown_connections 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f467c0a3db0 0x7f467c0a6270 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4694109fc0 0x7f469419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694104ee0 0x7f469419b2d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 --2- 172.21.15.86:0/726183782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694105890 0x7f469419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 >> 172.21.15.86:0/726183782 conn(0x7f4694100770 msgr2=0x7f46941012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 shutdown_connections 2024-04-25T19:21:56.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:21:56.257+0000 7f469c000700 1 -- 172.21.15.86:0/726183782 wait complete. 2024-04-25T19:21:56.260 INFO:teuthology.orchestra.run.smithi086.stderr:listed 16 entries 2024-04-25T19:21:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:56 smithi086 ceph-mon[27783]: pgmap v598: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:56.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:56 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/726183782' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T19:21:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:56 smithi183 ceph-mon[32344]: pgmap v598: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:56.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:56 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/726183782' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T19:21:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:56 smithi194 ceph-mon[32740]: pgmap v598: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:56.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:56 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/726183782' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T19:21:57.064 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T19:21:57.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:21:57.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T19:21:57.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:21:57.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:21:57.193 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T19:21:57.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:21:57.251 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T19:21:57.289 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T19:21:57.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:21:57.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:21:57.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:21:57.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T19:21:57.524 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T19:21:57.535 INFO:tasks.cephadm:Teardown begin 2024-04-25T19:21:57.535 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T19:21:57.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:21:57.948 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:21:58.666 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:21:58 smithi183 ceph-mon[32344]: pgmap v599: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:58.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:21:58 smithi194 ceph-mon[32740]: pgmap v599: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:21:58.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:21:58 smithi086 ceph-mon[27783]: pgmap v599: 97 pgs: 97 active+clean; 1.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T19:22:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 -- 172.21.15.86:0/2592093691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406b070 msgr2=0x7f0bb410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 --2- 172.21.15.86:0/2592093691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406b070 0x7f0bb410efb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009990 tx=0x7f0ba0009ca0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 -- 172.21.15.86:0/2592093691 shutdown_connections 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 --2- 172.21.15.86:0/2592093691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb410f4f0 0x7f0bb41118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 --2- 172.21.15.86:0/2592093691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb406a750 0x7f0bb406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 --2- 172.21.15.86:0/2592093691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406b070 0x7f0bb410efb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 -- 172.21.15.86:0/2592093691 >> 172.21.15.86:0/2592093691 conn(0x7f0bb40788f0 msgr2=0x7f0bb4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 -- 172.21.15.86:0/2592093691 shutdown_connections 2024-04-25T19:22:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.197+0000 7f0bbc200700 1 -- 172.21.15.86:0/2592093691 wait complete. 2024-04-25T19:22:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 Processor -- start 2024-04-25T19:22:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 -- start start 2024-04-25T19:22:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 0x7f0bb419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb406b070 0x7f0bb419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb410f4f0 0x7f0bb419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4114e50 con 0x7f0bb406b070 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4114cd0 con 0x7f0bb410f4f0 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.199+0000 7f0bbc200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4114fd0 con 0x7f0bb406a750 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.200+0000 7f0bb9800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 0x7f0bb419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.200+0000 7f0bb8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb406b070 0x7f0bb419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.200+0000 7f0bb9800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 0x7f0bb419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47068/0 (socket says 172.21.15.86:47068) 2024-04-25T19:22:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.200+0000 7f0bb9800700 1 -- 172.21.15.86:0/458891634 learned_addr learned my addr 172.21.15.86:0/458891634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.200+0000 7f0bba200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb410f4f0 0x7f0bb419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.200+0000 7f0bb9800700 1 -- 172.21.15.86:0/458891634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb410f4f0 msgr2=0x7f0bb419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb9800700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb410f4f0 0x7f0bb419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb9800700 1 -- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb406b070 msgr2=0x7f0bb419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb9800700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb406b070 0x7f0bb419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb9800700 1 -- 172.21.15.86:0/458891634 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb41a0370 con 0x7f0bb406a750 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb8e00700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb406b070 0x7f0bb419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bba200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb410f4f0 0x7f0bb419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:22:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb9800700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 0x7f0bb419b360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800ef10 tx=0x7f0ba800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:00.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800cad0 con 0x7f0bb406a750 2024-04-25T19:22:00.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba8019070 con 0x7f0bb406a750 2024-04-25T19:22:00.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.201+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80146e0 con 0x7f0bb406a750 2024-04-25T19:22:00.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.202+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb41a0600 con 0x7f0bb406a750 2024-04-25T19:22:00.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.202+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb41a7ee0 con 0x7f0bb406a750 2024-04-25T19:22:00.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.203+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb4069730 con 0x7f0bb406a750 2024-04-25T19:22:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.206+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ba8012050 con 0x7f0bb406a750 2024-04-25T19:22:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.207+0000 7f0bb2000700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0b9c0a3f00 0x7f0b9c0a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.208+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8637+0+0 (secure 0 0 0) 0x7f0ba80d0b70 con 0x7f0bb406a750 2024-04-25T19:22:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.208+0000 7f0bb8e00700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0b9c0a3f00 0x7f0b9c0a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.209+0000 7f0bb8e00700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0b9c0a3f00 0x7f0b9c0a63c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0006010 tx=0x7f0ba001a040 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.210+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ba8010040 con 0x7f0bb406a750 2024-04-25T19:22:00.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.503+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bb402df80 con 0x7f0bb406a750 2024-04-25T19:22:00.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:00 smithi086 ceph-mon[27783]: pgmap v600: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-25T19:22:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.504+0000 7f0bb2000700 1 -- 172.21.15.86:0/458891634 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7f0ba809d1c0 con 0x7f0bb406a750 2024-04-25T19:22:00.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:22:00.505 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:22:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0b9c0a3f00 msgr2=0x7f0b9c0a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0b9c0a3f00 0x7f0b9c0a63c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0006010 tx=0x7f0ba001a040 comp rx=0 tx=0).stop 2024-04-25T19:22:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 msgr2=0x7f0bb419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 0x7f0bb419b360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800ef10 tx=0x7f0ba800c3a0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 shutdown_connections 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bb410f4f0 0x7f0bb419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0b9c0a3f00 0x7f0b9c0a63c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bb406a750 0x7f0bb419b360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 --2- 172.21.15.86:0/458891634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb406b070 0x7f0bb419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 >> 172.21.15.86:0/458891634 conn(0x7f0bb40788f0 msgr2=0x7f0bb41109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 shutdown_connections 2024-04-25T19:22:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:00.507+0000 7f0bbc200700 1 -- 172.21.15.86:0/458891634 wait complete. 2024-04-25T19:22:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:22:00.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:00 smithi194 ceph-mon[32740]: pgmap v600: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-25T19:22:00.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:00 smithi183 ceph-mon[32344]: pgmap v600: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-25T19:22:01.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:22:01.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:01 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/458891634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:22:01.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:01 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/458891634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:22:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:22:01.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:01 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/458891634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:22:02.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:02 smithi086 ceph-mon[27783]: pgmap v601: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T19:22:02.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:02 smithi194 ceph-mon[32740]: pgmap v601: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T19:22:02.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:02 smithi183 ceph-mon[32344]: pgmap v601: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T19:22:04.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 -- 172.21.15.86:0/1632397509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe874075590 msgr2=0x7fe87410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:04.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 --2- 172.21.15.86:0/1632397509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe874075590 0x7fe87410ef00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009990 tx=0x7fe85c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 -- 172.21.15.86:0/1632397509 shutdown_connections 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 --2- 172.21.15.86:0/1632397509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe87410f440 0x7fe874111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 --2- 172.21.15.86:0/1632397509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe874074c70 0x7fe874075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 --2- 172.21.15.86:0/1632397509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe874075590 0x7fe87410ef00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.085+0000 7fe879a00700 1 -- 172.21.15.86:0/1632397509 >> 172.21.15.86:0/1632397509 conn(0x7fe874079ba0 msgr2=0x7fe874079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.086+0000 7fe879a00700 1 -- 172.21.15.86:0/1632397509 shutdown_connections 2024-04-25T19:22:04.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.086+0000 7fe879a00700 1 -- 172.21.15.86:0/1632397509 wait complete. 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.086+0000 7fe879a00700 1 Processor -- start 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 -- start start 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 0x7fe87419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe874075590 0x7fe87419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe87410f440 0x7fe87419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe874114e50 con 0x7fe87410f440 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe874114cd0 con 0x7fe874075590 2024-04-25T19:22:04.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe879a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe874114fd0 con 0x7fe874074c70 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe873400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe87410f440 0x7fe87419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe872a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 0x7fe87419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe873400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe87410f440 0x7fe87419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43928/0 (socket says 172.21.15.86:43928) 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe873400700 1 -- 172.21.15.86:0/4113387818 learned_addr learned my addr 172.21.15.86:0/4113387818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe872a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 0x7fe87419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:47082/0 (socket says 172.21.15.86:47082) 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.087+0000 7fe86a000700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe874075590 0x7fe87419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.088+0000 7fe872a00700 1 -- 172.21.15.86:0/4113387818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe874075590 msgr2=0x7fe87419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.088+0000 7fe872a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe874075590 0x7fe87419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.088+0000 7fe872a00700 1 -- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe87410f440 msgr2=0x7fe87419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.088+0000 7fe872a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe87410f440 0x7fe87419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.088+0000 7fe872a00700 1 -- 172.21.15.86:0/4113387818 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8741a0320 con 0x7fe874074c70 2024-04-25T19:22:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.088+0000 7fe872a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 0x7fe87419b360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d750 tx=0x7fe86400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:04.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.089+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864014030 con 0x7fe874074c70 2024-04-25T19:22:04.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.089+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a0610 con 0x7fe874074c70 2024-04-25T19:22:04.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.089+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8640044d0 con 0x7fe874074c70 2024-04-25T19:22:04.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.089+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a7e90 con 0x7fe874074c70 2024-04-25T19:22:04.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.089+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86401d060 con 0x7fe874074c70 2024-04-25T19:22:04.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.090+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe874069730 con 0x7fe874074c70 2024-04-25T19:22:04.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.092+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8640049e0 con 0x7fe874074c70 2024-04-25T19:22:04.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.093+0000 7fe86be00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe8580a3e30 0x7fe8580a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:04.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.094+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8637+0+0 (secure 0 0 0) 0x7fe8640d04e0 con 0x7fe874074c70 2024-04-25T19:22:04.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.094+0000 7fe86a000700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe8580a3e30 0x7fe8580a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:04.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.095+0000 7fe86a000700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe8580a3e30 0x7fe8580a62f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe874111090 tx=0x7fe85c00b540 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:04.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.096+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe864004750 con 0x7fe874074c70 2024-04-25T19:22:04.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.370+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87402e140 con 0x7fe874074c70 2024-04-25T19:22:04.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.371+0000 7fe86be00700 1 -- 172.21.15.86:0/4113387818 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11937 (secure 0 0 0) 0x7fe86409cab0 con 0x7fe874074c70 2024-04-25T19:22:04.373 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:22:04.374 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3780909923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3780909923},{"type":"v1","addr":"172.21.15.86:6835","nonce":3780909923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/2332559651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":2332559651},{"type":"v1","addr":"172.21.15.86:6841","nonce":2332559651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:58:51.446663+0000","modified":"2024-04-25T19:02:56.674071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24509,"mds_2":24494,"mds_3":24493,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24430":{"gid":24430,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/45783241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":45783241},{"type":"v1","addr":"172.21.15.194:6833","nonce":45783241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2135214296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2135214296},{"type":"v1","addr":"172.21.15.183:6833","nonce":2135214296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/3883478130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3883478130},{"type":"v1","addr":"172.21.15.86:6837","nonce":3883478130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/425079996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":425079996},{"type":"v1","addr":"172.21.15.194:6835","nonce":425079996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4174239962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4174239962},{"type":"v1","addr":"172.21.15.183:6835","nonce":4174239962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3328875881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3328875881},{"type":"v1","addr":"172.21.15.194:6837","nonce":3328875881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/2085744336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2085744336},{"type":"v1","addr":"172.21.15.194:6839","nonce":2085744336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.183:6837/3273542947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3273542947},{"type":"v1","addr":"172.21.15.183:6837","nonce":3273542947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.86:6839/3676043902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3676043902},{"type":"v1","addr":"172.21.15.86:6839","nonce":3676043902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6839/703487979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":703487979},{"type":"v1","addr":"172.21.15.183:6839","nonce":703487979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24509,24494,24493,24478,24527]},"id":1}]} 2024-04-25T19:22:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.374+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe8580a3e30 msgr2=0x7fe8580a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.374+0000 7fe879a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe8580a3e30 0x7fe8580a62f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe874111090 tx=0x7fe85c00b540 comp rx=0 tx=0).stop 2024-04-25T19:22:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.374+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 msgr2=0x7fe87419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 0x7fe87419b360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d750 tx=0x7fe86400da60 comp rx=0 tx=0).stop 2024-04-25T19:22:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 shutdown_connections 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe87410f440 0x7fe87419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fe8580a3e30 0x7fe8580a62f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe874074c70 0x7fe87419b360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 --2- 172.21.15.86:0/4113387818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe874075590 0x7fe87419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 >> 172.21.15.86:0/4113387818 conn(0x7fe874079ba0 msgr2=0x7fe8740ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 shutdown_connections 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:04.375+0000 7fe879a00700 1 -- 172.21.15.86:0/4113387818 wait complete. 2024-04-25T19:22:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T19:22:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:04 smithi194 ceph-mon[32740]: pgmap v602: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-25T19:22:04.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:04 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/4113387818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:22:04.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:04 smithi086 ceph-mon[27783]: pgmap v602: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-25T19:22:04.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:04 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/4113387818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:22:04.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:04 smithi183 ceph-mon[32344]: pgmap v602: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-25T19:22:04.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:04 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/4113387818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:22:05.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T19:22:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:22:06.718 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:06 smithi183 ceph-mon[32344]: pgmap v603: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-25T19:22:06.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:06 smithi194 ceph-mon[32740]: pgmap v603: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-25T19:22:06.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:06 smithi086 ceph-mon[27783]: pgmap v603: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-25T19:22:07.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:07 smithi086 ceph-mon[27783]: osdmap e121: 12 total, 12 up, 12 in 2024-04-25T19:22:07.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:07 smithi194 ceph-mon[32740]: osdmap e121: 12 total, 12 up, 12 in 2024-04-25T19:22:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.792+0000 7f815c400700 1 -- 172.21.15.86:0/2632599195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815410f440 msgr2=0x7f8154111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 --2- 172.21.15.86:0/2632599195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815410f440 0x7f8154111830 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f814c009990 tx=0x7f814c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 -- 172.21.15.86:0/2632599195 shutdown_connections 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 --2- 172.21.15.86:0/2632599195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815410f440 0x7f8154111830 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 --2- 172.21.15.86:0/2632599195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8154074c70 0x7f8154075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 --2- 172.21.15.86:0/2632599195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8154075590 0x7f815410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 -- 172.21.15.86:0/2632599195 >> 172.21.15.86:0/2632599195 conn(0x7f8154079ba0 msgr2=0x7f8154079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 -- 172.21.15.86:0/2632599195 shutdown_connections 2024-04-25T19:22:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.793+0000 7f815c400700 1 -- 172.21.15.86:0/2632599195 wait complete. 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 Processor -- start 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 -- start start 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8154074c70 0x7f815419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 0x7f815419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815410f440 0x7f815419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8154114e20 con 0x7f815410f440 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8154114ca0 con 0x7f8154075590 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.794+0000 7f815c400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8154114fa0 con 0x7f8154074c70 2024-04-25T19:22:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 0x7f815419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8154074c70 0x7f815419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 0x7f815419b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:56960/0 (socket says 172.21.15.86:56960) 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 -- 172.21.15.86:0/4167071170 learned_addr learned my addr 172.21.15.86:0/4167071170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f815a400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815410f440 0x7f815419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 -- 172.21.15.86:0/4167071170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8154074c70 msgr2=0x7f815419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8154074c70 0x7f815419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 -- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815410f440 msgr2=0x7f815419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815410f440 0x7f815419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 -- 172.21.15.86:0/4167071170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81541a0270 con 0x7f8154075590 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.795+0000 7f8159000700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 0x7f815419b7f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f814000d980 tx=0x7f814000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.796+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140017030 con 0x7f8154075590 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.796+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81400044d0 con 0x7f8154075590 2024-04-25T19:22:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.796+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81541a0560 con 0x7f8154075590 2024-04-25T19:22:07.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.796+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400039e0 con 0x7f8154075590 2024-04-25T19:22:07.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.796+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81541a7de0 con 0x7f8154075590 2024-04-25T19:22:07.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.797+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8154069730 con 0x7f8154075590 2024-04-25T19:22:07.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.799+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f814001a070 con 0x7f8154075590 2024-04-25T19:22:07.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.800+0000 7f8152000700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f813c0a3db0 0x7f813c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:07.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.800+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8681+0+0 (secure 0 0 0) 0x7f81400d0230 con 0x7f8154075590 2024-04-25T19:22:07.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.800+0000 7f8159a00700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f813c0a3db0 0x7f813c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:07.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.801+0000 7f8159a00700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f813c0a3db0 0x7f813c0a6270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8148009550 tx=0x7f81480091b0 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:07.804+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81400232f0 con 0x7f8154075590 2024-04-25T19:22:07.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:07 smithi183 ceph-mon[32344]: osdmap e121: 12 total, 12 up, 12 in 2024-04-25T19:22:08.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.034+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f815402df80 con 0x7f8154075590 2024-04-25T19:22:08.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.037+0000 7f8152000700 1 -- 172.21.15.86:0/4167071170 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+23404 (secure 0 0 0) 0x7f814009c830 con 0x7f8154075590 2024-04-25T19:22:08.039 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T19:22:08.039 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":121,"fsid":"1ec2eb56-0334-11ef-bc93-c7b262605968","created":"2024-04-25T18:49:23.294231+0000","modified":"2024-04-25T19:22:06.440371+0000","last_up_change":"2024-04-25T18:57:35.761081+0000","last_in_change":"2024-04-25T18:57:15.353854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:53:19.337576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:58:42.545870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:58:47.510628+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"983ae9a3-f0a7-436b-996b-e9916864d30e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6803","nonce":2684261173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6805","nonce":2684261173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6809","nonce":2684261173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2684261173},{"type":"v1","addr":"172.21.15.86:6807","nonce":2684261173}]},"public_addr":"172.21.15.86:6803/2684261173","cluster_addr":"172.21.15.86:6805/2684261173","heartbeat_back_addr":"172.21.15.86:6809/2684261173","heartbeat_front_addr":"172.21.15.86:6807/2684261173","state":["exists","up"]},{"osd":1,"uuid":"cf82beae-9a79-46f2-9ea8-59032823bc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6801","nonce":4147786843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6803","nonce":4147786843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6807","nonce":4147786843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4147786843},{"type":"v1","addr":"172.21.15.183:6805","nonce":4147786843}]},"public_addr":"172.21.15.183:6801/4147786843","cluster_addr":"172.21.15.183:6803/4147786843","heartbeat_back_addr":"172.21.15.183:6807/4147786843","heartbeat_front_addr":"172.21.15.183:6805/4147786843","state":["exists","up"]},{"osd":2,"uuid":"768f0112-52c8-452e-b56c-1d19c065d486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6801","nonce":1582539385}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6803","nonce":1582539385}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6807","nonce":1582539385}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1582539385},{"type":"v1","addr":"172.21.15.194:6805","nonce":1582539385}]},"public_addr":"172.21.15.194:6801/1582539385","cluster_addr":"172.21.15.194:6803/1582539385","heartbeat_back_addr":"172.21.15.194:6807/1582539385","heartbeat_front_addr":"172.21.15.194:6805/1582539385","state":["exists","up"]},{"osd":3,"uuid":"a91c123c-a363-482a-9d1a-c6eb6f121b33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6811","nonce":2203765843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6813","nonce":2203765843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6817","nonce":2203765843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2203765843},{"type":"v1","addr":"172.21.15.86:6815","nonce":2203765843}]},"public_addr":"172.21.15.86:6811/2203765843","cluster_addr":"172.21.15.86:6813/2203765843","heartbeat_back_addr":"172.21.15.86:6817/2203765843","heartbeat_front_addr":"172.21.15.86:6815/2203765843","state":["exists","up"]},{"osd":4,"uuid":"deb085ac-6e63-4429-a71a-e9e35e2c27f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6809","nonce":3004862386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6811","nonce":3004862386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6815","nonce":3004862386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3004862386},{"type":"v1","addr":"172.21.15.183:6813","nonce":3004862386}]},"public_addr":"172.21.15.183:6809/3004862386","cluster_addr":"172.21.15.183:6811/3004862386","heartbeat_back_addr":"172.21.15.183:6815/3004862386","heartbeat_front_addr":"172.21.15.183:6813/3004862386","state":["exists","up"]},{"osd":5,"uuid":"74a891b6-ac08-4b3a-bd2a-5b8d735a77de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6809","nonce":1967519671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6811","nonce":1967519671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6815","nonce":1967519671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1967519671},{"type":"v1","addr":"172.21.15.194:6813","nonce":1967519671}]},"public_addr":"172.21.15.194:6809/1967519671","cluster_addr":"172.21.15.194:6811/1967519671","heartbeat_back_addr":"172.21.15.194:6815/1967519671","heartbeat_front_addr":"172.21.15.194:6813/1967519671","state":["exists","up"]},{"osd":6,"uuid":"7b207644-8c34-4045-88b6-7cb304dcccec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6819","nonce":1838185370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6821","nonce":1838185370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6825","nonce":1838185370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1838185370},{"type":"v1","addr":"172.21.15.86:6823","nonce":1838185370}]},"public_addr":"172.21.15.86:6819/1838185370","cluster_addr":"172.21.15.86:6821/1838185370","heartbeat_back_addr":"172.21.15.86:6825/1838185370","heartbeat_front_addr":"172.21.15.86:6823/1838185370","state":["exists","up"]},{"osd":7,"uuid":"75d0252d-dd63-4aa7-982b-bd260501a0e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6817","nonce":2750054847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6819","nonce":2750054847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6823","nonce":2750054847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2750054847},{"type":"v1","addr":"172.21.15.183:6821","nonce":2750054847}]},"public_addr":"172.21.15.183:6817/2750054847","cluster_addr":"172.21.15.183:6819/2750054847","heartbeat_back_addr":"172.21.15.183:6823/2750054847","heartbeat_front_addr":"172.21.15.183:6821/2750054847","state":["exists","up"]},{"osd":8,"uuid":"dcd61ecf-016d-40fd-8a13-e4aa1828d49e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6817","nonce":1969375394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6819","nonce":1969375394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6823","nonce":1969375394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":1969375394},{"type":"v1","addr":"172.21.15.194:6821","nonce":1969375394}]},"public_addr":"172.21.15.194:6817/1969375394","cluster_addr":"172.21.15.194:6819/1969375394","heartbeat_back_addr":"172.21.15.194:6823/1969375394","heartbeat_front_addr":"172.21.15.194:6821/1969375394","state":["exists","up"]},{"osd":9,"uuid":"8006e686-e97b-4cda-a577-6b8dc8f9d5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6827","nonce":3110000530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6829","nonce":3110000530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6833","nonce":3110000530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3110000530},{"type":"v1","addr":"172.21.15.86:6831","nonce":3110000530}]},"public_addr":"172.21.15.86:6827/3110000530","cluster_addr":"172.21.15.86:6829/3110000530","heartbeat_back_addr":"172.21.15.86:6833/3110000530","heartbeat_front_addr":"172.21.15.86:6831/3110000530","state":["exists","up"]},{"osd":10,"uuid":"5f7367bb-2998-47a7-ae18-fd713be7d07e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6825","nonce":291756027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6827","nonce":291756027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6831","nonce":291756027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":291756027},{"type":"v1","addr":"172.21.15.183:6829","nonce":291756027}]},"public_addr":"172.21.15.183:6825/291756027","cluster_addr":"172.21.15.183:6827/291756027","heartbeat_back_addr":"172.21.15.183:6831/291756027","heartbeat_front_addr":"172.21.15.183:6829/291756027","state":["exists","up"]},{"osd":11,"uuid":"dc1df1f5-bf92-4d45-8c5d-e80fef77cccf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6825","nonce":745573568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6827","nonce":745573568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6831","nonce":745573568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":745573568},{"type":"v1","addr":"172.21.15.194:6829","nonce":745573568}]},"public_addr":"172.21.15.194:6825/745573568","cluster_addr":"172.21.15.194:6827/745573568","heartbeat_back_addr":"172.21.15.194:6831/745573568","heartbeat_front_addr":"172.21.15.194:6829/745573568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:25.430892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:50.759620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.191094+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:46.997341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:13.469284+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:39.062357+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:12.435229+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:55:39.473733+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:06.148880+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:56:39.778190+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:07.408062+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:57:33.567256+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2093097457":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1310908896":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1288701391":"2024-04-26T19:02:56.815518+0000","172.21.15.86:6800/237697797":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/1094517747":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/324236586":"2024-04-26T19:02:56.815518+0000","172.21.15.86:0/378785417":"2024-04-26T18:50:37.175300+0000","172.21.15.86:6801/4264123395":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3778539307":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/73406505":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6801/237697797":"2024-04-26T19:02:56.815518+0000","172.21.15.86:6801/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/777604946":"2024-04-26T18:50:04.542688+0000","172.21.15.86:0/2055127838":"2024-04-26T18:50:37.175300+0000","172.21.15.86:0/3781450984":"2024-04-26T18:50:04.542688+0000","172.21.15.86:6800/4264123395":"2024-04-26T18:50:37.175300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:22:08.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f813c0a3db0 msgr2=0x7f813c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:08.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f813c0a3db0 0x7f813c0a6270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8148009550 tx=0x7f81480091b0 comp rx=0 tx=0).stop 2024-04-25T19:22:08.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 msgr2=0x7f815419b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 0x7f815419b7f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f814000d980 tx=0x7f814000dc90 comp rx=0 tx=0).stop 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 shutdown_connections 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f815410f440 0x7f815419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f813c0a3db0 0x7f813c0a6270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8154074c70 0x7f815419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 --2- 172.21.15.86:0/4167071170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8154075590 0x7f815419b7f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 >> 172.21.15.86:0/4167071170 conn(0x7f8154079ba0 msgr2=0x7f81540ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 shutdown_connections 2024-04-25T19:22:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:08.040+0000 7f815c400700 1 -- 172.21.15.86:0/4167071170 wait complete. 2024-04-25T19:22:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:08 smithi194 ceph-mon[32740]: pgmap v605: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-25T19:22:08.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:08 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/4167071170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:22:08.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:08 smithi086 ceph-mon[27783]: pgmap v605: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-25T19:22:08.764 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:08 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/4167071170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:22:08.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-25T19:22:08.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:08 smithi183 ceph-mon[32344]: pgmap v605: 97 pgs: 97 active+clean; 1.7 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-25T19:22:08.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:08 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/4167071170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:22:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:22:10.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:10 smithi086 ceph-mon[27783]: pgmap v606: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T19:22:10.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:10 smithi194 ceph-mon[32740]: pgmap v606: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T19:22:10.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:10 smithi183 ceph-mon[32344]: pgmap v606: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T19:22:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 -- 172.21.15.86:0/2404496107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0010e3b0 msgr2=0x7f0d00115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 --2- 172.21.15.86:0/2404496107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0010e3b0 0x7f0d00115f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009fc0 tx=0x7f0cf800b330 comp rx=0 tx=0).stop 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 -- 172.21.15.86:0/2404496107 shutdown_connections 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 --2- 172.21.15.86:0/2404496107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0010e3b0 0x7f0d00115f80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 --2- 172.21.15.86:0/2404496107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d001092d0 0x7f0d001096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 --2- 172.21.15.86:0/2404496107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 0x7f0d0010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 -- 172.21.15.86:0/2404496107 >> 172.21.15.86:0/2404496107 conn(0x7f0d00076210 msgr2=0x7f0d00107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 -- 172.21.15.86:0/2404496107 shutdown_connections 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.471+0000 7f0d05800700 1 -- 172.21.15.86:0/2404496107 wait complete. 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.472+0000 7f0d05800700 1 Processor -- start 2024-04-25T19:22:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.472+0000 7f0d05800700 1 -- start start 2024-04-25T19:22:11.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0d05800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d001092d0 0x7f0d0019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:11.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0d05800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 0x7f0d0019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0d05800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 0x7f0d001a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0d05800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d001192a0 con 0x7f0d001092d0 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0d05800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00119120 con 0x7f0d00109c80 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0d05800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00119420 con 0x7f0d0010e3b0 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0cfea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d001092d0 0x7f0d0019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 0x7f0d0019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0cff400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 0x7f0d001a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0cff400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 0x7f0d001a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56138/0 (socket says 172.21.15.86:56138) 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 0x7f0d0019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.86:52694/0 (socket says 172.21.15.86:52694) 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.473+0000 7f0cff400700 1 -- 172.21.15.86:0/2962715955 learned_addr learned my addr 172.21.15.86:0/2962715955 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cff400700 1 -- 172.21.15.86:0/2962715955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 msgr2=0x7f0d0019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cff400700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 0x7f0d0019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cff400700 1 -- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d001092d0 msgr2=0x7f0d0019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cff400700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d001092d0 0x7f0d0019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cff400700 1 -- 172.21.15.86:0/2962715955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d001a46e0 con 0x7f0d0010e3b0 2024-04-25T19:22:11.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cfea00700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d001092d0 0x7f0d0019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:22:11.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.474+0000 7f0cff400700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 0x7f0d001a3fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009790 tx=0x7f0cf800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.478+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8021070 con 0x7f0d0010e3b0 2024-04-25T19:22:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.478+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf800ce60 con 0x7f0d0010e3b0 2024-04-25T19:22:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.478+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8005040 con 0x7f0d0010e3b0 2024-04-25T19:22:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.478+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d001a4970 con 0x7f0d0010e3b0 2024-04-25T19:22:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.478+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d001ac250 con 0x7f0d0010e3b0 2024-04-25T19:22:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.479+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d00069730 con 0x7f0d0010e3b0 2024-04-25T19:22:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.481+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0cf8024070 con 0x7f0d0010e3b0 2024-04-25T19:22:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.482+0000 7f0cf7400700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ce40a3e30 0x7f0ce40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.482+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8681+0+0 (secure 0 0 0) 0x7f0cf80d1760 con 0x7f0d0010e3b0 2024-04-25T19:22:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.482+0000 7f0cfea00700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ce40a3e30 0x7f0ce40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.483+0000 7f0cfea00700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ce40a3e30 0x7f0ce40a62f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009680 tx=0x7f0cf0009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.486+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cf809ddf0 con 0x7f0d0010e3b0 2024-04-25T19:22:11.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.764+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0d0002df80 con 0x7f0d0010e3b0 2024-04-25T19:22:11.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.775+0000 7f0cf7400700 1 -- 172.21.15.86:0/2962715955 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f0cf809d6f0 con 0x7f0d0010e3b0 2024-04-25T19:22:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.778+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ce40a3e30 msgr2=0x7f0ce40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.778+0000 7f0d05800700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ce40a3e30 0x7f0ce40a62f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009680 tx=0x7f0cf0009260 comp rx=0 tx=0).stop 2024-04-25T19:22:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.778+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 msgr2=0x7f0d001a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.778+0000 7f0d05800700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 0x7f0d001a3fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009790 tx=0x7f0cf800cc30 comp rx=0 tx=0).stop 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 shutdown_connections 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f0ce40a3e30 0x7f0ce40a62f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d0010e3b0 0x7f0d001a3fa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d001092d0 0x7f0d0019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 --2- 172.21.15.86:0/2962715955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d00109c80 0x7f0d0019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 >> 172.21.15.86:0/2962715955 conn(0x7f0d00076210 msgr2=0x7f0d00076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 shutdown_connections 2024-04-25T19:22:11.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:11.779+0000 7f0d05800700 1 -- 172.21.15.86:0/2962715955 wait complete. 2024-04-25T19:22:11.781 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T19:22:12.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a[27779]: 2024-04-25T19:22:11.766+0000 7f8c23a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g[67513]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g[67513]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-mds[67517]: -- 172.21.15.86:0/3465022182 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55568e69da40 con 0x55568e670c00 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g[67513]: starting mds.g at 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d[66682]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d[66682]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.071 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-mds[66690]: -- 172.21.15.86:0/4027120271 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555f1a03a40 con 0x5555f19d6c00 2024-04-25T19:22:12.072 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:22:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d[66682]: starting mds.d at 2024-04-25T19:22:12.166 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.167 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.167 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: starting mds.b at 2024-04-25T19:22:12.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e[62366]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e[62366]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-mds[62370]: -- 172.21.15.183:0/6025638 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d77929a40 con 0x557d778fc800 2024-04-25T19:22:12.167 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e[62366]: starting mds.e at 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h[62690]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h[62690]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-mds[62694]: -- 172.21.15.183:0/1958656677 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563473169a40 con 0x56347313cc00 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h[62690]: starting mds.h at 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k[62046]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k[62046]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-mds[62050]: -- 172.21.15.183:0/1429692647 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2a7b4da40 con 0x55d2a7b21000 2024-04-25T19:22:12.168 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:22:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k[62046]: starting mds.k at 2024-04-25T19:22:12.231 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c[63625]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.231 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c[63625]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.231 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-mds[63629]: -- 172.21.15.194:0/2916021148 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581c56f9a40 con 0x5581c56cc800 2024-04-25T19:22:12.232 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c[63625]: starting mds.c at 2024-04-25T19:22:12.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i[63932]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i[63932]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-mds[63936]: -- 172.21.15.194:0/452545291 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576f759da40 con 0x5576f7570c00 2024-04-25T19:22:12.232 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i[63932]: starting mds.i at 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l[63004]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l[63004]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-mds[63008]: -- 172.21.15.194:0/3015397636 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56394bd99a40 con 0x56394bd6c800 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l[63004]: starting mds.l at 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f[63314]: ignoring --setuser ceph since I am not root 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f[63314]: ignoring --setgroup ceph since I am not root 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-mds[63318]: -- 172.21.15.194:0/427026121 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a88505da40 con 0x55a885030800 2024-04-25T19:22:12.233 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:22:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f[63314]: starting mds.f at 2024-04-25T19:22:12.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T19:22:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: pgmap v607: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T19:22:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/2962715955' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:22:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:22:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T19:22:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:22:12.731 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T19:22:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: osdmap e122: 12 total, 12 up, 12 in 2024-04-25T19:22:12.732 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:12 smithi194 ceph-mon[32740]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T19:22:12.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: pgmap v607: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/2962715955' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: osdmap e122: 12 total, 12 up, 12 in 2024-04-25T19:22:12.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:12 smithi086 ceph-mon[27783]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T19:22:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: pgmap v607: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T19:22:12.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/2962715955' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:22:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:22:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T19:22:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:22:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T19:22:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: osdmap e122: 12 total, 12 up, 12 in 2024-04-25T19:22:12.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:12 smithi183 ceph-mon[32344]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T19:22:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6832/3688189388,v1:172.21.15.194:6833/3688189388] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6834/3234147851,v1:172.21.15.194:6835/3234147851] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6832/3502996795,v1:172.21.15.183:6836/3502996795] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6836/4037845081,v1:172.21.15.194:6837/4037845081] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6833/1109211977,v1:172.21.15.183:6837/1109211977] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6834/3956148223,v1:172.21.15.183:6838/3956148223] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.183:6835/1294683194,v1:172.21.15.183:6839/1294683194] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.194:6838/2502732719,v1:172.21.15.194:6839/2502732719] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6836/4195774625,v1:172.21.15.86:6838/4195774625] up:boot 2024-04-25T19:22:14.071 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: mds.? [v2:172.21.15.86:6837/3479543189,v1:172.21.15.86:6839/3479543189] up:boot 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T19:22:14.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:13 smithi086 ceph-mon[27783]: osdmap e123: 12 total, 12 up, 12 in 2024-04-25T19:22:14.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6832/3688189388,v1:172.21.15.194:6833/3688189388] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6834/3234147851,v1:172.21.15.194:6835/3234147851] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6832/3502996795,v1:172.21.15.183:6836/3502996795] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6836/4037845081,v1:172.21.15.194:6837/4037845081] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6833/1109211977,v1:172.21.15.183:6837/1109211977] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6834/3956148223,v1:172.21.15.183:6838/3956148223] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.183:6835/1294683194,v1:172.21.15.183:6839/1294683194] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.194:6838/2502732719,v1:172.21.15.194:6839/2502732719] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6836/4195774625,v1:172.21.15.86:6838/4195774625] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: mds.? [v2:172.21.15.86:6837/3479543189,v1:172.21.15.86:6839/3479543189] up:boot 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T19:22:14.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T19:22:14.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:13 smithi183 ceph-mon[32344]: osdmap e123: 12 total, 12 up, 12 in 2024-04-25T19:22:14.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6832/3688189388,v1:172.21.15.194:6833/3688189388] up:boot 2024-04-25T19:22:14.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6834/3234147851,v1:172.21.15.194:6835/3234147851] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6832/3502996795,v1:172.21.15.183:6836/3502996795] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6836/4037845081,v1:172.21.15.194:6837/4037845081] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6833/1109211977,v1:172.21.15.183:6837/1109211977] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6834/3956148223,v1:172.21.15.183:6838/3956148223] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.183:6835/1294683194,v1:172.21.15.183:6839/1294683194] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.194:6838/2502732719,v1:172.21.15.194:6839/2502732719] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6836/4195774625,v1:172.21.15.86:6838/4195774625] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: mds.? [v2:172.21.15.86:6837/3479543189,v1:172.21.15.86:6839/3479543189] up:boot 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:22:14.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:22:14.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T19:22:14.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:22:14.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:22:14.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T19:22:14.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: from='mgr.14991 172.21.15.86:0/182773429' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T19:22:14.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:13 smithi194 ceph-mon[32740]: osdmap e123: 12 total, 12 up, 12 in 2024-04-25T19:22:14.910 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:14 smithi086 ceph-mon[27783]: pgmap v609: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T19:22:15.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:14 smithi183 ceph-mon[32344]: pgmap v609: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 -- 172.21.15.86:0/2488839342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4102a90 msgr2=0x7f45f4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 --2- 172.21.15.86:0/2488839342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4102a90 0x7f45f4102f10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009990 tx=0x7f45dc009ca0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 -- 172.21.15.86:0/2488839342 shutdown_connections 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 --2- 172.21.15.86:0/2488839342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 0x7f45f410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 --2- 172.21.15.86:0/2488839342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f4108a50 0x7f45f4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 --2- 172.21.15.86:0/2488839342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4102a90 0x7f45f4102f10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.213+0000 7f45fb400700 1 -- 172.21.15.86:0/2488839342 >> 172.21.15.86:0/2488839342 conn(0x7f45f4079ba0 msgr2=0x7f45f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:15.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.214+0000 7f45fb400700 1 -- 172.21.15.86:0/2488839342 shutdown_connections 2024-04-25T19:22:15.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.214+0000 7f45fb400700 1 -- 172.21.15.86:0/2488839342 wait complete. 2024-04-25T19:22:15.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.215+0000 7f45fb400700 1 Processor -- start 2024-04-25T19:22:15.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.215+0000 7f45fb400700 1 -- start start 2024-04-25T19:22:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45fb400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f4102a90 0x7f45f419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45fb400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 0x7f45f419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4108a50 0x7f45f419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45fb400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4112c70 con 0x7f45f4108a50 2024-04-25T19:22:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45fb400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4112af0 con 0x7f45f4102a90 2024-04-25T19:22:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45fb400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45f4112df0 con 0x7f45f4103450 2024-04-25T19:22:15.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4108a50 0x7f45f419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45ebe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 0x7f45f419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45f8a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f4102a90 0x7f45f419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4108a50 0x7f45f419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36104/0 (socket says 172.21.15.86:36104) 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.216+0000 7f45f9400700 1 -- 172.21.15.86:0/3735808555 learned_addr learned my addr 172.21.15.86:0/3735808555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45ebe00700 1 -- 172.21.15.86:0/3735808555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f4102a90 msgr2=0x7f45f419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45ebe00700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f4102a90 0x7f45f419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45ebe00700 1 -- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4108a50 msgr2=0x7f45f419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45ebe00700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4108a50 0x7f45f419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45ebe00700 1 -- 172.21.15.86:0/3735808555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f41a0390 con 0x7f45f4103450 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45ebe00700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 0x7f45f419b8c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f45dc000c00 tx=0x7f45dc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc026070 con 0x7f45f4103450 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f41a0620 con 0x7f45f4103450 2024-04-25T19:22:15.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.217+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc021410 con 0x7f45f4103450 2024-04-25T19:22:15.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.218+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc017650 con 0x7f45f4103450 2024-04-25T19:22:15.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.218+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41a7f00 con 0x7f45f4103450 2024-04-25T19:22:15.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.219+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f4069730 con 0x7f45f4103450 2024-04-25T19:22:15.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.221+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45dc029070 con 0x7f45f4103450 2024-04-25T19:22:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.222+0000 7f45f2000700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45d80a3e30 0x7f45d80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.222+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9067+0+0 (secure 0 0 0) 0x7f45dc0d1690 con 0x7f45f4103450 2024-04-25T19:22:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.222+0000 7f45f8a00700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45d80a3e30 0x7f45d80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.223+0000 7f45f8a00700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45d80a3e30 0x7f45d80a62f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009680 tx=0x7f45e4009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:15.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.227+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45dc05dff0 con 0x7f45f4103450 2024-04-25T19:22:15.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:14 smithi194 ceph-mon[32740]: pgmap v609: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T19:22:15.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.502+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f45f402df80 con 0x7f45f4103450 2024-04-25T19:22:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.807+0000 7f45f2000700 1 -- 172.21.15.86:0/3735808555 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f45dc013070 con 0x7f45f4103450 2024-04-25T19:22:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45d80a3e30 msgr2=0x7f45d80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45d80a3e30 0x7f45d80a62f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009680 tx=0x7f45e4009260 comp rx=0 tx=0).stop 2024-04-25T19:22:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 msgr2=0x7f45f419b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 0x7f45f419b8c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f45dc000c00 tx=0x7f45dc0047b0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 shutdown_connections 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f4108a50 0x7f45f419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f4102a90 0x7f45f419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f45d80a3e30 0x7f45d80a62f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 --2- 172.21.15.86:0/3735808555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f4103450 0x7f45f419b8c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 >> 172.21.15.86:0/3735808555 conn(0x7f45f4079ba0 msgr2=0x7f45f410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 shutdown_connections 2024-04-25T19:22:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:15.811+0000 7f45fb400700 1 -- 172.21.15.86:0/3735808555 wait complete. 2024-04-25T19:22:16.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:15 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/3735808555' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:22:16.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:15 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:22:16.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:15 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/3735808555' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:22:16.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:15 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:22:16.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:15 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/3735808555' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:22:16.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:15 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:22:16.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T19:22:17.040 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:22:17.069 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:16 smithi086 ceph-mon[27783]: pgmap v611: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T19:22:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:16 smithi086 ceph-mon[27783]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T19:22:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:16 smithi086 ceph-mon[27783]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T19:22:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:16 smithi086 ceph-mon[27783]: Cluster is now healthy 2024-04-25T19:22:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:16 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T19:22:17.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:16 smithi086 ceph-mon[27783]: fsmap 12 up:standby 2024-04-25T19:22:17.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:16 smithi183 ceph-mon[32344]: pgmap v611: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T19:22:17.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:16 smithi183 ceph-mon[32344]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T19:22:17.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:16 smithi183 ceph-mon[32344]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T19:22:17.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:16 smithi183 ceph-mon[32344]: Cluster is now healthy 2024-04-25T19:22:17.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:16 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T19:22:17.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:16 smithi183 ceph-mon[32344]: fsmap 12 up:standby 2024-04-25T19:22:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:16 smithi194 ceph-mon[32740]: pgmap v611: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T19:22:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:16 smithi194 ceph-mon[32740]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T19:22:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:16 smithi194 ceph-mon[32740]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T19:22:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:16 smithi194 ceph-mon[32740]: Cluster is now healthy 2024-04-25T19:22:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:16 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T19:22:17.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:16 smithi194 ceph-mon[32740]: fsmap 12 up:standby 2024-04-25T19:22:18.994 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:18 smithi086 ceph-mon[27783]: pgmap v612: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-25T19:22:19.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:18 smithi183 ceph-mon[32344]: pgmap v612: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-25T19:22:19.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:18 smithi194 ceph-mon[32740]: pgmap v612: 97 pgs: 97 active+clean; 1.7 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 -- 172.21.15.86:0/1551188083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c4108a30 msgr2=0x7fc9c4108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 --2- 172.21.15.86:0/1551188083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c4108a30 0x7fc9c4108e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b0a0 tx=0x7fc9b400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 -- 172.21.15.86:0/1551188083 shutdown_connections 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 --2- 172.21.15.86:0/1551188083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c406b0f0 0x7fc9c410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 --2- 172.21.15.86:0/1551188083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c4108a30 0x7fc9c4108e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 --2- 172.21.15.86:0/1551188083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 -- 172.21.15.86:0/1551188083 >> 172.21.15.86:0/1551188083 conn(0x7fc9c40788f0 msgr2=0x7fc9c4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.278+0000 7fc9c9000700 1 -- 172.21.15.86:0/1551188083 shutdown_connections 2024-04-25T19:22:19.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.279+0000 7fc9c9000700 1 -- 172.21.15.86:0/1551188083 wait complete. 2024-04-25T19:22:19.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 Processor -- start 2024-04-25T19:22:19.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 -- start start 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c406b0f0 0x7fc9c419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c4108a30 0x7fc9c419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4112bd0 con 0x7fc9c4108a30 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4112a50 con 0x7fc9c406b0f0 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.280+0000 7fc9c9000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4112d50 con 0x7fc9c406a750 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c4108a30 0x7fc9c419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:19.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:19.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c4108a30 0x7fc9c419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36132/0 (socket says 172.21.15.86:36132) 2024-04-25T19:22:19.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.86:56216/0 (socket says 172.21.15.86:56216) 2024-04-25T19:22:19.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c2800700 1 -- 172.21.15.86:0/770766857 learned_addr learned my addr 172.21.15.86:0/770766857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:19.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9b9400700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c406b0f0 0x7fc9c419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 -- 172.21.15.86:0/770766857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c406b0f0 msgr2=0x7fc9c419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c406b0f0 0x7fc9c419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 -- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c4108a30 msgr2=0x7fc9c419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c4108a30 0x7fc9c419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9c1e00700 1 -- 172.21.15.86:0/770766857 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c41a0330 con 0x7fc9c406a750 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.281+0000 7fc9b9400700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c406b0f0 0x7fc9c419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:22:19.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.282+0000 7fc9c1e00700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c419b320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b070 tx=0x7fc9b4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:19.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.282+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4021070 con 0x7fc9c406a750 2024-04-25T19:22:19.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.282+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b4007e30 con 0x7fc9c406a750 2024-04-25T19:22:19.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.282+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c41a05c0 con 0x7fc9c406a750 2024-04-25T19:22:19.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.282+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4004560 con 0x7fc9c406a750 2024-04-25T19:22:19.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.282+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c41a7ea0 con 0x7fc9c406a750 2024-04-25T19:22:19.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.283+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c4069730 con 0x7fc9c406a750 2024-04-25T19:22:19.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.285+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc9b4004700 con 0x7fc9c406a750 2024-04-25T19:22:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.287+0000 7fc9bb400700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc9a80a3e30 0x7fc9a80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.287+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9067+0+0 (secure 0 0 0) 0x7fc9b400f020 con 0x7fc9c406a750 2024-04-25T19:22:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.287+0000 7fc9b9400700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc9a80a3e30 0x7fc9a80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.288+0000 7fc9b9400700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc9a80a3e30 0x7fc9a80a62f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4078fa0 tx=0x7fc9ac009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.291+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9b402a020 con 0x7fc9c406a750 2024-04-25T19:22:19.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.514+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc9c402df80 con 0x7fc9c406a750 2024-04-25T19:22:19.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.821+0000 7fc9bb400700 1 -- 172.21.15.86:0/770766857 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) ==== 181+0+0 (secure 0 0 0) 0x7fc9b409e300 con 0x7fc9c406a750 2024-04-25T19:22:19.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.827+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc9a80a3e30 msgr2=0x7fc9a80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:19.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.827+0000 7fc9c9000700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc9a80a3e30 0x7fc9a80a62f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4078fa0 tx=0x7fc9ac009260 comp rx=0 tx=0).stop 2024-04-25T19:22:19.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.827+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 msgr2=0x7fc9c419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:19.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.827+0000 7fc9c9000700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c419b320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b070 tx=0x7fc9b4004d00 comp rx=0 tx=0).stop 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.827+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 shutdown_connections 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9c4108a30 0x7fc9c419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7fc9a80a3e30 0x7fc9a80a62f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9c406a750 0x7fc9c419b320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 --2- 172.21.15.86:0/770766857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9c406b0f0 0x7fc9c419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 >> 172.21.15.86:0/770766857 conn(0x7fc9c40788f0 msgr2=0x7fc9c410de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:19.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 shutdown_connections 2024-04-25T19:22:19.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:19.828+0000 7fc9c9000700 1 -- 172.21.15.86:0/770766857 wait complete. 2024-04-25T19:22:19.830 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T19:22:20.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:19 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/770766857' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:20.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:19 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:20.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:19 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/770766857' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:20.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:19 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:20.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:19 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/770766857' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:20.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:19 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:20.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T19:22:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:20 smithi086 ceph-mon[27783]: pgmap v613: 97 pgs: 97 active+clean; 1.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-25T19:22:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:20 smithi086 ceph-mon[27783]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:22:21.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:20 smithi086 ceph-mon[27783]: osdmap e124: 12 total, 12 up, 12 in 2024-04-25T19:22:21.082 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:22:21.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:20 smithi183 ceph-mon[32344]: pgmap v613: 97 pgs: 97 active+clean; 1.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-25T19:22:21.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:20 smithi183 ceph-mon[32344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:22:21.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:20 smithi183 ceph-mon[32344]: osdmap e124: 12 total, 12 up, 12 in 2024-04-25T19:22:21.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:20 smithi194 ceph-mon[32740]: pgmap v613: 97 pgs: 97 active+clean; 1.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-25T19:22:21.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:20 smithi194 ceph-mon[32740]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:22:21.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:20 smithi194 ceph-mon[32740]: osdmap e124: 12 total, 12 up, 12 in 2024-04-25T19:22:23.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:22 smithi086 ceph-mon[27783]: pgmap v615: 65 pgs: 65 active+clean; 577 KiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:22:23.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:22 smithi183 ceph-mon[32344]: pgmap v615: 65 pgs: 65 active+clean; 577 KiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:22:23.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:22 smithi194 ceph-mon[32740]: pgmap v615: 65 pgs: 65 active+clean; 577 KiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.253+0000 7f3f03e00700 1 -- 172.21.15.86:0/2378986578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1138f0 msgr2=0x7f3efc115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.253+0000 7f3f03e00700 1 --2- 172.21.15.86:0/2378986578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1138f0 0x7f3efc115ce0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b0a0 tx=0x7f3ef400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 -- 172.21.15.86:0/2378986578 shutdown_connections 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 --2- 172.21.15.86:0/2378986578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc1138f0 0x7f3efc115ce0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 --2- 172.21.15.86:0/2378986578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc076f70 0x7f3efc077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 --2- 172.21.15.86:0/2378986578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc077890 0x7f3efc1133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 -- 172.21.15.86:0/2378986578 >> 172.21.15.86:0/2378986578 conn(0x7f3efc100720 msgr2=0x7f3efc102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:23.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 -- 172.21.15.86:0/2378986578 shutdown_connections 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.254+0000 7f3f03e00700 1 -- 172.21.15.86:0/2378986578 wait complete. 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.255+0000 7f3f03e00700 1 Processor -- start 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.255+0000 7f3f03e00700 1 -- start start 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.255+0000 7f3f03e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 0x7f3efc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.255+0000 7f3f03e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc077890 0x7f3efc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f03e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1138f0 0x7f3efc1a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f03e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc119270 con 0x7f3efc076f70 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f03e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1190f0 con 0x7f3efc077890 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f03e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1193f0 con 0x7f3efc1138f0 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 0x7f3efc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:23.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 0x7f3efc19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36592/0 (socket says 172.21.15.86:36592) 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 -- 172.21.15.86:0/405606054 learned_addr learned my addr 172.21.15.86:0/405606054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f00a00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc077890 0x7f3efc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1138f0 0x7f3efc1a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 -- 172.21.15.86:0/405606054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1138f0 msgr2=0x7f3efc1a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1138f0 0x7f3efc1a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 -- 172.21.15.86:0/405606054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc077890 msgr2=0x7f3efc19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc077890 0x7f3efc19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f01400700 1 -- 172.21.15.86:0/405606054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a4730 con 0x7f3efc076f70 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.256+0000 7f3f00a00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc077890 0x7f3efc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.257+0000 7f3f01400700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 0x7f3efc19f720 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000e9a0 tx=0x7f3ef000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.257+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000ca10 con 0x7f3efc076f70 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.257+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef000cbb0 con 0x7f3efc076f70 2024-04-25T19:22:23.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.257+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0018680 con 0x7f3efc076f70 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.257+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a4a20 con 0x7f3efc076f70 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.257+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1ac2a0 con 0x7f3efc076f70 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.258+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc069730 con 0x7f3efc076f70 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.260+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ef0010430 con 0x7f3efc076f70 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.262+0000 7f3efa000700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3ee40a3db0 0x7f3ee40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.262+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8490+0+0 (secure 0 0 0) 0x7f3ef00d09b0 con 0x7f3efc076f70 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.262+0000 7f3f00a00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3ee40a3db0 0x7f3ee40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.263+0000 7f3f00a00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3ee40a3db0 0x7f3ee40a6270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3efc1041d0 tx=0x7f3ee8009260 comp rx=0 tx=0).ready entity=mgr.14991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:22:23.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.266+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ef009d010 con 0x7f3efc076f70 2024-04-25T19:22:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.490+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3efc02df80 con 0x7f3efc076f70 2024-04-25T19:22:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.831+0000 7f3efa000700 1 -- 172.21.15.86:0/405606054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) ==== 169+0+0 (secure 0 0 0) 0x7f3ef009c910 con 0x7f3efc076f70 2024-04-25T19:22:23.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.835+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3ee40a3db0 msgr2=0x7f3ee40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.836+0000 7f3f03e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3ee40a3db0 0x7f3ee40a6270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3efc1041d0 tx=0x7f3ee8009260 comp rx=0 tx=0).stop 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.836+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 msgr2=0x7f3efc19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.836+0000 7f3f03e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 0x7f3efc19f720 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000e9a0 tx=0x7f3ef000ecb0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.837+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 shutdown_connections 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.837+0000 7f3f03e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:6800/421083475,v1:172.21.15.86:6801/421083475] conn(0x7f3ee40a3db0 0x7f3ee40a6270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.837+0000 7f3f03e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1138f0 0x7f3efc1a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.837+0000 7f3f03e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3efc076f70 0x7f3efc19f720 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.837+0000 7f3f03e00700 1 --2- 172.21.15.86:0/405606054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc077890 0x7f3efc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:22:23.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.837+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 >> 172.21.15.86:0/405606054 conn(0x7f3efc100720 msgr2=0x7f3efc114560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:22:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.839+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 shutdown_connections 2024-04-25T19:22:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T19:22:23.839+0000 7f3f03e00700 1 -- 172.21.15.86:0/405606054 wait complete. 2024-04-25T19:22:23.844 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T19:22:24.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:23 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/405606054' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:24.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:23 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/405606054' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:24.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:23 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/405606054' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:22:24.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:22:24.858 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:22:24.892 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:22:24.927 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T19:22:24.927 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T19:22:24.980 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T19:22:24.998 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T19:22:25.016 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T19:22:25.016 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T19:22:25.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a 2024-04-25T19:22:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:24 smithi086 ceph-mon[27783]: pgmap v616: 65 pgs: 65 active+clean; 577 KiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:22:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:24 smithi086 ceph-mon[27783]: from='client.? 172.21.15.86:0/405606054' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:22:25.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:24 smithi086 ceph-mon[27783]: osdmap e125: 12 total, 12 up, 12 in 2024-04-25T19:22:25.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:24 smithi183 ceph-mon[32344]: pgmap v616: 65 pgs: 65 active+clean; 577 KiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:22:25.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:24 smithi183 ceph-mon[32344]: from='client.? 172.21.15.86:0/405606054' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:22:25.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:24 smithi183 ceph-mon[32344]: osdmap e125: 12 total, 12 up, 12 in 2024-04-25T19:22:25.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:24 smithi194 ceph-mon[32740]: pgmap v616: 65 pgs: 65 active+clean; 577 KiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:22:25.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:24 smithi194 ceph-mon[32740]: from='client.? 172.21.15.86:0/405606054' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:22:25.231 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:24 smithi194 ceph-mon[32740]: osdmap e125: 12 total, 12 up, 12 in 2024-04-25T19:22:25.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:25 smithi086 systemd[1]: Stopping Ceph mon.a for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:26.063 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a[27779]: 2024-04-25T19:22:25.594+0000 7f8c2a800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:26.064 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a[27779]: 2024-04-25T19:22:25.594+0000 7f8c2a800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T19:22:26.064 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:25 smithi086 podman[106292]: 2024-04-25 19:22:25.728903675 +0000 UTC m=+0.276983344 container died 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T19:22:26.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:26 smithi086 podman[106292]: 2024-04-25 19:22:26.196571934 +0000 UTC m=+0.744651600 container cleanup 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:22:26.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:26 smithi086 bash[106292]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a 2024-04-25T19:22:26.820 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 25 19:22:26 smithi086 podman[106306]: 2024-04-25 19:22:26.431133646 +0000 UTC m=+0.689886560 container remove 2c6bbf4e309111335798382873ff1462099cd5e8c729c05e99890e1dfcef4fc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:22:26.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T19:22:27.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:27.100 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T19:22:27.100 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T19:22:27.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.b 2024-04-25T19:22:27.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:27 smithi183 systemd[1]: Stopping Ceph mon.b for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:27.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-b[32340]: 2024-04-25T19:22:27.423+0000 7f3333e15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:27.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-b[32340]: 2024-04-25T19:22:27.423+0000 7f3333e15700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T19:22:27.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:27 smithi183 podman[70964]: 2024-04-25 19:22:27.648608824 +0000 UTC m=+0.349567835 container died e7a7f96bbe259f79da2e400615c88f6962d207fb0353ba855a069133abec08dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:22:28.320 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:28 smithi183 podman[70964]: 2024-04-25 19:22:28.06020098 +0000 UTC m=+0.761159968 container cleanup e7a7f96bbe259f79da2e400615c88f6962d207fb0353ba855a069133abec08dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T19:22:28.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 25 19:22:28 smithi183 bash[70964]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-b 2024-04-25T19:22:28.482 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T19:22:28.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:28.529 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T19:22:28.529 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T19:22:28.529 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.c 2024-04-25T19:22:28.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:28 smithi194 systemd[1]: Stopping Ceph mon.c for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:29.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-c[32736]: 2024-04-25T19:22:28.987+0000 7f91ea4a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:29.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-c[32736]: 2024-04-25T19:22:28.987+0000 7f91ea4a6700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T19:22:29.481 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:29 smithi194 podman[71200]: 2024-04-25 19:22:29.212448169 +0000 UTC m=+0.479569431 container died 556dd49b81d06f3727e3ec3f0b9db903ce299b4cdbed12f9b7fe6daca7277486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T19:22:29.866 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:29 smithi194 podman[71200]: 2024-04-25 19:22:29.639202492 +0000 UTC m=+0.906323755 container cleanup 556dd49b81d06f3727e3ec3f0b9db903ce299b4cdbed12f9b7fe6daca7277486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T19:22:29.866 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:29 smithi194 bash[71200]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-c 2024-04-25T19:22:30.125 INFO:journalctl@ceph.mon.c.smithi194.stdout:Apr 25 19:22:29 smithi194 podman[71213]: 2024-04-25 19:22:29.865962053 +0000 UTC m=+0.648097777 container remove 556dd49b81d06f3727e3ec3f0b9db903ce299b4cdbed12f9b7fe6daca7277486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T19:22:30.139 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T19:22:30.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:30.183 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T19:22:30.183 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T19:22:30.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x 2024-04-25T19:22:30.570 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:30 smithi086 systemd[1]: Stopping Ceph mgr.x for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:31.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:22:30.781+0000 7ff0c5800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:31.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x[28013]: 2024-04-25T19:22:30.781+0000 7ff0c5800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T19:22:31.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:30 smithi086 podman[106412]: 2024-04-25 19:22:30.878448501 +0000 UTC m=+0.235985086 container died 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:22:31.764 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:31 smithi086 podman[106412]: 2024-04-25 19:22:31.493619592 +0000 UTC m=+0.851156221 container cleanup 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T19:22:31.764 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:31 smithi086 bash[106412]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x 2024-04-25T19:22:32.070 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 25 19:22:31 smithi086 podman[106426]: 2024-04-25 19:22:31.762318695 +0000 UTC m=+0.871373189 container remove 06ae06a759950c91c7deab48ad5b073703630331705d2c71ceef7d0ab023955a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:22:32.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T19:22:32.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:32.441 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T19:22:32.442 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T19:22:32.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.y 2024-04-25T19:22:32.786 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:22:32 smithi183 systemd[1]: Stopping Ceph mgr.y for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:33.141 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:22:32 smithi183 podman[71087]: 2024-04-25 19:22:32.785340609 +0000 UTC m=+0.150898458 container stop f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T19:22:33.142 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:22:32 smithi183 podman[71087]: 2024-04-25 19:22:32.811015783 +0000 UTC m=+0.176573622 container died f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T19:22:33.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:22:33 smithi183 podman[71087]: 2024-04-25 19:22:33.240502798 +0000 UTC m=+0.606060636 container cleanup f26e03817af608af1a55adc586ab6c8be989d3fd99be77019bfedd449ec2ad4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T19:22:33.416 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 25 19:22:33 smithi183 bash[71087]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-y 2024-04-25T19:22:33.649 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T19:22:33.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:33.681 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T19:22:33.682 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T19:22:33.682 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.z 2024-04-25T19:22:33.981 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:22:33 smithi194 systemd[1]: Stopping Ceph mgr.z for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:34.388 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:22:34 smithi194 podman[71332]: 2024-04-25 19:22:34.03072183 +0000 UTC m=+0.161716613 container died c92e9a25aeb63552e80c119ad222896220c92c19dae65b6ff19a332a3e510e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T19:22:34.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:22:34 smithi194 podman[71332]: 2024-04-25 19:22:34.486625818 +0000 UTC m=+0.617620599 container cleanup c92e9a25aeb63552e80c119ad222896220c92c19dae65b6ff19a332a3e510e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T19:22:34.731 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Apr 25 19:22:34 smithi194 bash[71332]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mgr-z 2024-04-25T19:22:34.960 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T19:22:34.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:34.994 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T19:22:34.995 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T19:22:34.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.0 2024-04-25T19:22:35.320 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:35 smithi086 systemd[1]: Stopping Ceph osd.0 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:35.820 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0[40174]: 2024-04-25T19:22:35.531+0000 7ff95c600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:35.820 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0[40174]: 2024-04-25T19:22:35.531+0000 7ff95c600700 -1 osd.0 125 *** Got signal Terminated *** 2024-04-25T19:22:35.821 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0[40174]: 2024-04-25T19:22:35.531+0000 7ff95c600700 -1 osd.0 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:22:41.266 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:40 smithi086 podman[106546]: 2024-04-25 19:22:40.944642356 +0000 UTC m=+5.553369438 container died d1370c9d29ee187c4fe7f24650dc0bd37c0177121dc5367c8ae1ff8804ca5a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T19:22:41.570 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:41 smithi086 podman[106546]: 2024-04-25 19:22:41.432219247 +0000 UTC m=+6.040946352 container cleanup d1370c9d29ee187c4fe7f24650dc0bd37c0177121dc5367c8ae1ff8804ca5a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:22:41.570 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:41 smithi086 bash[106546]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0 2024-04-25T19:22:42.070 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:41 smithi086 podman[106565]: 2024-04-25 19:22:41.750479854 +0000 UTC m=+0.783986856 container remove d1370c9d29ee187c4fe7f24650dc0bd37c0177121dc5367c8ae1ff8804ca5a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T19:22:42.707 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:42 smithi086 podman[106658]: 2024-04-25 19:22:42.269368686 +0000 UTC m=+0.036527172 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:22:42.707 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:42 smithi086 podman[106658]: 2024-04-25 19:22:42.418164529 +0000 UTC m=+0.185323061 container create 95bd86b8605ebd3db4cf196ff8cd4072d97a48d2d0a856379dcbb4bfc8de3b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T19:22:42.707 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:42 smithi086 podman[106658]: 2024-04-25 19:22:42.704986535 +0000 UTC m=+0.472145049 container init 95bd86b8605ebd3db4cf196ff8cd4072d97a48d2d0a856379dcbb4bfc8de3b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:22:43.070 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:42 smithi086 podman[106658]: 2024-04-25 19:22:42.718694659 +0000 UTC m=+0.485853174 container start 95bd86b8605ebd3db4cf196ff8cd4072d97a48d2d0a856379dcbb4bfc8de3b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T19:22:43.070 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:42 smithi086 podman[106658]: 2024-04-25 19:22:42.804026522 +0000 UTC m=+0.571185024 container attach 95bd86b8605ebd3db4cf196ff8cd4072d97a48d2d0a856379dcbb4bfc8de3b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:22:43.531 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 25 19:22:43 smithi086 podman[106658]: 2024-04-25 19:22:43.166541935 +0000 UTC m=+0.933700445 container died 95bd86b8605ebd3db4cf196ff8cd4072d97a48d2d0a856379dcbb4bfc8de3b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:22:43.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T19:22:44.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:44.056 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T19:22:44.056 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T19:22:44.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.1 2024-04-25T19:22:44.371 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:44 smithi183 systemd[1]: Stopping Ceph osd.1 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:44.666 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1[36922]: 2024-04-25T19:22:44.369+0000 7f4d5c188700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:44.666 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1[36922]: 2024-04-25T19:22:44.369+0000 7f4d5c188700 -1 osd.1 125 *** Got signal Terminated *** 2024-04-25T19:22:44.666 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1[36922]: 2024-04-25T19:22:44.369+0000 7f4d5c188700 -1 osd.1 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:22:50.130 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:49 smithi183 podman[71227]: 2024-04-25 19:22:49.80758042 +0000 UTC m=+5.559358501 container died f0ef81040bdf597f05e8bb5b4f9815bac8ee1c60412ca2a4b4c4c5456a55a928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T19:22:50.416 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:50 smithi183 podman[71227]: 2024-04-25 19:22:50.2285069 +0000 UTC m=+5.980284982 container cleanup f0ef81040bdf597f05e8bb5b4f9815bac8ee1c60412ca2a4b4c4c5456a55a928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T19:22:50.417 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:50 smithi183 bash[71227]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1 2024-04-25T19:22:50.896 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:50 smithi183 podman[71242]: 2024-04-25 19:22:50.480054333 +0000 UTC m=+0.665173542 container remove f0ef81040bdf597f05e8bb5b4f9815bac8ee1c60412ca2a4b4c4c5456a55a928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T19:22:51.166 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:50 smithi183 podman[71337]: 2024-04-25 19:22:50.798646226 +0000 UTC m=+0.018893190 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:22:51.167 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:50 smithi183 podman[71337]: 2024-04-25 19:22:50.931343124 +0000 UTC m=+0.151590116 container create 00cdfa9ae833c1e7a3a529e443daf61a4482e8b12f85e6c056895247c8a8d9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T19:22:51.493 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:51 smithi183 podman[71337]: 2024-04-25 19:22:51.192558775 +0000 UTC m=+0.412805721 container init 00cdfa9ae833c1e7a3a529e443daf61a4482e8b12f85e6c056895247c8a8d9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T19:22:51.493 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:51 smithi183 podman[71337]: 2024-04-25 19:22:51.195570499 +0000 UTC m=+0.415817454 container start 00cdfa9ae833c1e7a3a529e443daf61a4482e8b12f85e6c056895247c8a8d9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T19:22:51.494 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:51 smithi183 podman[71337]: 2024-04-25 19:22:51.284530138 +0000 UTC m=+0.504777113 container attach 00cdfa9ae833c1e7a3a529e443daf61a4482e8b12f85e6c056895247c8a8d9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True) 2024-04-25T19:22:51.825 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 25 19:22:51 smithi183 podman[71337]: 2024-04-25 19:22:51.493692303 +0000 UTC m=+0.713939252 container died 00cdfa9ae833c1e7a3a529e443daf61a4482e8b12f85e6c056895247c8a8d9e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T19:22:52.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T19:22:52.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:22:52.239 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T19:22:52.240 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T19:22:52.240 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.2 2024-04-25T19:22:52.584 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:52 smithi194 systemd[1]: Stopping Ceph osd.2 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:22:52.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2[37881]: 2024-04-25T19:22:52.583+0000 7fc6ad2c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:22:52.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2[37881]: 2024-04-25T19:22:52.583+0000 7fc6ad2c6700 -1 osd.2 125 *** Got signal Terminated *** 2024-04-25T19:22:52.981 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2[37881]: 2024-04-25T19:22:52.583+0000 7fc6ad2c6700 -1 osd.2 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:22:58.177 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:57 smithi194 podman[71455]: 2024-04-25 19:22:57.897341162 +0000 UTC m=+5.469803124 container died a79bc65e24bb7fde6aabac5bd4d3e0c22fa2820260a3858fff0ac85bcfa70fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T19:22:58.481 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:58 smithi194 podman[71455]: 2024-04-25 19:22:58.27567791 +0000 UTC m=+5.848139869 container cleanup a79bc65e24bb7fde6aabac5bd4d3e0c22fa2820260a3858fff0ac85bcfa70fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T19:22:58.481 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:58 smithi194 bash[71455]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2 2024-04-25T19:22:58.916 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:58 smithi194 podman[71474]: 2024-04-25 19:22:58.585989668 +0000 UTC m=+0.683462293 container remove a79bc65e24bb7fde6aabac5bd4d3e0c22fa2820260a3858fff0ac85bcfa70fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:22:59.198 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:58 smithi194 podman[71567]: 2024-04-25 19:22:58.818598607 +0000 UTC m=+0.018601756 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:22:59.199 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:58 smithi194 podman[71567]: 2024-04-25 19:22:58.936921398 +0000 UTC m=+0.136924584 container create 7a3ce5ba62b103c29b4fbe0b641606761ee7a66c63a43015780638fc52b6fc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:22:59.199 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:59 smithi194 podman[71567]: 2024-04-25 19:22:59.198131602 +0000 UTC m=+0.398134754 container init 7a3ce5ba62b103c29b4fbe0b641606761ee7a66c63a43015780638fc52b6fc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T19:22:59.481 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:59 smithi194 podman[71567]: 2024-04-25 19:22:59.202125474 +0000 UTC m=+0.402128615 container start 7a3ce5ba62b103c29b4fbe0b641606761ee7a66c63a43015780638fc52b6fc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T19:22:59.482 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:59 smithi194 podman[71567]: 2024-04-25 19:22:59.306068769 +0000 UTC m=+0.506071919 container attach 7a3ce5ba62b103c29b4fbe0b641606761ee7a66c63a43015780638fc52b6fc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:22:59.885 INFO:journalctl@ceph.osd.2.smithi194.stdout:Apr 25 19:22:59 smithi194 podman[71567]: 2024-04-25 19:22:59.501641069 +0000 UTC m=+0.701644211 container died 7a3ce5ba62b103c29b4fbe0b641606761ee7a66c63a43015780638fc52b6fc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T19:23:00.211 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T19:23:00.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:00.253 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T19:23:00.254 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T19:23:00.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.3 2024-04-25T19:23:00.820 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:00 smithi086 systemd[1]: Stopping Ceph osd.3 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:01.321 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:00 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3[47072]: 2024-04-25T19:23:00.884+0000 7fbc72e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:01.321 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:00 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3[47072]: 2024-04-25T19:23:00.884+0000 7fbc72e00700 -1 osd.3 125 *** Got signal Terminated *** 2024-04-25T19:23:01.321 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:00 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3[47072]: 2024-04-25T19:23:00.884+0000 7fbc72e00700 -1 osd.3 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:01.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:01 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:01.434+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:02.478 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:02.170+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:02.518 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:02 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:02.433+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:03.416 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:03.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:03.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:03 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:03.437+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:04.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:04.211+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:04.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:04.408+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:04.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:04.458+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:04.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:04 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:04.470+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:05.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:05.173+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:05.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:05.455+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:05.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:05.454+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:05.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:05 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:05.470+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:06.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:06 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:06.422+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:06.570 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:06 smithi086 podman[106762]: 2024-04-25 19:23:06.345661222 +0000 UTC m=+5.631129390 container died 8ed25e25e24be84ec296e129e7c497843fcf02a825646a53ac710cb809aa1cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T19:23:06.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:06 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:06.188+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:06.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:06 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:06.224+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:06.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:06.502+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:06.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:06.470+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:07.070 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:06 smithi086 podman[106762]: 2024-04-25 19:23:06.869071271 +0000 UTC m=+6.154539476 container cleanup 8ed25e25e24be84ec296e129e7c497843fcf02a825646a53ac710cb809aa1cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:23:07.070 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:06 smithi086 bash[106762]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3 2024-04-25T19:23:07.387 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:07 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:07.224+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:07.388 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:07 smithi086 podman[106793]: 2024-04-25 19:23:07.154039666 +0000 UTC m=+0.794084641 container remove 8ed25e25e24be84ec296e129e7c497843fcf02a825646a53ac710cb809aa1cb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T19:23:07.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:07 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:07.261+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:07.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:07.540+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:07.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:07.424+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:07.819 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:07 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:07.385+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:08.209 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:07 smithi086 podman[106884]: 2024-04-25 19:23:07.859302158 +0000 UTC m=+0.035047510 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:08.210 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:07 smithi086 podman[106884]: 2024-04-25 19:23:07.971932451 +0000 UTC m=+0.147677831 container create abe1523c0a41777836761b06efdc1291a0cec6cbb6a62d291864d3be38dd9129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T19:23:08.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:08 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:08.401+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:08.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:08 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:08.207+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:08.571 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:08 smithi086 podman[106884]: 2024-04-25 19:23:08.292155242 +0000 UTC m=+0.467900632 container init abe1523c0a41777836761b06efdc1291a0cec6cbb6a62d291864d3be38dd9129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T19:23:08.571 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:08 smithi086 podman[106884]: 2024-04-25 19:23:08.303564936 +0000 UTC m=+0.479310323 container start abe1523c0a41777836761b06efdc1291a0cec6cbb6a62d291864d3be38dd9129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T19:23:08.571 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:08 smithi086 podman[106884]: 2024-04-25 19:23:08.430095138 +0000 UTC m=+0.605840575 container attach abe1523c0a41777836761b06efdc1291a0cec6cbb6a62d291864d3be38dd9129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:23:08.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:08 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:08.259+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:08.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:08.586+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:08.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:08.386+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:09.058 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 25 19:23:08 smithi086 podman[106884]: 2024-04-25 19:23:08.755273603 +0000 UTC m=+0.931018989 container died abe1523c0a41777836761b06efdc1291a0cec6cbb6a62d291864d3be38dd9129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T19:23:09.320 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:09 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:09.165+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:09.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T19:23:09.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:09.584 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T19:23:09.584 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T19:23:09.585 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.4 2024-04-25T19:23:09.611 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:09 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:09.302+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:09.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:09.573+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:09.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:09.398+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:09.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:09 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:09.423+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:09.916 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:09 smithi183 systemd[1]: Stopping Ceph osd.4 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:10.318 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:09 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T19:23:09.924+0000 7ff65d543700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:10.319 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:09 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T19:23:09.924+0000 7ff65d543700 -1 osd.4 125 *** Got signal Terminated *** 2024-04-25T19:23:10.319 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:09 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T19:23:09.924+0000 7ff65d543700 -1 osd.4 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:10.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:10.210+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:10.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:10.395+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:10.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:10 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:10.395+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:10.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:10 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:10.318+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:10.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:10.556+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:10.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:10.352+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:11.346 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:11.041+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:11.434 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:11.178+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:11.444 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:11.357+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:11.444 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:11.130+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:11.622 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:11.344+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:11.622 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:11 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:11.344+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:11.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:11.528+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:11.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:11.433+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:11.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:11 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:11.433+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:12.320 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:12 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:12.039+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:12.460 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:12 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:12.161+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:12.460 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:12 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:12.161+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:12.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:12.324+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:12.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:12.160+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:12.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:12 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:12.318+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:12.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:12 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:12.318+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:12.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:12 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:12.459+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:12.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:12 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:12.459+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:12.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:12.539+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:12.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:12.539+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:13.167 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:12 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T19:23:12.909+0000 7ff659b37700 -1 osd.4 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.587465+0000 front 2024-04-25T19:22:46.587496+0000 (oldest deadline 2024-04-25T19:23:12.487669+0000) 2024-04-25T19:23:13.167 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:13 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:13.076+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:13.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:13.187+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:13.364+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:13.364+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:13.501 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:13 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:13.184+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:13.502 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:13 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:13.184+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:13.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:13 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:13.368+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:13.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:13 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:13.368+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:13.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:13 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:13.500+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:13.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:13 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:13.500+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:13.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:13.528+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:13.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:13.528+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:14.166 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:13 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T19:23:13.913+0000 7ff659b37700 -1 osd.4 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.587465+0000 front 2024-04-25T19:22:46.587496+0000 (oldest deadline 2024-04-25T19:23:12.487669+0000) 2024-04-25T19:23:14.166 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:14.105+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:14.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:14 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:14.142+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:14.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:14 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:14.344+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:14.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:14 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:14.344+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:14.488 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:14 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:14.142+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:14.488 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:14 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:14.142+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:14.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:14.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:14.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:14.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:14.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:14 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:14.486+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:14.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:14 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:14.486+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:14.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:14 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:14.571+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:14.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:14 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:14.571+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:15.166 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:14 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4[43883]: 2024-04-25T19:23:14.905+0000 7ff659b37700 -1 osd.4 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.587465+0000 front 2024-04-25T19:22:46.587496+0000 (oldest deadline 2024-04-25T19:23:12.487669+0000) 2024-04-25T19:23:15.167 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:15 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:15.073+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:15.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:15 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:15.169+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:15.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:15 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:15.302+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:15.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:15 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:15.302+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:15.531 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:15 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:15.174+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:15.532 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:15 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:15.174+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:15.618 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:15 smithi183 podman[71447]: 2024-04-25 19:23:15.315409778 +0000 UTC m=+5.537905717 container died d9d9c13d1a56fa8dd7d1d046e258412d52fbb0c11291a821c47f65f2cfdf0262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T19:23:15.618 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:15 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:15.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:15.619 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:15 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:15.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:15.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:15 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:15.529+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:15.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:15 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:15.529+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:15.916 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:15 smithi183 podman[71447]: 2024-04-25 19:23:15.716580205 +0000 UTC m=+5.939076126 container cleanup d9d9c13d1a56fa8dd7d1d046e258412d52fbb0c11291a821c47f65f2cfdf0262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T19:23:15.916 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:15 smithi183 bash[71447]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4 2024-04-25T19:23:15.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:15 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:15.533+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:15.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:15 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:15.533+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:16.287 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71478]: 2024-04-25 19:23:16.010132638 +0000 UTC m=+0.686107352 container remove d9d9c13d1a56fa8dd7d1d046e258412d52fbb0c11291a821c47f65f2cfdf0262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:23:16.287 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:16 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:16.061+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:16.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:16.164+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:16.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:16.300+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:16.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:16.300+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:16.495 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:16 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:16.212+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:16.496 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:16 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:16.212+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:16.581 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:16 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:16.285+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:16.581 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:16 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:16.285+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:16.581 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71570]: 2024-04-25 19:23:16.203879809 +0000 UTC m=+0.018328937 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:16.581 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71570]: 2024-04-25 19:23:16.327770903 +0000 UTC m=+0.142220021 container create 3ce7c655c83ad4510c839dce27b3133ec7d90f175856ab65fb0e9d1d4705b971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T19:23:16.581 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71570]: 2024-04-25 19:23:16.580798633 +0000 UTC m=+0.395247755 container init 3ce7c655c83ad4510c839dce27b3133ec7d90f175856ab65fb0e9d1d4705b971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T19:23:16.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:16 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:16.493+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:16.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:16 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:16.493+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:16.873 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71570]: 2024-04-25 19:23:16.583963127 +0000 UTC m=+0.398412245 container start 3ce7c655c83ad4510c839dce27b3133ec7d90f175856ab65fb0e9d1d4705b971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T19:23:16.873 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71570]: 2024-04-25 19:23:16.706056294 +0000 UTC m=+0.520505427 container attach 3ce7c655c83ad4510c839dce27b3133ec7d90f175856ab65fb0e9d1d4705b971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T19:23:16.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:16.559+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:16.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:16 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:16.559+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:17.166 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:17 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:17.042+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:17.166 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 25 19:23:16 smithi183 podman[71570]: 2024-04-25 19:23:16.872729931 +0000 UTC m=+0.687179053 container died 3ce7c655c83ad4510c839dce27b3133ec7d90f175856ab65fb0e9d1d4705b971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:23:17.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:17.127+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:17.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:17.275+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:17.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:17.275+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:17.540 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:17 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:17.230+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:17.540 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:17 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:17.230+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:17.543 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:17 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:17.270+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:17.544 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:17 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:17.270+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:17.571 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T19:23:17.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:17.653 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T19:23:17.654 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T19:23:17.654 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.5 2024-04-25T19:23:17.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:17 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:17.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:17.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:17 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:17.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:17.972 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:17.538+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:17.972 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:17.538+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:17.972 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:17 smithi194 systemd[1]: Stopping Ceph osd.5 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:18.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:18 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:18.174+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:18.231 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:17.970+0000 7f23fc8df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:18.232 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:17.970+0000 7f23fc8df700 -1 osd.5 125 *** Got signal Terminated *** 2024-04-25T19:23:18.232 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:17 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:17.970+0000 7f23fc8df700 -1 osd.5 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:18.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:18 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:18.085+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:18.416 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:18 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:18.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:18.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:18 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:18.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:18.531 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:18 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:18.271+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:18.531 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:18 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:18.271+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:18.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:18 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:18.301+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:18.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:18 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:18.301+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:18.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:18 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:18.513+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:18.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:18 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:18.513+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:18.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:18 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:18.513+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.010734+0000 front 2024-04-25T19:22:57.010687+0000 (oldest deadline 2024-04-25T19:23:18.110998+0000) 2024-04-25T19:23:18.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:18 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:18.529+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:18.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:18 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:18.529+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:18.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:18 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:18.529+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:19.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:19 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:19.121+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:19.416 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:19 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:19.298+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:19.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:19 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:19.298+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:19.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:19.254+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:19.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:19.254+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:19.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:19.165+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:19.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:19.165+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:19.555 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:19.255+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:19.555 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:19.255+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:19.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:19.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:19.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:19.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:19.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:19 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:19.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:19.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:19.553+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:19.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:19.553+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:19.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:19 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:19.553+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.010734+0000 front 2024-04-25T19:22:57.010687+0000 (oldest deadline 2024-04-25T19:23:18.110998+0000) 2024-04-25T19:23:20.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:20 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:20.105+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:20.417 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:20 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:20.105+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:20.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:20 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:20.265+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:20.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:20 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:20.265+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:20.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:20.209+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:20.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:20.209+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:20.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:20.162+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:20.482 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:20.162+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:20.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:20 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:20.520+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:20.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:20 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:20.520+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:20.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:20 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:20.520+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:20.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:20 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:20.295+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:20.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:20 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:20.295+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:20.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:20 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:20.295+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:20.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:20.512+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:20.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:20.512+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:20.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:20 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:20.512+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.010734+0000 front 2024-04-25T19:22:57.010687+0000 (oldest deadline 2024-04-25T19:23:18.110998+0000) 2024-04-25T19:23:21.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:21 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:21.148+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:21.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:21 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:21.148+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:21.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:21 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:21.270+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:21.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:21 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:21.270+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:21.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:21 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:21.270+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:21.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:21.157+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:21.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:21.157+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:21.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:21.159+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:21.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:21.159+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:21.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:21 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:21.320+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:21.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:21 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:21.320+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:21.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:21 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:21.320+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:21.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:21 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:21.513+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:21.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:21 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:21.513+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:21.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:21 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:21.513+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:21.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:21.540+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:21.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:21.540+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:21.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:21 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:21.540+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.010734+0000 front 2024-04-25T19:22:57.010687+0000 (oldest deadline 2024-04-25T19:23:18.110998+0000) 2024-04-25T19:23:22.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:22 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:22.162+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:22.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:22 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:22.162+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:22.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:22 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:22.223+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:22.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:22 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:22.223+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:22.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:22 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:22.223+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:22.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:22.116+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:22.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:22.116+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:22.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:22.149+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:22.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:22.149+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:22.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:22.149+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:22.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:22.312+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:22.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:22.312+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:22.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:22.312+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:22.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:22.466+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:22.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:22.466+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:22.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:22.466+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:22.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:22.517+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:39.408466+0000 front 2024-04-25T19:22:39.408596+0000 (oldest deadline 2024-04-25T19:23:04.108410+0000) 2024-04-25T19:23:22.981 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:22.517+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:44.109067+0000 front 2024-04-25T19:22:44.108990+0000 (oldest deadline 2024-04-25T19:23:12.309781+0000) 2024-04-25T19:23:22.982 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:22 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5[44665]: 2024-04-25T19:23:22.517+0000 7f23f8ed3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.010734+0000 front 2024-04-25T19:22:57.010687+0000 (oldest deadline 2024-04-25T19:23:18.110998+0000) 2024-04-25T19:23:23.475 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:23 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:23.091+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:23.475 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:23 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:23.091+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:23.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:23 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:23.114+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:23.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:23 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:23.114+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:23.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:23 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:23.114+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:23.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:23 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:23.204+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:23.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:23 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:23.204+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:23.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:23 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:23.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:23.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:23 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:23.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:23.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:23 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:23.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:23.731 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:23 smithi194 podman[71685]: 2024-04-25 19:23:23.474936309 +0000 UTC m=+5.627696243 container stop 6f613c386e9be10359e672623fe944288cc9b6958a4613a41f6dfaad5b1ed6fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T19:23:23.732 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:23 smithi194 podman[71685]: 2024-04-25 19:23:23.475659023 +0000 UTC m=+5.628418961 container died 6f613c386e9be10359e672623fe944288cc9b6958a4613a41f6dfaad5b1ed6fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T19:23:23.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:23.337+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:23.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:23.337+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:23.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:23.337+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:23.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:23.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:23.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:23.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:23.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:23 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:23.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:24.066 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:23 smithi194 podman[71685]: 2024-04-25 19:23:23.887082452 +0000 UTC m=+6.039842379 container cleanup 6f613c386e9be10359e672623fe944288cc9b6958a4613a41f6dfaad5b1ed6fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T19:23:24.066 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:23 smithi194 bash[71685]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5 2024-04-25T19:23:24.481 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:24 smithi194 podman[71702]: 2024-04-25 19:23:24.14679945 +0000 UTC m=+0.664607163 container remove 6f613c386e9be10359e672623fe944288cc9b6958a4613a41f6dfaad5b1ed6fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T19:23:24.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:24 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:24.064+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:24.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:24 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:24.064+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:24.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:24 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:24.117+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:24.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:24 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:24.117+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:24.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:24 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:24.117+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:24.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:24 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:24.305+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:24.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:24 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:24.305+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:24.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:24 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:24.305+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:24.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:24 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:24.519+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:24.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:24 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:24.519+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:24.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:24 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:24.519+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:24.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:24 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:24.253+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:24.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:24 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:24.253+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:24.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:24 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:24.288+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:24.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:24 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:24.288+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:24.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:24 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:24.288+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:24.768 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:24 smithi194 podman[71792]: 2024-04-25 19:23:24.401264029 +0000 UTC m=+0.017788619 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:24.768 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:24 smithi194 podman[71792]: 2024-04-25 19:23:24.539709675 +0000 UTC m=+0.156234263 container create f661e5560655921f4b7c8b8d2ebedf4ac1c94ceb267cfdc373665c5fbf41528e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T19:23:25.018 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:24 smithi194 podman[71792]: 2024-04-25 19:23:24.767365575 +0000 UTC m=+0.383890167 container init f661e5560655921f4b7c8b8d2ebedf4ac1c94ceb267cfdc373665c5fbf41528e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T19:23:25.019 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:24 smithi194 podman[71792]: 2024-04-25 19:23:24.770575103 +0000 UTC m=+0.387099693 container start f661e5560655921f4b7c8b8d2ebedf4ac1c94ceb267cfdc373665c5fbf41528e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T19:23:25.019 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:24 smithi194 podman[71792]: 2024-04-25 19:23:24.867295862 +0000 UTC m=+0.483820461 container attach f661e5560655921f4b7c8b8d2ebedf4ac1c94ceb267cfdc373665c5fbf41528e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:23:25.421 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 25 19:23:25 smithi194 podman[71792]: 2024-04-25 19:23:25.080151493 +0000 UTC m=+0.696676082 container died f661e5560655921f4b7c8b8d2ebedf4ac1c94ceb267cfdc373665c5fbf41528e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:23:25.421 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:25 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:25.017+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:25.421 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:25 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:25.017+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:25.422 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:25 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:25.124+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:25.422 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:25 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:25.124+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:25.422 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:25 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:25.124+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:25.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:25 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:25.209+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:25.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:25 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:25.209+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:25.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:25 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:25.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:25.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:25 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:25.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:25.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:25 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:25.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:25.748 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T19:23:25.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:25.787 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T19:23:25.787 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T19:23:25.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.6 2024-04-25T19:23:25.819 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:25.334+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:25.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:25.334+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:25.820 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:25.334+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:25.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:25.562+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:25.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:25.562+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:25.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:25.562+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:26.305 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:25 smithi086 systemd[1]: Stopping Ceph osd.6 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:26.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:26 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:26.063+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:26.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:26 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:26.063+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:26.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:26 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:26.171+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:26.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:26 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:26.171+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:26.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:26 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:26.171+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:26.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:26.303+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:26.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:26.303+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:26.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:26.303+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:26.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:26.362+0000 7f9a76c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:26.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:26.362+0000 7f9a76c00700 -1 osd.6 125 *** Got signal Terminated *** 2024-04-25T19:23:26.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:26.362+0000 7f9a76c00700 -1 osd.6 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:26.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:26 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:26.240+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:26.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:26 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:26.240+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:26.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:26 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:26.340+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:26.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:26 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:26.340+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:26.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:26 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:26.340+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:27.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:26.573+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:27.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:26.573+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:27.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:26 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:26.573+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:27.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:27 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:27.042+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:27.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:27 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:27.042+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:27.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:27 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:27.199+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:27.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:27 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:27.199+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:27.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:27 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:27.199+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:27.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:27 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:27.272+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:27.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:27 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:27.272+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:27.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:27 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:27.272+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:27.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:27.256+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:27.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:27.256+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:27.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:27.256+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:27.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:27.324+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:27.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:27.324+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:27.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:27 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:27.324+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:28.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:27 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:27.598+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:28.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:27 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:27.598+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:28.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:27 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:27.598+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:28.382 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:28.062+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:28.382 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:28.062+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:28.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:28.170+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:28.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:28.170+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:28.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:28 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:28.170+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:28.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:28.261+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:28.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:28.261+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:28.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:28.261+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:28.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:28 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:28.248+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:28.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:28 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:28.248+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:28.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:28 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:28.248+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:28.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:28 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:28.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:28.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:28 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:28.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:28.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:28 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:28.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:29.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:28.583+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:29.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:28.583+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:29.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:28 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:28.583+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:29.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:29 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:29.073+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:29.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:29 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:29.073+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:29.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:29 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:29.207+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:29.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:29 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:29.207+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:29.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:29 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:29.207+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:29.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:29.256+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:29.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:29.256+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:29.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:29.256+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:29.571 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:29.256+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:03.933524+0000 front 2024-04-25T19:23:03.933893+0000 (oldest deadline 2024-04-25T19:23:28.633425+0000) 2024-04-25T19:23:29.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:29 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:29.245+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:29.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:29 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:29.245+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:29.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:29 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:29.245+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:29.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:29 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:29.199+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:29.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:29 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:29.199+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:29.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:29 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:29.199+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:30.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:29.584+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:30.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:29.584+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:30.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:29.584+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:30.433 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:30.282+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:30.433 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:30.282+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:30.433 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:30.282+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:30.433 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:30.282+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:03.933524+0000 front 2024-04-25T19:23:03.933893+0000 (oldest deadline 2024-04-25T19:23:28.633425+0000) 2024-04-25T19:23:30.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:30 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:30.100+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:30.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:30 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:30.100+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:30.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:30 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:30.203+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:30.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:30 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:30.203+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:30.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:30 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:30.203+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:30.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:30.247+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:30.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:30.247+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:30.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:30.247+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:30.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:30.253+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:30.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:30.253+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:30.668 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:30.253+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:30.668 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:30 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:30.253+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:30.756 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:30.539+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:30.756 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:30.539+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:30.756 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:30 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:30.539+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:31.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:31 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:31.062+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:31.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:31 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:31.062+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:31.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:31 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:31.062+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:31.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:31 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:31.182+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:31.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:31 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:31.182+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:31.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:31 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:31.182+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:31.659 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:31.506+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:31.659 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:31.506+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:31.659 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:31.506+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:31.659 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:31.506+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:31.659 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:31.330+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.224565+0000 front 2024-04-25T19:22:40.224408+0000 (oldest deadline 2024-04-25T19:23:05.524500+0000) 2024-04-25T19:23:31.660 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:31.330+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.727550+0000 front 2024-04-25T19:22:47.727129+0000 (oldest deadline 2024-04-25T19:23:11.227303+0000) 2024-04-25T19:23:31.660 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:31.330+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.129691+0000 front 2024-04-25T19:22:57.129477+0000 (oldest deadline 2024-04-25T19:23:20.029628+0000) 2024-04-25T19:23:31.660 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:31 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6[53859]: 2024-04-25T19:23:31.330+0000 7f9a6d400700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:03.933524+0000 front 2024-04-25T19:23:03.933893+0000 (oldest deadline 2024-04-25T19:23:28.633425+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:31.232+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:31.232+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:31.232+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:31.303+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:31.303+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:31.303+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:31.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:31 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:31.303+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:32.270 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:31 smithi086 podman[106987]: 2024-04-25 19:23:31.926061177 +0000 UTC m=+5.712476438 container died a3303775e4b9b2be2a73572ae48ab2938dd2e5febf3c5716aee0499a804144c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T19:23:32.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:32 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:32.043+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:32.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:32 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:32.043+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:32.482 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:32 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:32.043+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:32.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:32 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:32.203+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:32.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:32 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:32.203+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:32.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:32 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:32.203+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:32.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:32 smithi086 podman[106987]: 2024-04-25 19:23:32.397312517 +0000 UTC m=+6.183727818 container cleanup a3303775e4b9b2be2a73572ae48ab2938dd2e5febf3c5716aee0499a804144c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:23:32.570 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:32 smithi086 bash[106987]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6 2024-04-25T19:23:32.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:32.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:32.570 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:32.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:32.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:32.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:32.571 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:32 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:32.510+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:32.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:32.235+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:32.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:32.235+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:32.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:32.235+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:32.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:32.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:32.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:32.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:32.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:32.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:32.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:32 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:32.332+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:33.070 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:32 smithi086 podman[107017]: 2024-04-25 19:23:32.600661619 +0000 UTC m=+0.661014194 container remove a3303775e4b9b2be2a73572ae48ab2938dd2e5febf3c5716aee0499a804144c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T19:23:33.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:33.021+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:33.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:33.021+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:33.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:33.021+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:33.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:33.163+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:33.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:33.163+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:33.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:33.163+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:33.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:33.245+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:33.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:33.245+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:33.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:33.245+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:33.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:33.325+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:33.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:33.325+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:33.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:33.325+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:33.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:33 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:33.325+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:33 smithi086 podman[107104]: 2024-04-25 19:23:33.424096065 +0000 UTC m=+0.106611553 container create d784ed52d42a7aee5c1f56b8f69a0e24baceca61e05a0a7ad51f23a7cbea3bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:33 smithi086 podman[107104]: 2024-04-25 19:23:33.349594863 +0000 UTC m=+0.032110337 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:33 smithi086 podman[107104]: 2024-04-25 19:23:33.727547718 +0000 UTC m=+0.410063232 container init d784ed52d42a7aee5c1f56b8f69a0e24baceca61e05a0a7ad51f23a7cbea3bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:33.472+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:33.472+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:33.472+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:33.729 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:33 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:33.472+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:34.070 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:33 smithi086 podman[107104]: 2024-04-25 19:23:33.738930922 +0000 UTC m=+0.421446457 container start d784ed52d42a7aee5c1f56b8f69a0e24baceca61e05a0a7ad51f23a7cbea3bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T19:23:34.070 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:33 smithi086 podman[107104]: 2024-04-25 19:23:33.818132444 +0000 UTC m=+0.500648012 container attach d784ed52d42a7aee5c1f56b8f69a0e24baceca61e05a0a7ad51f23a7cbea3bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T19:23:34.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:33.972+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:34.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:33.972+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:34.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:33 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:33.972+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:34.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:34.131+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:34.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:34.131+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:34.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:34.131+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:34.517 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 25 19:23:34 smithi086 podman[107104]: 2024-04-25 19:23:34.189926536 +0000 UTC m=+0.872442101 container died d784ed52d42a7aee5c1f56b8f69a0e24baceca61e05a0a7ad51f23a7cbea3bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T19:23:34.567 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:34.515+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:34.567 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:34.515+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:34.567 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:34.515+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:34.567 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:34 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:34.515+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:34.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:34.206+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:34.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:34.206+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:34.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:34.206+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:34.668 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:34.367+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:34.668 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:34.367+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:34.668 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:34.367+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:34.668 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:34 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:34.367+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:34.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T19:23:35.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:35.136 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T19:23:35.137 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T19:23:35.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.7 2024-04-25T19:23:35.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:34.932+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:35.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:34.932+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:35.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:34 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:34.932+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:35.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:35 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:35.087+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:35.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:35 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:35.087+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:35.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:35 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:35.087+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:35.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 systemd[1]: Stopping Ceph osd.7 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:35.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:35.222+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:35.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:35.222+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:35.416 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:35.222+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:35.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:35.376+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:35.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:35.376+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:35.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:35.376+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:35.417 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:35.376+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:35.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:35.534+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:35.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:35.534+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:35.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:35.534+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:35.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:35 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:35.534+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:35.916 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:35.470+0000 7f77cec96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:35.916 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:35.470+0000 7f77cec96700 -1 osd.7 125 *** Got signal Terminated *** 2024-04-25T19:23:35.917 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:35.470+0000 7f77cec96700 -1 osd.7 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:36.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:35 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:35.888+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:36.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:35 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:35.888+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:36.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:35 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:35.888+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:36.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:36 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:36.105+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:36.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:36 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:36.105+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:36.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:36 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:36.105+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:36.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:36.180+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:36.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:36.180+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:36.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:36.181+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:36.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:36.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:36.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:36.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:36.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:36.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:36.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:36 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:36.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:36.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:36 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:36.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:36.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:36 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:36.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:36.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:36 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:36.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:36.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:36 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:36.553+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:37.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:36 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:36.891+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:37.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:36 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:36.891+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:37.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:36 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:36.891+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:37.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:37.075+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:37.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:37.075+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:37.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:37.075+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:37.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:37.198+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:37.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:37.198+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:37.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:37.198+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:37.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:37.295+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:37.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:37.295+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:37.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:37.295+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:37.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:37 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:37.295+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:37.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:37 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:37.505+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:37.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:37 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:37.505+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:37.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:37 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:37.505+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:37.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:37 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:37.505+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:38.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:38.103+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:38.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:38.103+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:38.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:38.103+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:38.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:37.893+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:38.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:37.893+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:38.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:37.893+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:38.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:37 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:37.893+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:38.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:38.233+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:38.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:38.233+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:38.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:38.233+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:38.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:38.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:38.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:38.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:38.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:38.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:38.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:38.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:38.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:38 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:38.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:38.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:38 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:38.455+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:38.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:38 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:38.455+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:38.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:38 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:38.455+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:38.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:38 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:38.455+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:38.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:38 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:38.456+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:39.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:38.860+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:39.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:38.860+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:39.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:38.860+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:39.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:38 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:38.860+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:39.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:39.067+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:39.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:39.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:39.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:39.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:39.232 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:39.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:39.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:39.196+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:39.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:39.196+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:39.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:39.196+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:39.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:39.345+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:39.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:39.345+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:39.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:39.345+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:39.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:39.345+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:39.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:39 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:39.345+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:39.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:39 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:39.449+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:39.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:39 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:39.449+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:39.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:39 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:39.449+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:39.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:39 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:39.449+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:39.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:39 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:39.449+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:39.981 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:39.852+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:39.981 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:39.852+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:39.981 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:39.852+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:39.981 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:39 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:39.852+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:40.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:40.088+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:40.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:40.088+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:40.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:40.088+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:40.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:40.088+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:40.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:40.225+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:47.171881+0000 front 2024-04-25T19:22:47.172077+0000 (oldest deadline 2024-04-25T19:23:10.672012+0000) 2024-04-25T19:23:40.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:40.225+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.373109+0000 front 2024-04-25T19:22:55.373162+0000 (oldest deadline 2024-04-25T19:23:20.073237+0000) 2024-04-25T19:23:40.666 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:40.225+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:04.075151+0000 front 2024-04-25T19:23:04.075012+0000 (oldest deadline 2024-04-25T19:23:26.374734+0000) 2024-04-25T19:23:40.667 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T19:23:40.225+0000 7f77cb28a700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.876562+0000 front 2024-04-25T19:23:13.876583+0000 (oldest deadline 2024-04-25T19:23:39.776595+0000) 2024-04-25T19:23:40.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:40.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:40.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:40.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:40.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:40.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:40.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:40.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:40.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:40.337+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:40.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:40 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:40.489+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:40.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:40 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:40.489+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:40.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:40 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:40.489+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:40.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:40 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:40.489+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:40.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:40 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:40.489+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:41.113 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:40 smithi183 podman[71677]: 2024-04-25 19:23:40.836299425 +0000 UTC m=+5.505590594 container died 2576ec10982d6a5eb1072f2814dd73a2c52d1d73518f0aad5b8e3d83f29b49c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T19:23:41.118 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:40.827+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:41.118 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:40.827+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:41.118 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:40.827+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:41.118 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:40 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:40.827+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:41.366 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:41 smithi183 podman[71677]: 2024-04-25 19:23:41.211928227 +0000 UTC m=+5.881219406 container cleanup 2576ec10982d6a5eb1072f2814dd73a2c52d1d73518f0aad5b8e3d83f29b49c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T19:23:41.366 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:41 smithi183 bash[71677]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7 2024-04-25T19:23:41.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:41.116+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:41.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:41.116+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:41.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:41.116+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:41.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:41.116+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:41.669 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:41 smithi183 podman[71694]: 2024-04-25 19:23:41.538441277 +0000 UTC m=+0.695524641 container remove 2576ec10982d6a5eb1072f2814dd73a2c52d1d73518f0aad5b8e3d83f29b49c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T19:23:41.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:41 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:41.364+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:41.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:41 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:41.364+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:41.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:41 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:41.364+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:41.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:41 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:41.364+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:41.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:41 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:41.364+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:41.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:41.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:41.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:41.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:41.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:41.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:41.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:41.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:41.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:41 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:41.538+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:42.120 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:41.851+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:42.121 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:41.851+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:42.121 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:41.851+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:42.121 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:41 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:41.851+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:42.166 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:41 smithi183 podman[71785]: 2024-04-25 19:23:41.788965532 +0000 UTC m=+0.017658697 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:42.166 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:41 smithi183 podman[71785]: 2024-04-25 19:23:41.915338086 +0000 UTC m=+0.144031220 container create 2cb567a1d3a4b5966925783e282dbfac239a924a11297b12cc3f50045f29045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:23:42.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:42.119+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:42.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:42.119+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:42.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:42.119+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:42.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:42.119+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:42 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:42.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:42 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:42.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:42 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:42.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:42 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:42.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:42 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:42.322+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:42 smithi183 podman[71785]: 2024-04-25 19:23:42.211415165 +0000 UTC m=+0.440108299 container init 2cb567a1d3a4b5966925783e282dbfac239a924a11297b12cc3f50045f29045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:42 smithi183 podman[71785]: 2024-04-25 19:23:42.214807697 +0000 UTC m=+0.443500838 container start 2cb567a1d3a4b5966925783e282dbfac239a924a11297b12cc3f50045f29045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:23:42.505 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:42 smithi183 podman[71785]: 2024-04-25 19:23:42.318043757 +0000 UTC m=+0.546736895 container attach 2cb567a1d3a4b5966925783e282dbfac239a924a11297b12cc3f50045f29045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T19:23:42.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 25 19:23:42 smithi183 podman[71785]: 2024-04-25 19:23:42.504702167 +0000 UTC m=+0.733395312 container died 2cb567a1d3a4b5966925783e282dbfac239a924a11297b12cc3f50045f29045e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T19:23:43.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:42.578+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:43.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:42.578+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:43.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:42.578+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:43.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:42.578+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:43.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:42 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:42.578+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:43.098 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:42.838+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:43.099 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:42.838+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:43.099 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:42.838+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:43.099 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:42 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:42.838+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:43.190 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T19:23:43.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:43.231 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T19:23:43.231 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T19:23:43.231 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.8 2024-04-25T19:23:43.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:43 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:43.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:43.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:43 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:43.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:43.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:43 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:43.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:43.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:43 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:43.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:43.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:43 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:43.319+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:43.481 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 systemd[1]: Stopping Ceph osd.8 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:43.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:43.097+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:43.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:43.097+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:43.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:43.097+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:43.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:43.097+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:43.835 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.564+0000 7f38319f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:43.835 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.564+0000 7f38319f3700 -1 osd.8 125 *** Got signal Terminated *** 2024-04-25T19:23:43.835 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.564+0000 7f38319f3700 -1 osd.8 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:43.835 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.834+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:43.836 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.834+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:44.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:43 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:43.613+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:44.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:43 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:43.613+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:44.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:43 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:43.613+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:44.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:43 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:43.613+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:44.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:43 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:43.613+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:44.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.834+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:44.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:43 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:43.834+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:44.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:44.072+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:44.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:44.072+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:44.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:44.072+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:44.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:44.072+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:44.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:44.280+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:44.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:44.280+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:44.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:44.280+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:44.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:44.280+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:44.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:44 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:44.280+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:45.061 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:44.796+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:45.061 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:44.796+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:45.061 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:44.796+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:45.062 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:44 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:44.796+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:45.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:44.589+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:45.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:44.589+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:45.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:44.589+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:45.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:44.589+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:45.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:44 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:44.589+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:45.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:45.060+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:45.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:45.060+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:45.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:45.060+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:45.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:45.060+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:45.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:45.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:45.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:45.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:45.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:45.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:45.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:45.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:45.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:45.279+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:46.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:45.594+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:46.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:45.594+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:46.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:45.594+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:46.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:45.594+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:46.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:45 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:45.594+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:46.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:46.081+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:46.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:46.081+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:46.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:46.081+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:46.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:46.081+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:46.231 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:45.843+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:46.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:45.843+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:46.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:45.843+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:46.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:45.843+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:46.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:45 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:45.843+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.297143+0000 front 2024-04-25T19:23:22.297107+0000 (oldest deadline 2024-04-25T19:23:45.097656+0000) 2024-04-25T19:23:46.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:46 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:46.276+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:46.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:46 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:46.276+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:46.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:46 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:46.276+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:46.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:46 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:46.276+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:46.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:46 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:46.276+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:47.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:46.608+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:47.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:46.608+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:47.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:46.608+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:47.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:46.608+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:47.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:46.608+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:47.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:46 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:46.608+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:47.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:47.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:47.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:47.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:47.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:47.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:47.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:47.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:47.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:46.871+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:47.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:46.871+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:47.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:46.871+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:47.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:46.871+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:47.232 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:46 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:46.871+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.297143+0000 front 2024-04-25T19:23:22.297107+0000 (oldest deadline 2024-04-25T19:23:45.097656+0000) 2024-04-25T19:23:47.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:47.232+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:47.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:47.232+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:47.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:47.232+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:47.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:47.232+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:47.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:47 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:47.232+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:48.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:47.624+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:48.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:47.624+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:48.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:47.624+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:48.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:47.624+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:48.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:47.624+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:48.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:47 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:47.624+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:48.234 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:48 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:48.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:48.234 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:48 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:48.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:48.234 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:48 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:48.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:48.234 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:48 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:48.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:48.235 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:47.867+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.692364+0000 front 2024-04-25T19:22:46.692319+0000 (oldest deadline 2024-04-25T19:23:10.692899+0000) 2024-04-25T19:23:48.235 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:47.867+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.993783+0000 front 2024-04-25T19:22:55.993822+0000 (oldest deadline 2024-04-25T19:23:18.293974+0000) 2024-04-25T19:23:48.235 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:47.867+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.295924+0000 front 2024-04-25T19:23:05.295971+0000 (oldest deadline 2024-04-25T19:23:30.595458+0000) 2024-04-25T19:23:48.235 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:47.867+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.896526+0000 front 2024-04-25T19:23:12.896403+0000 (oldest deadline 2024-04-25T19:23:37.596340+0000) 2024-04-25T19:23:48.235 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:47 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8[51593]: 2024-04-25T19:23:47.867+0000 7f382dfe7700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.297143+0000 front 2024-04-25T19:23:22.297107+0000 (oldest deadline 2024-04-25T19:23:45.097656+0000) 2024-04-25T19:23:48.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:48.273+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:48.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:48.273+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:48.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:48.273+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:48.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:48.273+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:48.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:48 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:48.273+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:49.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:48.575+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:49.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:48.575+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:49.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:48.575+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:49.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:48.575+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:49.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:48.575+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:49.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:48 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:48.575+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:49.382 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:49 smithi194 podman[71912]: 2024-04-25 19:23:49.10128104 +0000 UTC m=+5.674979689 container died f95b0a6d7aa0a56b614fd3f367140af8f1717ca94051cc0ca14035efd20499e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:23:49.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.022+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:49.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.022+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:49.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.022+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:49.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.022+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:49.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.022+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:49.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:49.248+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:49.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:49.248+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:49.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:49.248+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:49.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:49.248+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:49.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:49.248+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:49.670 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:49.248+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:49.731 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:49 smithi194 podman[71912]: 2024-04-25 19:23:49.480946767 +0000 UTC m=+6.054645419 container cleanup f95b0a6d7aa0a56b614fd3f367140af8f1717ca94051cc0ca14035efd20499e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T19:23:49.731 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:49 smithi194 bash[71912]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8 2024-04-25T19:23:50.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:49.595+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:50.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:49.595+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:50.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:49.595+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:50.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:49.595+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:50.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:49.595+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:50.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:49 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:49.595+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:50.101 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:49 smithi194 podman[71929]: 2024-04-25 19:23:49.77372248 +0000 UTC m=+0.671510796 container remove f95b0a6d7aa0a56b614fd3f367140af8f1717ca94051cc0ca14035efd20499e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:23:50.101 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.978+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:50.101 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.978+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:50.101 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.978+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:50.101 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.978+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:50.101 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:49 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:49.978+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:50.353 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:50 smithi194 podman[72023]: 2024-04-25 19:23:50.003699367 +0000 UTC m=+0.018112804 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:50.353 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:50 smithi194 podman[72023]: 2024-04-25 19:23:50.132795858 +0000 UTC m=+0.147209288 container create c0c33cbf6d9c6786a66ad85d21325b1405a87696bc581d4e36e121737e80abbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T19:23:50.353 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:50 smithi194 podman[72023]: 2024-04-25 19:23:50.352357992 +0000 UTC m=+0.366771425 container init c0c33cbf6d9c6786a66ad85d21325b1405a87696bc581d4e36e121737e80abbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T19:23:50.647 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:50 smithi194 podman[72023]: 2024-04-25 19:23:50.355621725 +0000 UTC m=+0.370035153 container start c0c33cbf6d9c6786a66ad85d21325b1405a87696bc581d4e36e121737e80abbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T19:23:50.647 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:50 smithi194 podman[72023]: 2024-04-25 19:23:50.43564436 +0000 UTC m=+0.450057797 container attach c0c33cbf6d9c6786a66ad85d21325b1405a87696bc581d4e36e121737e80abbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T19:23:50.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:50.228+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:50.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:50.228+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:50.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:50.228+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:50.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:50.228+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:50.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:50.228+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:50.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:50 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:50.228+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:50.981 INFO:journalctl@ceph.osd.8.smithi194.stdout:Apr 25 19:23:50 smithi194 podman[72023]: 2024-04-25 19:23:50.646858656 +0000 UTC m=+0.661272100 container died c0c33cbf6d9c6786a66ad85d21325b1405a87696bc581d4e36e121737e80abbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True) 2024-04-25T19:23:51.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:50.632+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:51.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:50.632+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:51.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:50.632+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:51.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:50.632+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:51.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:50.632+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:51.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:50 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:50.632+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:51.298 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:51.016+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:51.298 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:51.016+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:51.298 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:51.016+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:51.298 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:51.016+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:51.298 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:51 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:51.016+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:51.327 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T19:23:51.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:23:51.365 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T19:23:51.365 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T19:23:51.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.9 2024-04-25T19:23:51.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:51.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:51.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:51.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:51.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:51.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:51.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:51.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:51.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:51.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:51.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:51 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:51.244+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:51.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 systemd[1]: Stopping Ceph osd.9 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:23:51.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.621+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:51.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.621+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:51.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.621+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:51.820 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.621+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:51.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.621+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:51.821 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.621+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:52.320 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.980+0000 7f1bffa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:23:52.320 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.980+0000 7f1bffa00700 -1 osd.9 125 *** Got signal Terminated *** 2024-04-25T19:23:52.320 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:51 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:51.980+0000 7f1bffa00700 -1 osd.9 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:23:52.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:52.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:52.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:52.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:52.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:52.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:52.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:52.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:52.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:52 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:52.052+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:52.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:52.199+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:52.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:52.199+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:52.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:52.199+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:52.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:52.199+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:52.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:52.199+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:52.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:52 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:52.199+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:53.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:52.649+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:53.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:52.649+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:53.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:52.649+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:53.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:52.649+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:53.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:52.649+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:53.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:52 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:52.649+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:53.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:53.079+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:53.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:53.079+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:53.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:53.079+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:53.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:53.079+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:53.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:53.079+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:53.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:53.079+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:53.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:53.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:53.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:53.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:53.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:53.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:53.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:53 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:53.197+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:54.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:53.604+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:54.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:53.604+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:54.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:53.604+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:54.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:53.604+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:54.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:53.604+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:54.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:53 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:53.604+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:54.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:54.051+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:54.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:54.051+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:54.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:54.051+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:54.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:54.051+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:54.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:54.051+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:54.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:54 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:54.051+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:54.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:54.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:54.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:54.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:54.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:54.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:54.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:54 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:54.187+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:55.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:54.628+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:55.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:54.628+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:55.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:54.628+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:55.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:54.628+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:55.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:54.628+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:55.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:54 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:54.628+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:55.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:55.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:55.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:55.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:55.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:55.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:55.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:55.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:55.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:55.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:55.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:55 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:55.077+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:55.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:55.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:55.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:55.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:55.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:55.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:55.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:55 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:55.198+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:56.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:55.645+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:56.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:55.645+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:56.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:55.645+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:56.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:55.645+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:56.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:55.645+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:56.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:55 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:55.645+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:56.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:56.118+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:56.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:56.118+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:56.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:56.118+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:56.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:56.118+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:56.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:56.118+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:56.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:56 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:56.118+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:56.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.215+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:56.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.215+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:56.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.215+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:56.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.215+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:56.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.215+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:56.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.216+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:56.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:56 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:56.216+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:57.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:56.693+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:40.529859+0000 front 2024-04-25T19:22:40.529585+0000 (oldest deadline 2024-04-25T19:23:01.028829+0000) 2024-04-25T19:23:57.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:56.693+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:49.033114+0000 front 2024-04-25T19:22:49.033069+0000 (oldest deadline 2024-04-25T19:23:10.133264+0000) 2024-04-25T19:23:57.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:56.693+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:55.936344+0000 front 2024-04-25T19:22:55.936221+0000 (oldest deadline 2024-04-25T19:23:18.236363+0000) 2024-04-25T19:23:57.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:56.693+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.239629+0000 front 2024-04-25T19:23:05.239946+0000 (oldest deadline 2024-04-25T19:23:31.139382+0000) 2024-04-25T19:23:57.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:56.693+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:14.041033+0000 front 2024-04-25T19:23:14.041436+0000 (oldest deadline 2024-04-25T19:23:38.140864+0000) 2024-04-25T19:23:57.071 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:56 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9[60900]: 2024-04-25T19:23:56.693+0000 7f1bfb200700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.842745+0000 front 2024-04-25T19:23:22.842685+0000 (oldest deadline 2024-04-25T19:23:46.342974+0000) 2024-04-25T19:23:57.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:57.076+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:57.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:57.076+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:57.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:57.076+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:57.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:57.076+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:57.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:57.076+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:57.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:57.076+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:57.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:57.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:57.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:57.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:57.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:57.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:57.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:57 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:57.226+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:57.748 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:57 smithi086 podman[107208]: 2024-04-25 19:23:57.486227004 +0000 UTC m=+5.649275102 container died b4d06c7b4088c0c13ddd19eaed46b005e785646b8df73bc2094163a300333853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:23:58.002 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:57 smithi086 podman[107208]: 2024-04-25 19:23:57.973045941 +0000 UTC m=+6.136094067 container cleanup b4d06c7b4088c0c13ddd19eaed46b005e785646b8df73bc2094163a300333853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:23:58.003 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:57 smithi086 bash[107208]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9 2024-04-25T19:23:58.320 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:58 smithi086 podman[107227]: 2024-04-25 19:23:58.199617492 +0000 UTC m=+0.703721845 container remove b4d06c7b4088c0c13ddd19eaed46b005e785646b8df73bc2094163a300333853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T19:23:58.382 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:58.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:58.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:58.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:58.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:58.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:58.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:58.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:58.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:58.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:58.383 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:58 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:58.068+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:58.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:58.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:58.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:58.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:58.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:58.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:58.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:58 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:58.258+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:59.320 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:59 smithi086 podman[107320]: 2024-04-25 19:23:58.962999885 +0000 UTC m=+0.032212240 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:23:59.320 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:59 smithi086 podman[107320]: 2024-04-25 19:23:59.075273195 +0000 UTC m=+0.144485572 container create 15d07b1e95cbb0de3e0109017278062374fe812f4a8f7746bf51c746cfd1968e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T19:23:59.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:59.073+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:23:59.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:59.073+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:23:59.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:59.073+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:23:59.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:59.073+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:23:59.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:59.073+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:23:59.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:23:59 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:23:59.073+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:23:59.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:23:59.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:23:59.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:23:59.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:23:59.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:23:59.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:23:59.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:23:59 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:23:59.278+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:23:59.689 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:59 smithi086 podman[107320]: 2024-04-25 19:23:59.336885873 +0000 UTC m=+0.406098265 container init 15d07b1e95cbb0de3e0109017278062374fe812f4a8f7746bf51c746cfd1968e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T19:23:59.689 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:59 smithi086 podman[107320]: 2024-04-25 19:23:59.34871236 +0000 UTC m=+0.417924778 container start 15d07b1e95cbb0de3e0109017278062374fe812f4a8f7746bf51c746cfd1968e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T19:23:59.689 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:59 smithi086 podman[107320]: 2024-04-25 19:23:59.435996711 +0000 UTC m=+0.505209070 container attach 15d07b1e95cbb0de3e0109017278062374fe812f4a8f7746bf51c746cfd1968e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T19:24:00.070 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 25 19:23:59 smithi086 podman[107320]: 2024-04-25 19:23:59.81826983 +0000 UTC m=+0.887482178 container died 15d07b1e95cbb0de3e0109017278062374fe812f4a8f7746bf51c746cfd1968e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:24:00.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:00.121+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:00.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:00.121+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:00.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:00.121+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:00.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:00.121+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:00.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:00.121+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:00.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:00 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:00.121+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:00.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T19:24:00.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:00.584 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T19:24:00.584 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T19:24:00.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.10 2024-04-25T19:24:00.611 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:24:00.611 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:24:00.611 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:24:00.611 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:24:00.612 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:24:00.612 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:24:00.612 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.315+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:24:00.893 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 systemd[1]: Stopping Ceph osd.10 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:01.166 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.892+0000 7f624c884700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:01.166 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.892+0000 7f624c884700 -1 osd.10 125 *** Got signal Terminated *** 2024-04-25T19:24:01.166 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:00 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:00.892+0000 7f624c884700 -1 osd.10 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:01.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:01.162+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:01.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:24:01.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:24:01.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:24:01.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:24:01.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:24:01.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:24:01.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:01 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:01.352+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:02.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:02 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:02.120+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:02.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:24:02.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:24:02.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:24:02.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:24:02.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:24:02.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:24:02.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:02 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:02.311+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:03.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:03 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:03.109+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:03.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:24:03.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:24:03.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:24:03.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:24:03.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:24:03.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:24:03.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:03 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:03.291+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:24:04.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:04.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:04.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:04.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:04.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:04.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:04.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:04 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:04.130+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:04.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:24:04.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:24:04.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:24:04.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:24:04.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:24:04.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:24:04.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:04 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:04.296+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:24:05.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:05.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:05.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:05.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:05.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:05.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:05.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:05.152+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:05.666 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.552466+0000 front 2024-04-25T19:22:38.552606+0000 (oldest deadline 2024-04-25T19:23:01.452401+0000) 2024-04-25T19:24:05.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:46.753256+0000 front 2024-04-25T19:22:46.753292+0000 (oldest deadline 2024-04-25T19:23:10.853471+0000) 2024-04-25T19:24:05.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.056082+0000 front 2024-04-25T19:22:57.055969+0000 (oldest deadline 2024-04-25T19:23:20.555954+0000) 2024-04-25T19:24:05.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-25T19:23:05.058258+0000 front 2024-04-25T19:23:05.058090+0000 (oldest deadline 2024-04-25T19:23:29.757836+0000) 2024-04-25T19:24:05.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:12.658625+0000 front 2024-04-25T19:23:12.658659+0000 (oldest deadline 2024-04-25T19:23:37.958782+0000) 2024-04-25T19:24:05.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.560579+0000 front 2024-04-25T19:23:22.560505+0000 (oldest deadline 2024-04-25T19:23:48.960744+0000) 2024-04-25T19:24:05.667 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:05 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10[57472]: 2024-04-25T19:24:05.249+0000 7f6244e70700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.961501+0000 front 2024-04-25T19:23:28.961525+0000 (oldest deadline 2024-04-25T19:23:52.461207+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:06.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:06 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:06.137+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:06.552 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:06 smithi183 podman[71903]: 2024-04-25 19:24:06.28671462 +0000 UTC m=+5.511682789 container died f8fe1d8b3a7afa3cc458559b9ac1d0a3597d1b4ce33bfaff69238b07b16cf24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:24:06.916 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:06 smithi183 podman[71903]: 2024-04-25 19:24:06.650797069 +0000 UTC m=+5.875765235 container cleanup f8fe1d8b3a7afa3cc458559b9ac1d0a3597d1b4ce33bfaff69238b07b16cf24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T19:24:06.916 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:06 smithi183 bash[71903]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10 2024-04-25T19:24:07.268 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:06 smithi183 podman[71918]: 2024-04-25 19:24:06.928114127 +0000 UTC m=+0.636426992 container remove f8fe1d8b3a7afa3cc458559b9ac1d0a3597d1b4ce33bfaff69238b07b16cf24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T19:24:07.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:07.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:07.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:07.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:07.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:07.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:07.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:07 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:07.185+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:07.523 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:07 smithi183 podman[72013]: 2024-04-25 19:24:07.170888543 +0000 UTC m=+0.017928254 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:24:07.523 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:07 smithi183 podman[72013]: 2024-04-25 19:24:07.270112601 +0000 UTC m=+0.117152311 container create 62d97a7a6139686493e649ec881b0cc28c5c24314062e07f545254ade8c61540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T19:24:07.523 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:07 smithi183 podman[72013]: 2024-04-25 19:24:07.522683135 +0000 UTC m=+0.369722844 container init 62d97a7a6139686493e649ec881b0cc28c5c24314062e07f545254ade8c61540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T19:24:07.822 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:07 smithi183 podman[72013]: 2024-04-25 19:24:07.525907131 +0000 UTC m=+0.372946842 container start 62d97a7a6139686493e649ec881b0cc28c5c24314062e07f545254ade8c61540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T19:24:07.822 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:07 smithi183 podman[72013]: 2024-04-25 19:24:07.614467903 +0000 UTC m=+0.461507614 container attach 62d97a7a6139686493e649ec881b0cc28c5c24314062e07f545254ade8c61540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:24:08.117 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 25 19:24:07 smithi183 podman[72013]: 2024-04-25 19:24:07.821917178 +0000 UTC m=+0.668956888 container died 62d97a7a6139686493e649ec881b0cc28c5c24314062e07f545254ade8c61540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:24:08.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T19:24:08.436 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:08.437 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:08.437 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:08.437 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:08.437 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:08.437 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:08.437 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.148+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:08.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:08.487 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T19:24:08.488 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T19:24:08.488 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.11 2024-04-25T19:24:08.731 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 systemd[1]: Stopping Ceph osd.11 for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.798+0000 7f8bea3c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.798+0000 7f8bea3c7700 -1 osd.11 125 *** Got signal Terminated *** 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:08 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:08.798+0000 7f8bea3c7700 -1 osd.11 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:09.111 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:09.112 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:09.112 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:09.112 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:09 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:09.110+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:10.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:10 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:10.090+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:11.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:11.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:11.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:11.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:11.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:11.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:11.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:11.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:11 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:11.055+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-04-25T19:23:46.416105+0000 front 2024-04-25T19:23:46.416064+0000 (oldest deadline 2024-04-25T19:24:11.016533+0000) 2024-04-25T19:24:12.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:12.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:12.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:12.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:12.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:12.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:12.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:12.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:12 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:12.018+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-04-25T19:23:46.416105+0000 front 2024-04-25T19:23:46.416064+0000 (oldest deadline 2024-04-25T19:24:11.016533+0000) 2024-04-25T19:24:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-25T19:22:38.106475+0000 front 2024-04-25T19:22:38.106773+0000 (oldest deadline 2024-04-25T19:23:04.006424+0000) 2024-04-25T19:24:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-25T19:22:48.707488+0000 front 2024-04-25T19:22:48.707351+0000 (oldest deadline 2024-04-25T19:23:12.807382+0000) 2024-04-25T19:24:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-04-25T19:22:57.408659+0000 front 2024-04-25T19:22:57.408710+0000 (oldest deadline 2024-04-25T19:23:21.508871+0000) 2024-04-25T19:24:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-25T19:23:13.612178+0000 front 2024-04-25T19:23:13.612032+0000 (oldest deadline 2024-04-25T19:23:38.311901+0000) 2024-04-25T19:24:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-04-25T19:23:22.913052+0000 front 2024-04-25T19:23:22.913100+0000 (oldest deadline 2024-04-25T19:23:48.813259+0000) 2024-04-25T19:24:13.481 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-25T19:23:28.814070+0000 front 2024-04-25T19:23:28.814156+0000 (oldest deadline 2024-04-25T19:23:52.313868+0000) 2024-04-25T19:24:13.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-25T19:23:37.615278+0000 front 2024-04-25T19:23:37.615222+0000 (oldest deadline 2024-04-25T19:24:01.115110+0000) 2024-04-25T19:24:13.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:13 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11[58207]: 2024-04-25T19:24:13.056+0000 7f8be29b3700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-04-25T19:23:46.416105+0000 front 2024-04-25T19:23:46.416064+0000 (oldest deadline 2024-04-25T19:24:11.016533+0000) 2024-04-25T19:24:14.482 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:14 smithi194 podman[72142]: 2024-04-25 19:24:14.219876135 +0000 UTC m=+5.549188832 container died c8e0be7f52a3d8233559dff607f719809db2a64fe2702d4eb88bbf02cc5a5ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T19:24:14.841 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:14 smithi194 podman[72142]: 2024-04-25 19:24:14.629244766 +0000 UTC m=+5.958557469 container cleanup c8e0be7f52a3d8233559dff607f719809db2a64fe2702d4eb88bbf02cc5a5ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:24:14.842 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:14 smithi194 bash[72142]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11 2024-04-25T19:24:15.231 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:14 smithi194 podman[72159]: 2024-04-25 19:24:14.84124668 +0000 UTC m=+0.622747105 container remove c8e0be7f52a3d8233559dff607f719809db2a64fe2702d4eb88bbf02cc5a5ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T19:24:15.569 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:15 smithi194 podman[72250]: 2024-04-25 19:24:15.149851494 +0000 UTC m=+0.018208064 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:24:15.569 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:15 smithi194 podman[72250]: 2024-04-25 19:24:15.250230418 +0000 UTC m=+0.118587030 container create c50e9dca87c632b4e9b7df2978da8f18b9ae5f8c30885d35adfe4038f7e79cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T19:24:15.569 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:15 smithi194 podman[72250]: 2024-04-25 19:24:15.485859109 +0000 UTC m=+0.354215679 container init c50e9dca87c632b4e9b7df2978da8f18b9ae5f8c30885d35adfe4038f7e79cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T19:24:15.570 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:15 smithi194 podman[72250]: 2024-04-25 19:24:15.48899533 +0000 UTC m=+0.357351904 container start c50e9dca87c632b4e9b7df2978da8f18b9ae5f8c30885d35adfe4038f7e79cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:24:15.570 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:15 smithi194 podman[72250]: 2024-04-25 19:24:15.568936288 +0000 UTC m=+0.437292858 container attach c50e9dca87c632b4e9b7df2978da8f18b9ae5f8c30885d35adfe4038f7e79cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T19:24:15.981 INFO:journalctl@ceph.osd.11.smithi194.stdout:Apr 25 19:24:15 smithi194 podman[72250]: 2024-04-25 19:24:15.782932367 +0000 UTC m=+0.651288936 container died c50e9dca87c632b4e9b7df2978da8f18b9ae5f8c30885d35adfe4038f7e79cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T19:24:16.370 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T19:24:16.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:16.411 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T19:24:16.411 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T19:24:16.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.a 2024-04-25T19:24:16.820 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:16 smithi086 systemd[1]: Stopping Ceph mds.a for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:17.319 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:16 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a[67955]: 2024-04-25T19:24:16.981+0000 7f5525e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:17.320 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:16 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a[67955]: 2024-04-25T19:24:16.981+0000 7f5525e00700 -1 mds.a *** got signal Terminated *** 2024-04-25T19:24:20.638 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:20 smithi086 podman[107437]: 2024-04-25 19:24:20.364041164 +0000 UTC m=+3.504077060 container died 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T19:24:21.009 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:20 smithi086 podman[107437]: 2024-04-25 19:24:20.832199278 +0000 UTC m=+3.972235167 container cleanup 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T19:24:21.009 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:20 smithi086 bash[107437]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a 2024-04-25T19:24:21.320 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 25 19:24:21 smithi086 podman[107451]: 2024-04-25 19:24:21.008212724 +0000 UTC m=+0.626488902 container remove 14e2c7e5f3b684407b356ca0cb4d97980f68c3f5d3f8978c5beedfe2d47a0e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T19:24:21.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T19:24:21.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:21.603 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T19:24:21.603 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T19:24:21.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.d 2024-04-25T19:24:21.820 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:21 smithi086 systemd[1]: Stopping Ceph mds.d for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:22.570 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d[66682]: 2024-04-25T19:24:22.157+0000 7f5a89000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:22.570 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:22 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d[66682]: 2024-04-25T19:24:22.157+0000 7f5a89000700 -1 mds.d *** got signal Terminated *** 2024-04-25T19:24:24.217 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:23 smithi086 podman[107557]: 2024-04-25 19:24:23.941787476 +0000 UTC m=+1.909400589 container died 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T19:24:24.479 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:24 smithi086 podman[107557]: 2024-04-25 19:24:24.327120751 +0000 UTC m=+2.294733890 container cleanup 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:24:24.479 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:24 smithi086 bash[107557]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d 2024-04-25T19:24:24.820 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 25 19:24:24 smithi086 podman[107571]: 2024-04-25 19:24:24.478015935 +0000 UTC m=+0.522052174 container remove 3f9e16f472386b2357043e101accf6597f94e87ccc34370599b8d2445dc81044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:24:24.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T19:24:25.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:25.085 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T19:24:25.085 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T19:24:25.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.g 2024-04-25T19:24:25.570 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:25 smithi086 systemd[1]: Stopping Ceph mds.g for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:26.070 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g[67513]: 2024-04-25T19:24:25.719+0000 7fdbeec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:26.070 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:25 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g[67513]: 2024-04-25T19:24:25.719+0000 7fdbeec00700 -1 mds.g *** got signal Terminated *** 2024-04-25T19:24:28.294 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:27 smithi086 podman[107677]: 2024-04-25 19:24:27.936591392 +0000 UTC m=+2.347137289 container stop 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T19:24:28.295 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:27 smithi086 podman[107677]: 2024-04-25 19:24:27.96601977 +0000 UTC m=+2.376565668 container died 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T19:24:28.570 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:28 smithi086 podman[107677]: 2024-04-25 19:24:28.405967938 +0000 UTC m=+2.816513860 container cleanup 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T19:24:28.570 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:28 smithi086 bash[107677]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g 2024-04-25T19:24:29.070 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 25 19:24:28 smithi086 podman[107691]: 2024-04-25 19:24:28.573305029 +0000 UTC m=+0.617142875 container remove 5e4f719ae3d2fb7f064d5c5c09860e07701e52ecaec3dcf8531c87ed5dbaf023 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T19:24:29.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T19:24:29.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:29.215 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T19:24:29.215 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T19:24:29.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.j 2024-04-25T19:24:29.751 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:29 smithi086 systemd[1]: Stopping Ceph mds.j for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:29.751 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j[66269]: 2024-04-25T19:24:29.749+0000 7fb905e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:29.752 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:29 smithi086 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j[66269]: 2024-04-25T19:24:29.749+0000 7fb905e00700 -1 mds.j *** got signal Terminated *** 2024-04-25T19:24:34.038 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:33 smithi086 podman[107797]: 2024-04-25 19:24:33.76660468 +0000 UTC m=+4.138940711 container died d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T19:24:34.320 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:34 smithi086 podman[107797]: 2024-04-25 19:24:34.225170809 +0000 UTC m=+4.597506834 container cleanup d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T19:24:34.320 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:34 smithi086 bash[107797]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j 2024-04-25T19:24:34.810 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 25 19:24:34 smithi086 podman[107823]: 2024-04-25 19:24:34.418038751 +0000 UTC m=+0.634590936 container remove d1c7615f0661f6f140defae297156491986c60ad8f9b6a778fb23dd628942a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:24:34.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T19:24:34.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:34.939 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T19:24:34.939 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T19:24:34.939 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.b 2024-04-25T19:24:35.235 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:34 smithi183 systemd[1]: Stopping Ceph mds.b for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:35.500 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: 2024-04-25T19:24:35.233+0000 7f8720eb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:35.500 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:35 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b[63001]: 2024-04-25T19:24:35.233+0000 7f8720eb5700 -1 mds.b *** got signal Terminated *** 2024-04-25T19:24:39.534 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:39 smithi183 podman[72118]: 2024-04-25 19:24:39.248628765 +0000 UTC m=+4.119852018 container died 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T19:24:39.868 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:39 smithi183 podman[72118]: 2024-04-25 19:24:39.675054879 +0000 UTC m=+4.546278140 container cleanup 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:24:39.868 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:39 smithi183 bash[72118]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b 2024-04-25T19:24:40.104 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T19:24:40.125 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:39 smithi183 podman[72144]: 2024-04-25 19:24:39.867758315 +0000 UTC m=+0.618449420 container remove 9ff187b36c4e3cd18835f21ce95d1c540a6db74d670b7b1ced346574e03058be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:24:40.126 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:40 smithi183 systemd[1]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-25T19:24:40.126 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:40 smithi183 systemd[1]: Stopped Ceph mds.b for 1ec2eb56-0334-11ef-bc93-c7b262605968. 2024-04-25T19:24:40.126 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 25 19:24:40 smithi183 systemd[1]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.b.service: Consumed 15.980s CPU time. 2024-04-25T19:24:40.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:40.140 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T19:24:40.140 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T19:24:40.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.e 2024-04-25T19:24:40.416 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:24:40 smithi183 systemd[1]: Stopping Ceph mds.e for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:40.916 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:24:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e[62366]: 2024-04-25T19:24:40.478+0000 7fca016b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:40.916 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:24:40 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e[62366]: 2024-04-25T19:24:40.478+0000 7fca016b8700 -1 mds.e *** got signal Terminated *** 2024-04-25T19:24:44.166 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:24:43 smithi183 podman[72251]: 2024-04-25 19:24:43.851362962 +0000 UTC m=+3.486090891 container died aa9aefaa080eea85797836029351e47a7e6d1654cbe94ab2b319a5e5dc4af66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T19:24:44.580 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:24:44 smithi183 podman[72251]: 2024-04-25 19:24:44.336819551 +0000 UTC m=+3.971547480 container cleanup aa9aefaa080eea85797836029351e47a7e6d1654cbe94ab2b319a5e5dc4af66e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T19:24:44.580 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 25 19:24:44 smithi183 bash[72251]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-e 2024-04-25T19:24:44.691 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T19:24:44.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:44.723 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T19:24:44.723 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T19:24:44.723 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.h 2024-04-25T19:24:44.916 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:24:44 smithi183 systemd[1]: Stopping Ceph mds.h for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:45.416 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:24:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h[62690]: 2024-04-25T19:24:45.049+0000 7fce0ee86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:45.416 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:24:45 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h[62690]: 2024-04-25T19:24:45.049+0000 7fce0ee86700 -1 mds.h *** got signal Terminated *** 2024-04-25T19:24:48.166 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:24:47 smithi183 podman[72374]: 2024-04-25 19:24:47.845461219 +0000 UTC m=+2.905342151 container died 877b444a2ecd639068269df37da812ce3d50821c8a6f14d579ad458c53e81824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T19:24:48.484 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:24:48 smithi183 podman[72374]: 2024-04-25 19:24:48.265784416 +0000 UTC m=+3.325665349 container cleanup 877b444a2ecd639068269df37da812ce3d50821c8a6f14d579ad458c53e81824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T19:24:48.484 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 25 19:24:48 smithi183 bash[72374]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-h 2024-04-25T19:24:48.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T19:24:48.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:48.730 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T19:24:48.730 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T19:24:48.730 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.k 2024-04-25T19:24:49.053 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:24:48 smithi183 systemd[1]: Stopping Ceph mds.k for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:49.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:24:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k[62046]: 2024-04-25T19:24:49.052+0000 7f83c4519700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:49.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:24:49 smithi183 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k[62046]: 2024-04-25T19:24:49.052+0000 7f83c4519700 -1 mds.k *** got signal Terminated *** 2024-04-25T19:24:52.120 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:24:51 smithi183 podman[72495]: 2024-04-25 19:24:51.849111453 +0000 UTC m=+2.904933743 container died 99f24bcc8c826775f8d7be606e1d1ed31654b995ca53ddd9ecaa205f446d2762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T19:24:52.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:24:52 smithi183 podman[72495]: 2024-04-25 19:24:52.202464564 +0000 UTC m=+3.258286860 container cleanup 99f24bcc8c826775f8d7be606e1d1ed31654b995ca53ddd9ecaa205f446d2762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T19:24:52.416 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 25 19:24:52 smithi183 bash[72495]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-k 2024-04-25T19:24:52.674 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T19:24:52.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:52.705 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T19:24:52.705 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T19:24:52.705 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.c 2024-04-25T19:24:53.012 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:24:52 smithi194 systemd[1]: Stopping Ceph mds.c for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:53.481 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:24:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c[63625]: 2024-04-25T19:24:53.011+0000 7fc9512bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:53.481 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:24:53 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c[63625]: 2024-04-25T19:24:53.011+0000 7fc9512bf700 -1 mds.c *** got signal Terminated *** 2024-04-25T19:24:56.158 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:24:55 smithi194 podman[72357]: 2024-04-25 19:24:55.84089609 +0000 UTC m=+2.939465443 container died 73eed52809e28ffffec198b4f69700dcf6fb59f99c004aa4055bd036e6d6e408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:24:56.481 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:24:56 smithi194 podman[72357]: 2024-04-25 19:24:56.236357836 +0000 UTC m=+3.334927185 container cleanup 73eed52809e28ffffec198b4f69700dcf6fb59f99c004aa4055bd036e6d6e408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T19:24:56.481 INFO:journalctl@ceph.mds.c.smithi194.stdout:Apr 25 19:24:56 smithi194 bash[72357]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-c 2024-04-25T19:24:56.659 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T19:24:56.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:24:56.694 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T19:24:56.694 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T19:24:56.694 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.f 2024-04-25T19:24:57.231 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:24:56 smithi194 systemd[1]: Stopping Ceph mds.f for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:24:57.231 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:24:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f[63314]: 2024-04-25T19:24:57.014+0000 7f66a7b08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:24:57.231 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:24:57 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f[63314]: 2024-04-25T19:24:57.014+0000 7f66a7b08700 -1 mds.f *** got signal Terminated *** 2024-04-25T19:25:00.133 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:24:59 smithi194 podman[72490]: 2024-04-25 19:24:59.856806893 +0000 UTC m=+2.944966918 container died aebf8b3de1cc76722292ab3eb9fb73dbb56f7f17103caa3df55ca3167cbbb43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T19:25:00.481 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:25:00 smithi194 podman[72490]: 2024-04-25 19:25:00.219727116 +0000 UTC m=+3.307887138 container cleanup aebf8b3de1cc76722292ab3eb9fb73dbb56f7f17103caa3df55ca3167cbbb43c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T19:25:00.481 INFO:journalctl@ceph.mds.f.smithi194.stdout:Apr 25 19:25:00 smithi194 bash[72490]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-f 2024-04-25T19:25:00.644 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T19:25:00.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:25:00.677 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T19:25:00.677 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T19:25:00.677 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.i 2024-04-25T19:25:00.981 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:25:00 smithi194 systemd[1]: Stopping Ceph mds.i for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:25:01.481 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:25:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i[63932]: 2024-04-25T19:25:01.009+0000 7fa4659f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:25:01.481 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:25:01 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i[63932]: 2024-04-25T19:25:01.009+0000 7fa4659f0700 -1 mds.i *** got signal Terminated *** 2024-04-25T19:25:04.127 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:25:03 smithi194 podman[72611]: 2024-04-25 19:25:03.848617296 +0000 UTC m=+2.961914084 container died 68818f808e5f733eac275a9ea466b65d85f20db291b63a63422c29cc8aea15bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T19:25:04.471 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:25:04 smithi194 podman[72611]: 2024-04-25 19:25:04.226297541 +0000 UTC m=+3.339594329 container cleanup 68818f808e5f733eac275a9ea466b65d85f20db291b63a63422c29cc8aea15bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:25:04.471 INFO:journalctl@ceph.mds.i.smithi194.stdout:Apr 25 19:25:04 smithi194 bash[72611]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-i 2024-04-25T19:25:04.655 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T19:25:04.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:25:04.687 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T19:25:04.687 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T19:25:04.687 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.l 2024-04-25T19:25:05.005 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:25:04 smithi194 systemd[1]: Stopping Ceph mds.l for 1ec2eb56-0334-11ef-bc93-c7b262605968... 2024-04-25T19:25:05.481 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:25:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l[63004]: 2024-04-25T19:25:05.003+0000 7f4a3b77a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:25:05.481 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:25:05 smithi194 ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l[63004]: 2024-04-25T19:25:05.003+0000 7f4a3b77a700 -1 mds.l *** got signal Terminated *** 2024-04-25T19:25:08.130 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:25:07 smithi194 podman[72733]: 2024-04-25 19:25:07.832745135 +0000 UTC m=+2.935157474 container died bcbf93062965d05ab74ec785373f3975bcab9d7ca61fc8eb3e104ce28d5bc41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T19:25:08.455 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:25:08 smithi194 podman[72733]: 2024-04-25 19:25:08.229370527 +0000 UTC m=+3.331782854 container cleanup bcbf93062965d05ab74ec785373f3975bcab9d7ca61fc8eb3e104ce28d5bc41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T19:25:08.455 INFO:journalctl@ceph.mds.l.smithi194.stdout:Apr 25 19:25:08 smithi194 bash[72733]: ceph-1ec2eb56-0334-11ef-bc93-c7b262605968-mds-l 2024-04-25T19:25:08.605 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ec2eb56-0334-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T19:25:08.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:25:08.637 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T19:25:08.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:25:08.954 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:25:13.613 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:25:13.804 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:25:16.221 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:25:16.409 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:25:18.728 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:25:18.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:25:18.843 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:25:18.869 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T19:25:18.871 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi086/crash 2024-04-25T19:25:18.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:25:18.970 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:25:18.970 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:25:18.975 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi183/crash 2024-04-25T19:25:18.977 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:25:19.017 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:25:19.017 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:25:19.019 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi194/crash 2024-04-25T19:25:19.020 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:25:19.060 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:25:19.060 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:25:19.061 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T19:25:19.061 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T19:25:19.163 INFO:tasks.cephadm:Compressing logs... 2024-04-25T19:25:19.164 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:25:19.168 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:25:19.170 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:25:19.193 INFO:teuthology.orchestra.run.smithi194.stderr:find: gzip -5‘/var/log/rbd-target-api’ --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:25:19.193 INFO:teuthology.orchestra.run.smithi194.stderr:: No such file or directory 2024-04-25T19:25:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T19:25:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T19:25:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:25:19.195 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T19:25:19.195 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T19:25:19.195 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T19:25:19.195 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T19:25:19.196 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T19:25:19.198 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.6% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:25:19.198 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:25:19.198 INFO:teuthology.orchestra.run.smithi194.stderr: 90.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:25:19.202 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:25:19.202 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose --/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.h.log: 87.7% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log.gz 90.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:25:19.203 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:25:19.207 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.i.log: 92.5% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T19:25:19.207 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T19:25:19.207 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T19:25:19.208 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log: /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.11.log: 83.3% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:25:19.208 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:25:19.209 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T19:25:19.218 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T19:25:19.220 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T19:25:19.222 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:25:19.222 INFO:teuthology.orchestra.run.smithi086.stderr:: No such file or directory 2024-04-25T19:25:19.223 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:25:19.223 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T19:25:19.224 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T19:25:19.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T19:25:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T19:25:19.227 INFO:teuthology.orchestra.run.smithi086.stderr: 90.2% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:25:19.227 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:25:19.228 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T19:25:19.228 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T19:25:19.230 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:25:19.230 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:25:19.230 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T19:25:19.231 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T19:25:19.232 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:25:19.234 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T19:25:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:25:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T19:25:19.237 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:25:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.a.log: /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T19:25:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T19:25:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T19:25:19.247 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:25:19.247 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:25:19.249 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:25:19.253 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:25:19.265 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:25:19.269 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:25:19.280 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:25:19.294 INFO:teuthology.orchestra.run.smithi194.stderr: 94.8% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T19:25:19.329 INFO:teuthology.orchestra.run.smithi194.stderr: 93.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T19:25:19.330 INFO:teuthology.orchestra.run.smithi194.stderr: 93.1% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T19:25:19.334 INFO:teuthology.orchestra.run.smithi183.stderr: 94.7% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T19:25:19.336 INFO:teuthology.orchestra.run.smithi183.stderr: 93.2% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T19:25:19.367 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T19:25:19.370 INFO:teuthology.orchestra.run.smithi194.stderr: 94.6% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T19:25:19.376 INFO:teuthology.orchestra.run.smithi086.stderr: 94.5% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T19:25:19.593 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T19:25:19.775 INFO:teuthology.orchestra.run.smithi194.stderr: 91.7% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T19:25:19.780 INFO:teuthology.orchestra.run.smithi086.stderr: 90.1% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T19:25:20.017 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T19:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T19:25:20.540 INFO:teuthology.orchestra.run.smithi183.stderr: 91.4% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T19:25:20.564 INFO:teuthology.orchestra.run.smithi194.stderr: 93.0% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T19:25:20.712 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T19:25:20.794 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T19:25:20.893 INFO:teuthology.orchestra.run.smithi194.stderr: 91.8% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T19:25:20.959 INFO:teuthology.orchestra.run.smithi183.stderr: 91.7% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T19:25:21.207 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T19:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr: 92.6% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T19:25:21.311 INFO:teuthology.orchestra.run.smithi086.stderr: 92.5% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T19:25:21.315 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-25T19:25:21.315 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m2.116s 2024-04-25T19:25:21.315 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m9.067s 2024-04-25T19:25:21.316 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.786s 2024-04-25T19:25:21.408 INFO:teuthology.orchestra.run.smithi194.stderr: 92.3% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T19:25:21.438 INFO:teuthology.orchestra.run.smithi183.stderr: 92.9% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T19:25:21.544 INFO:teuthology.orchestra.run.smithi194.stderr: 91.6% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T19:25:21.546 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-04-25T19:25:21.547 INFO:teuthology.orchestra.run.smithi194.stderr:real 0m2.362s 2024-04-25T19:25:21.547 INFO:teuthology.orchestra.run.smithi194.stderr:user 0m8.359s 2024-04-25T19:25:21.547 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m0.298s 2024-04-25T19:25:22.134 INFO:teuthology.orchestra.run.smithi183.stderr: 92.3% -- replaced with /var/log/ceph/1ec2eb56-0334-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T19:25:22.135 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-04-25T19:25:22.136 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m2.951s 2024-04-25T19:25:22.136 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m10.799s 2024-04-25T19:25:22.136 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.357s 2024-04-25T19:25:22.136 INFO:tasks.cephadm:Archiving logs... 2024-04-25T19:25:22.136 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi086/log 2024-04-25T19:25:22.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:25:23.523 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi183/log 2024-04-25T19:25:23.524 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:25:25.079 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi194/log 2024-04-25T19:25:25.080 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:25:26.397 INFO:tasks.cephadm:Removing cluster... 2024-04-25T19:25:26.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --force 2024-04-25T19:25:26.713 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:25:27.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --force 2024-04-25T19:25:27.563 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:25:27.827 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ec2eb56-0334-11ef-bc93-c7b262605968 --force 2024-04-25T19:25:28.018 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 1ec2eb56-0334-11ef-bc93-c7b262605968 2024-04-25T19:25:28.284 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T19:25:28.284 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:25:28.342 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:25:28.358 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:25:28.373 INFO:tasks.cephadm:Teardown complete 2024-04-25T19:25:28.374 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T19:25:28.387 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T19:25:28.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:25:28.395 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:25:28.401 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:25:28.558 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T19:25:28.570 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T19:25:28.570 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:25:28.578 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:25:28.581 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:25:28.593 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:25:28.594 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:25:28.598 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:25:28.598 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-25T19:25:28.598 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1097us[+1094us] +/- 62ms 2024-04-25T19:25:28.598 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -1336us[-1336us] +/- 53ms 2024-04-25T19:25:28.598 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +342us[ +342us] +/- 62ms 2024-04-25T19:25:28.598 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:25:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:25:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-04-25T19:25:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1099us[+1122us] +/- 62ms 2024-04-25T19:25:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 39 -1312us[-1312us] +/- 53ms 2024-04-25T19:25:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +330us[ +353us] +/- 62ms 2024-04-25T19:25:28.599 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:25:28.625 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:25:28.634 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:25:28.636 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-25T19:25:28.636 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 8 377 104 +1053us[ +950us] +/- 62ms 2024-04-25T19:25:28.636 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 101 -1381us[-1381us] +/- 53ms 2024-04-25T19:25:28.636 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 100 +303us[ +303us] +/- 95ms 2024-04-25T19:25:28.636 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:25:28.637 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T19:25:28.648 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T19:25:28.649 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T19:25:28.693 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:25:28.913 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:25:28.983 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:25:29.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:25:29.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:25:29.142 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:25:29.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:25:29.171 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:25:29.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:25:29.198 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T19:25:29.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T19:25:29.237 INFO:teuthology.task.internal:Duration was 2806.353215 seconds 2024-04-25T19:25:29.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T19:25:29.248 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T19:25:29.248 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:25:29.256 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:25:29.259 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:25:29.294 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:25:29.294 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:25:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:25:29.768 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T19:25:29.768 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T19:25:29.768 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:25:29.845 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-04-25T19:25:29.846 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:25:29.869 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-04-25T19:25:29.869 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:25:29.898 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T19:25:29.898 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:25:29.905 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:25:29.911 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:25:29.989 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T19:25:29.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:25:29.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:25:29.997 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:25:30.961 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T19:25:30.973 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T19:25:30.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:25:31.047 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:25:31.073 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:25:31.100 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T19:25:31.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:25:31.181 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:25:31.183 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:25:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-04-25T19:25:31.207 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-04-25T19:25:31.273 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-25T19:25:31.316 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:25:31.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:25:31.363 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:25:31.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:25:31.380 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:25:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:25:31.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T19:25:31.408 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T19:25:31.409 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi086 2024-04-25T19:25:31.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:25:31.527 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi183 2024-04-25T19:25:31.527 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:25:31.567 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673619/remote/smithi194 2024-04-25T19:25:31.567 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:25:31.604 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T19:25:31.604 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:25:31.612 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:25:31.614 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:25:31.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T19:25:31.676 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T19:25:31.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T19:25:31.706 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T19:25:31.706 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:25:31.713 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:25:31.716 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:25:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:25 /home/ubuntu/cephtest 2024-04-25T19:25:31.731 INFO:teuthology.orchestra.run.smithi194.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:25 /home/ubuntu/cephtest 2024-04-25T19:25:31.763 INFO:teuthology.orchestra.run.smithi086.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:25 /home/ubuntu/cephtest 2024-04-25T19:25:31.766 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T19:25:31.779 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T19:25:31.882 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2806.3532149791718 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-25T19:25:31.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T19:25:31.972 INFO:teuthology.run:pass